Deirdre Duffy

Evaluation and governing in the 21^st Century: Disciplinary measures, transformative possibilities.

Palgrave Macmillan 2017

£49.99 (hardback) £39.99 (e-book)

 

It is generally agreed in and beyond the youth sector that policy, programmes and services ought to be ‘evidence-based’. Yet, as we have learned from critical theorists –Paulo Friere, bell hooks, Michel Foucault, Judith Butler and many others –it is the most prominent and taken-for-granted ideas that most need to be subjected to critical questioning and reflection, so that we can understand, make visible and perhaps challenge the role they play in our lives. This is the task that Deirdre Duffy takes on in her excellent book,Evaluation and governing in the 21^st Century, in which she applies critical scrutiny to the disciplinary role played by evaluation in governance,and asks how evaluation can instead be used as a radical means of transformation.

The book takes evaluation and evidence-based policy as its central focus, proposing that evaluation has become a technology of government – in other words, evaluation has come to constitute a collection of practices, strategies, techniques and mechanisms for governing in the public sector and civil society.In this endeavour, the book uses youth work as its central case study, alongside a less prominent focus on evaluation in higher education. Duffy takes a Foucault-inspired genealogical approach. This involves critiquing the present through a historically informed questioning of the dominant narratives surrounding evaluation and evidence-based policy, asking: how did evaluation became so prominent in policy at this particular time? What were the social conditions for its emergence and proliferation? And why does it matter?

One of the strengths of the book is its detailed engagement with policy texts – relating both to evidence-based policy in general, and to youth work specifically, investigating how evaluation has played out in youth policy since the Albemarle Report in 1960. As Duffy points out, this seminal text was itself a form of policy evaluation – but evaluation here was understood as the gathering of different perspectives on the potential value of youth work, used as the basis of planning. Today, in contrast, evaluation tends to be defined more narrowly,often as the collection of quantitative evidence of individual change attributable to a particular intervention. This approach to evaluation in youth work is characteristic of – and enables – neoliberal marketisation and managerialism, in which measurement enables comparability and competition (de St Croix, 2018). It also creates the conditions for modes of commissioning and delivery that are founded on ‘late neoliberal’ financialised notions of investment and return (McGimpsey, 2018).

While measurement shares much of the processes and logic of neoliberalism and finance capital, Duffy develops a sophisticated argument that avoids presenting evaluation simply as an effect of neoliberalism. The proliferation of measurement is also enabled by technological and communication developments, and by an emphasis on scientific theories of knowledge. I found the focus on epistemology (understandings of knowledge) particularly useful; in my experience, the supporters of impact measurement in youth work are rarely motivated by a belief in neoliberalism, but rather by a passion for science, facts, and evidence. Even if the current dominant evaluation mechanisms have a symbiotic relationship with neoliberal governance and management,it is nevertheless important to acknowledge that a growth in policy and funding based on particular approaches to impact measurement cannot be traced to neoliberalism alone.

This is pertinent in relation to the contemporary concept of ‘post-truth’. While powerful policy-makers justify their positions and policies on the basis of faulty evidence or even a rejection of evidence (an obvious example here is Donald Trump’s disbelief in climate change and its effects – e.g. see BBC, 2018), a progressive argument in favour of‘facts’ is understandable.Many supporters of evidence-based policy and practice have a principled faith in the value of science and evidence to counter socially unjust, ineffective, harmful and undemocratic policy and practice. Yet, as Duffy argues, this constructs a conflict between extremist misrepresentation and evidence-based policy (the latter presented as neutral, rational and therefore almost unarguable). Thus, any criticisms and questions around current trends in impact measurement might be dismissed as being overly ideological, irrational and unnecessarily disputatious.

Post-structuralism and post-modernism(whatever their limitations) have contributed a great deal to analysis of how knowledge is constructed, to the questioning of taken-for-granted truths, and thus to the challenging of hegemonic forms of knowledge. In drawing on theoretical contributions such as Foucault’s panoptican and Deleuze’s control society, Duffy argues that evaluation policy enables the exercise of government power through the manipulation of emotions, in which policy subjects are cultivated, distributed and serialised.

While Duffy’s study is a theoretical contribution and does not include empirical research, her arguments resonate with my research with part-time and volunteer youth workers in England, undertaken between 2011 and 2014. At that time, evaluation was dominated by the meeting of abstract outcomes in ways that often felt intrusive and onerous, and this had tangible effects on the everyday work of youth workers, their identities and feelings, and their relationships with young people. Their experience of working towards targets was both emotional and embodied – they reported feeling ‘conflicted’, ‘burnt out’ and ‘sick’. One youth worker spoke of sitting in a management meeting feeling ‘like you were sat in the headmaster’s office while he was wielding a bloody cane or whatever’ (de St Croix, 2016, p.103). Another said ‘it gives me a stomach ache… I’m absolutely exhausted and low…’ (p.104). That both of these practitioners were seen by their managers as particularly effective and successful did not insulate them from the disciplinary effects of outcome-based monitoring.

As someone who can follow theoretical arguments more readily when they relate to such specific instances in people’s everyday lives, I have to admit that I did not find this book an easy read. Yet this is not intended to dissuade anyone from making the effort. In our busy, high-tech, hyper-productive world, where ‘TL-DR’ (too long, didn’t read) has entered the social media lexicon, it is difficult and increasingly unfamiliar to find time and space for slow and challenging reading. While the discussion in this book may have been a little theoretically intense at times, it was good to be challenged.

Overall, Deirdre Duffy’s book is a theoretically rigorous and original text, which has the potential to make a significant contribution to research on evaluation and its role in policy, both within and beyond youth work. Perhaps it may be released in a more affordable paperback edition in due course; otherwise, it appears to be available at a discounted price if you wish to acquire a copy, or ask your library to order one. The widespread changes to evaluation policy and practice in recent years alongside the current ‘post-truth’ context make this a timely and important piece of research.

Finally,can evaluation lead to ‘transformative possibilities’, as noted in the book’s title?I want to say yes… I have gained a great deal from engaging in creative and deliberative evaluation with young people and colleagues, most recently as a facilitator of In Defence of Youth Work storytelling workshops (see story-tellinginyouthwork.org). These workshops involve bringing practitioners and/or young people together in dialogue to reflect critically on ‘real life’ stories from youth work,discussing what these tell us about the impact of youth work and the constraints affecting this practice. Practitioners tell us that such opportunities to evaluate collectively and critically are all too rare, as evaluation has increasingly become a site of regulation.

Duffy suggests that there is potential to reclaim evaluation, including measurement and numbers, as a way of enlivening political debate and working towards social change.This would require a radically more open conceptualisation of evaluation, avoiding pre-determined outcomes:

As soon as a possible future is imagined it becomes regulatory. My key point is that, through evaluating in an open, ambivalent sense rather than a closed, deterministic sense, evaluation begins to move from being solely a disciplinary measure to a transformative potential. (Duffy, 2017, p.139)

Evaluation, like every other aspect of life and work, exists within a wider political and ideological context; such a transformative approach would therefore be almost impossible in a context where youth work feels besieged by funding cuts and redundancies, competitive contracts and commissioning, course closures and precarious employment prospects. Any approach to evaluation is likely to have uncertain results at a time when the youth work sector has lost a great deal of its confidence and resources. Whether or not the tide turns in favour of critical and democratic approaches to youth work (IDYW, 2018), it is important to think and talk anew about how evaluation can best serve young people, youth workers, communities, and the practice and profession of youth work. This book provides theoretical resources to help us with this task.