Towards a balanced scorecard: A critical analysis of the Culture and Sport Evidence (CASE) programme Dr Ben Walmsley, School of Performance and Cultural Industries, University of Leeds Abstract This article provides a critical analysis of the methods employed in the Culture and Sport Evidence (CASE) programme. Based on a comprehensive review of the arts management and cultural policy literature, it contests recent claims that the cultural sector should state its value in the economic language of policy appraisal and evaluation (O’Brien, 2010) and proposes alternative methods for evaluating the drivers, impact and value of engagement in the arts, including the balanced scorecard approach. The literature identifies a number of fundamental problems in quantifying the social and personal impact of the arts, and an underlying policy issue is that the arts have become increasingly subject to the benchmarks of incompatible disciplines and practices. This paper seeks to redress the balance by questioning the argument that economic cost benefit analysis is the best way to understand cultural value and influence public policy. As the CASE programme aimed to make the business case for optimum Government investment in sport and culture, it adopted the framework set out in HM Treasury’s Green Book and took a quantitative, evidence-based approach to measuring the drivers, impact and instrumental value of engagement, disregarding established qualitative studies and approaches, which have been shown to articulate cultural value through a more personal, intrinsic and holistic lens. This article makes the case for a more balanced approach to cultural evaluation and a more holistic articulation of cultural value, which would combine intrinsic and instrumental benefits and comprise both qualitative and quantitative methods. The key implication of this re-conception of value is that cultural policy should be evaluated not on return on investment but rather against a balanced range of objectives and articulated in a language that reflects artistic practice and speaks directly to existing and potential audiences. Keywords: cultural policy; impact of the arts; wellbeing; public value; audiences; cost benefit analysis. Context Debate about the drivers, impact and value of engagement in culture can be traced back to Aristotle and Plato. As Vuyk (2010, p. 177) reminds us: “The history of philosophy is full of ideas about the positive and negative effects of the arts on both the individual and society”. From Kant’s focus on aesthetic pleasure and Schopenhauer’s view of the arts as a spiritual refuge to the Romantics’ artistic representation of human emotions and natural beauty, the nature of the relationship between the arts and the human condition has remained contested. In the modern and post-modern eras, the arts have been identified with a vast range of private and public goals, including reflection, representation, solace, outrage, escapism, protest and celebration. Today, in the midst of one of the deepest global financial crises ever known, it is argued that the arts and culture need to argue their case better than ever before (EPPI Centre, 2010; Jennings, 2012; O'Brien, 2012). In his report to the Department for Culture, Media and Sport, for example, O’Brien (2010, p. 4) charges that “the cultural sector will need to use the tools and concepts of economics to fully state their benefits in the prevailing language of policy appraisal and evaluation”. The aim of this paper is to challenge this assertion by providing a critical commentary of the evaluation methods employed by the Culture and Sport Evidence (CASE) programme and to propose an alternative, more balanced approach to evaluating the drivers, impact and value of public engagement in the arts. It has been argued that evaluation is “fundamentally about values […] not an abstract, quasiscientific process through which objective truths can be identified” (Matarasso, 1996, p. 2). This definition frames the critical stance taken in this paper, which will begin with a critical exploration of the role and purpose of artistic evaluation before moving on to critique the instrumentalist approach to evaluation taken by the CASE programme. The role of evaluation: who’s the audience? Vuyk (2010, p. 178) contends that cultural policy evaluation often fails to reflect artistic values: “Current instrumental cultural policies, fed by a neoliberal political climate, regard the arts mainly as production systems within a consumption society that have to be managed in such a way that they yield contented citizens. Artists act against this image”. Artists are often forgotten in debates about the impact and value of their work, as indeed they were in the CASE programme, where the focus lay entirely on participants. This biased perspective confirms Glinkowski’s (2012) contention that artists continue to lack visibility in the cultural sphere which in turn limits their influence on cultural policy. It also highlights a more general issue with evaluation in the arts and cultural sector: namely that its audience and purpose is often unclear and conflicts with the objectives of the activity being evaluated. A good example of this is Arts Council England’s artistic evaluation policy, which requires assessors to focus not just on artistic quality and impact, but also on innovation and artistic development. However, the macro evaluation expected from DCMS by HM Treasury needs to justify value for money in the form of public utility, which is measured in terms of “consumption satisfaction” and by any tangible change in participants’ welfare or well-being (Fujiwara & Campbell, 2011). There is no room in utilitarian economic evaluation for artistic values such as risk, innovation and development, and this mismatch in expectations highlights the reality that there are “fundamental tensions and contradictions inherent in the strategic convergence of the social, the cultural […] and the economic” (Stevenson, Rowe & McKay, 2010, p. 249). These contradictions have significant implications for cultural policy, not least for the organizations and audiences whose activities are subsidised through it, and it seems that at the heart of these problems lie questions of ownership and power: “The important, and essentially political, question about evaluation is which value system is used to provide benchmarks against which work will be measured – in other words, who defines value” (Matarasso, 1996, p. 2). Or, as Scott (2010, p. 276) puts it: “At its core, Public Value focuses attention on the on-going and contested questions of ‘whose values’ and what values provide the benchmarks against which the worth of arts and cultural heritage is assessed and measured.” The CASE programme The CASE programme was a three-year research project established in 2008 to inform policy development in culture and sport. Its aim was to understand the drivers, impact and value of public engagement in culture and sport and it took an instrumentalist approach to evaluate these, which focussed on social utility. As the programme was intended to make the business case for optimum Government investment in sport and culture (EPPI Centre, 2010), it adopted the frameworks set out in HM Treasury’s Green Book and Magenta Book and therefore took a quantitative approach. However, considering that the Magenta Book advocates tailoring evaluations to “the type of policy being considered and the types of questions it is hoped to answer” (HM Treasury, 2011, p. 11), it could be argued that this approach was misguided from the outset, if indeed the programme really hoped to develop understanding of cultural engagement. The programme claimed to adopt “interdisciplinary research methods and analysis” (EPPI Centre, 2010, p. 4) and set out to explore “new methods for determining the value of engagement that go beyond standard economic/monetary valuation” (p. 5). One of the programme’s primary objectives was to increase public participation in culture and sport, and its premise was one of cultural deficit, based on the assertion that many non-participants simply lacked the awareness, interest, know-how or time to engage in cultural activities. Correspondingly, it aimed to assess return on investment in terms of increase in (rather than depth of) public engagement. Another fundamental problem with the programme is that it disregarded the vast range of qualitative methods that have informed both practice and research on the drivers, benefits and value of public engagement in the arts in recent years. Qualitative studies are widely credited with the potential to provide rich, nuanced and context-dependent analysis (Rubin & Rubin, 2005), so to ignore their insights when exploring complex questions of engagement appears to be a serious methodological misjudgement. A clue to the reasoning behind this decision appears on page 14 of the report, where the literature on engagement drivers is reduced to the fields of economics and sociology, with no mention of cultural policy, marketing, consumer behaviour or psychology, and where the majority of even this limited field is discredited as being of “limited policy relevance”. The ensuing statistical analysis of engagement factors proceeds to fuse cultural with sporting engagement and then segment participants by purely geo-demographic variables, disregarding intrinsic drivers and the more sophisticated insights provided by behavioural and psychographic segmentation. Unsurprisingly, then, the key findings of the report are somewhat limited. We learn, for example, that as they age, people engage more with culture and less with sport; that those with higher education levels engage more with culture; and that while men are more likely to engage in sport, they are less disposed to cultural engagement than women: so nothing new there. The recommendation is to target those less likely to engage, which defies recent marketing and audience development theories regarding return on investment and again reaffirms the discredited cultural deficit model. The evaluative approach of the programme was also hampered by its scope. The programme focused predominantly on the instrumental benefits of sport and culture for young people and on the long-term health benefits of participation, rather than exploring more intrinsic or holistic aspects of value and wellbeing. As the ‘Impacts of engagement’ strand only evaluated attendance at museums, galleries and heritage sites, it is of limited use to practitioners, researchers and policy makers exploring the impact of the performing arts on adults (which accounts for the vast majority of public spending on the arts). The final strand of the programme explored the value of engagement and measured value in a simplistic equation combining short-term subjective wellbeing (SWB) with long-term health. Disregarding established models on the benefits of the arts (e.g. Brown, 2006; McCarthy, Ondaatje, Zakaras & Brooks, 2004; White & Hede, 2008), it embarked on a ‘stakeholder engagement exercise’ which just produced a list of benefits already identified in these existing models. Falsely equating value for money with the economic value of engagement, it then conducted a statistical survey to estimate the effect of cultural engagement on SWB, before monetizing wellbeing through the contested ‘income generation approach’. An inevitable outcome of this process is findings like going to a concert at least once a week generates SWB equivalent to a £9,000 increase in annual household income. This methodology is both reductive and flawed, as it is based on two false premises: firstly, that the public is capable of isolating the value and benefits of complex, multi-dimensional experiences; and secondly, that this value can (or indeed should) be expressed in monetary terms. The wellbeing debate The CASE programme’s objective of exploring the relationship between cultural engagement and SWB serves a useful purpose in further discrediting SWB as a preferred alternative to the so-called stated preference techniques for the justification of arts funding. Evidence from other sectors indicates that SWB is a highly affective construct strongly influenced by personality traits (Moum, 2007). This might well have validity implications for adopting the methodology in the arts sector: for example, when interviewing generally positive people immediately after an emotional opera. The other methodological problem is that there are literally hundreds of scales that claim to measure SWB despite the absence of a widely accepted definition of the term (Davern, Cummins & Stokes, 2007). The advantage of SWB is that it focuses on the public’s internal judgements of well-being, rather than imposing the views of policymakers and academics (Diener and Suh, 1997, as cited in Galloway, Hamilton, Scullion & Bell, 2005, p. 33). It also taps into a wider topical debate about how to understand (or measure) the general state of a nation. However, perhaps precisely because they cannot be easily measured, the Office of National Statistic recently decided to omit the arts, culture and heritage from the headline measures of national wellbeing (Holden, 2012). The implications of this are potentially very damaging for the sector: “once implemented the measurement of wellbeing will have a significant impact on the design, targeting and impact of policy making across government – in which case the absence of proper measurement of arts, culture and heritage would be disastrous” (Jennings, 2012). But as the CASE programme illustrates, the problem with measuring wellbeing is that there remains “no definite set of indicators which can measure the contribution of culture and sport to quality of life and well-being, regardless of how these terms are defined” (Hamilton & Scullion, as cited in Galloway, et al., 2005, p. 155). Measuring the immeasurable In the past decades, the arts have become increasingly subject to the benchmarks of incompatible disciplines and practices. While economic practice can be quantified and evaluated in its own terms, socio-cultural practices demand qualitative, situational evaluation. But to fit into standardised public policy frameworks, they have increasingly been forced into reductive paradigms to justify their social utility. Sanderson (2000) blames this trend on the dominance of rationalist-modernism and Galloway (2009, p. 127) identifies it with a bias towards a “successionist model of causation”. A famous joke, based on an Oscar Wilde quotation, defines an economist as “a man who knows the price of everything and the value of nothing”. There is a serious point behind this aphorism and it is perhaps significant that it was coined by an artist. It is a truism to claim that some things don’t have a price, and many of the values and benefits that populate the literature on the impact of the arts could indeed be argued to be priceless. But public spending in England is controlled by HM Treasury, which, as its name suggests, is governed by economists; and on 2nd November 2011, the former Cabinet Secretary Gus O’Donnell publically warned the Parliamentary Group on Wellbeing Economics: “If you treasure it, measure it!” (Jennings, 2012). However, there is a growing movement away from this economic rationalism which is gradually uniting artists, social scientists and cultural economists. Throsby (2001, p. 1659) explains the limitations of the economic model as follows: “Although it may be tempting to economists […] to claim that the economic value of a cultural good gives a complete account of both its economic and its cultural worth, thereby making a separate measure of cultural value redundant, it has to be remembered that the economic model itself is limited in its reach and specific in its coverage”. This point is supported by the ‘performative’ school of economics (Callon, 2007), which holds that economic theory must coexist with “empirical knowledge and operational tools of many sorts” (MacKenzie, Muniesa & Siu, 2007, p. 6). The Uneconomics movement marks a further challenge to the dominance of orthodox economics in public policy, arguing that by ignoring ambiguity and complexity, economists are losing the authority to describe truth and reality in a credible, disinterested fashion (Davies, 2012). In a similar vein, Scott (2010, p. 285) reminds us that “measurable does not mean valuable” and warns that when public funding decisions rely solely on measurable results “we are back in the bind of instrumentality”. A brief review of the arts literature soon explicates why any meaningful measurement of the impact of the arts remains elusive. Belfiore and Bennet (2007) enumerate the complex range of artistic, personal and circumstantial determinants comprising the aesthetic experience and Holden (2012) points out that while some economic and social benefits of culture can be measured, personal, intrinsic impacts take us into the immensurable realms of spirituality and emotion. Throsby (2006) also concedes that certain expressions of cultural value “transcend economic valuation, as they are rooted in shared social experiences rather than individual utility” (O'Brien, 2010, p. 19). O’Brien (2010) notes that intrinsic value cannot be measured, and this is essentially where the CASE programme falls down: How can complex, multisensory experiences be measured? What scale can measure captivation, escapism, ritual and pity? What price can be placed on hushed anticipation, a shed tear or a shiver down the spine? As experiential marketers have discovered the hard way, the simple answer is that complex human experiences transcend measurement (Vuyk, 2010). But value can be articulated (Hewison, 2006). It can also be created, experienced, absorbed, felt, identified, captured, deconstructed and evaluated. This is where qualitative research comes to the fore through established methods such as participant observation, narrative enquiry and guided introspection (Wallendorf & Brucks, 1993). However, the challenge does not only lie in explicating the arts experience itself, but also the pre- and post-liminal activities that surround it. Brown and Novak (2007), for example, highlight the role of anticipation and distinguish between immediate and cumulative impact. We should remember here that the overarching aim of the CASE programme was to understand the drivers, impact and value of engagement in culture and sport. This begs the question to what extent measuring cultural engagement can help us to understand its social value. McMaster (2008) declared the need to progress from a culture of measurement to one of judgement and Matarasso (1996, p. 13) proposes that the arts could provide their own solution here by developing “sensitive, creative, people-centred approaches to evaluation which begin to address the outcomes, rather than the outputs, of policy initiatives”. Putting the public back into value Public value is not an easy concept to define, but it has been argued that to avoid the dangers of artistic solipsism, cultural policy must engage better with the public (Gray, 2008) and take a bottom-up approach (Hamilton and Scullion, as cited in Galloway, et al., 2005). Arts Council England’s Taking Part survey attempts to achieve this by collating annual attendance and participation data directly from audiences. Based on face to face interviews, the survey also explores drivers and barriers to attendance and has proved useful for segmentation purposes. However, the survey is used essentially to provide quantitative data to provide national benchmarks of engagement to highlight trends and regional disparities and thus comprises closed questions designed for comparison and validity purposes rather than to elicit any depth of insight. The problem with both the Taking Part survey and the CASE programme is that neither study gives the public a free or genuine voice. As Scott (2010, p. 285) points out, “it is precisely because ‘value’ encompasses multiple outcomes, including those not easily captured by quantitative measurement, that the voice of the public is important […]. Their words can lift the discourse with government decision makers to consideration of the multiple levels of Public Value that arts and cultural heritage can and do create.” Scott maintains that the role of citizens in identifying and defining public value remains contested and calls for a more consensual approach to delivering and evaluating it. Kelly, Mulgan and Muers (2002) go even further, calling for the public to be placed at the heart of public policy because ultimately only they can know what is of value to them. This argument raises a number of questions about the language used to articulate public value in the arts and culture. For example, do audiences perceive the impact of the arts in economic terms? Is value for money and return on public investment what drive them to attend? Do they equate the value of their artistic experiences with their willingness to pay for them? If public funding of the arts responds to an acceptance of ‘market failure’, it follows that an authentic impact evaluation of the arts should not be based on market values: the public are citizens before they are consumers (O'Brien, 2010), and they engage in leisure activities as much to create shared meaning as to consume (Arai & Pedlar, 2003). To give the public an authentic voice, studies into value and impact should surely employ a methodology that listens and provides sufficient space for participants’ voices to emerge. Plenty of studies have achieved this, providing rich insights into how the public perceives the drivers, impact and value of cultural engagement. For example, the innovative methods employed by the Impacts ‘08 team gave a voice to the “lived experiences” of Liverpool residents (Garcia, Melville & Cox, 2010, p. 5), exploring impact in a holistic, longitudinal way before, during and after the city’s year as European Capital of Culture. Another mixedmethods study of audiences isolated five key dimensions of their arts experience: engagement and concentration; learning and challenge; energy and tension; shared experience and atmosphere; personal resonance and emotional connection (New Economics Foundation, 2008). And a qualitative study of theatre-goers at West Yorkshire Playhouse and Melbourne Theatre Company found that the key driver for attendance was the pursuit of emotional experiences and impact (Walmsley, 2011), where impact was articulated in terms of captivation, escapism, empathy, quality of life, relationship building, world view, camaraderie and buzz (Walmsley, 2012). These qualitative studies provide an alternative to the facts and figures style insights provided by the CASE programme. But as O’Brien (2010, p. 8) points out, they will fail to resonate with HM Treasury because they don’t “represent the benefits of culture in a manner that is commensurable with other calls on the public purse”. However, O’Brien concedes that narrative accounts provide an important framework for understanding cultural value and “remind us of the need to make the case for culture in a variety of ways” (ibid.). Galloway contests the standard dismissal of narrative evidence in cultural policy, arguing that it successfully encapsulates subjective perceptions of impact (Galloway, et al., 2005). And in their own narrative study of the impact of art on individuals, White and Hede (2008) call for a paradigm shift in modes of enquiry, highlighting the uncomfortable truth that impact is complex, subjective and contingent. A balanced evaluation of artistic activity should consider both professional practice and the audience or participant experience. Matarasso (1996) argues that what matters in arts evaluation is performance and context. Evaluation should therefore be approached reflexively, since reflexivity is a form of critical analysis of context and theorises practice as both spatial and situational. A reflexive approach to evaluation would reject the HM Treasury conception of value as quantifiable, fixed and given, and regard it instead as emergent, “constantly under negotiation and in-the-making” (Oliver & Walmsley, 2011, p. 88). It would also reflect and articulate cultural value in the authentic language of artists, practitioners and audiences. Towards a balanced scorecard There is an identifiable need for closer symbiosis between the qualitative micro evaluation expected by public funders of arts and cultural organizations and the quantitative macro evaluation demanded by HM Treasury. At the moment, there appears to be a serious disconnect here, with the former seeking more subjective, qualitative evaluations of the accomplishment of artistic objectives (ROO) and the latter demanding cost benefit analyses to assess the return on their investment (ROI). These well established battle lines take us back to the debate surrounding the fundamental role of evaluation. Although there lacks a coherent rationale for evaluation (Matarasso, 2009), for artists, practitioners and audiences, evaluation typically involves reflecting on artistic processes to improve future activity. But at the macro level, evaluation appears to be “inherently political, concerned with the distribution of power and the allocation of resources and opportunities in society” (Simons, 2009, p. 17). It is worth noting here that four fifths of the UK population support the public funding of the arts and many of these report that their lives are enriched by them (Foster, 2009). This finding supports the idea of taking the social value of the arts as a given, which is a recurrent theme in the literature. For example, Galloway et al (2005) praise Morris Hargreaves McIntyre’s work for Shropshire County Council and Colin Mercer’s research in Essex, both predicated on the assumption that cultural activity makes a positive contribution to quality of life. These are good examples of local government taking the lead and there are lessons to be learned here by central government. This message appears to be getting through, with current and former Culture ministers and shadow ministers, including Ed Vaizey, Don Foster and Tessa Jowell, all openly accepting the intrinsic value of the arts. In an era of unprecedented cuts in local government funding for the arts, this ministerial support is more important than ever. So if Culture ministers need no persuasion, why is HM Treasury so resistant to change? There seems to be no valid reason why every Whitehall department’s spending needs to be calculated, evaluated and justified in the same way in a one-size-fits-all homogeneity of public policy. As Holden (2012) remarks: “Any fool can compare apples with apples. Politics involves judgement, not managerialism.” There have been calls in recent years to move towards a more balanced evaluation of organizational performance. Kaplan and Norton (1992) developed the Balanced Scorecard as a strategic management tool to evaluate performance based on organizations’ missions and objectives. The Balanced Scorecard combines financial indicators with softer, less tangible assets such as organizational culture, processes and innovation and Boorsma and Chiaravalloti (2010) encourage arts organizations to adopt it, arguing that it places the artistic mission at the heart of performance management and explicitly addresses the different types of artistic value demanded by key stakeholders. Another example of a successfully applied multi-dimensional approach is Holden’s (2006) value triangle, which deconstructs cultural value into instrumental, institutional and intrinsic value and endorses evaluation across all three realms. Other advocates of a more balanced approach include Schalock (2004), who calls for methodological pluralism in any research assessing indicators of quality of life; and Scott (2010), who endorses a holistic approach to impact research that could both describe and measure artistic value. The legacy of the CASE programme So where does all this leave the database of 5,733 quantitative abstracts and the computer simulation model of people’s “pathway to engagement” produced by the CASE programme? The CASE board claims to have “dipped its toes” into the complexity of public engagement in culture (EPPI Centre, 2010, p. 13). If this is the case, it could be countered that it withdrew them as rapidly as possible as the water was apparently too hot. Although it provides a useful starting point for social and cultural policy researchers, by persisting in the futile goal of measuring the immeasurable and by discounting the complex insights offered by more reflexive qualitative studies, the programme ultimately fails to provide any further understanding of public engagement in culture to influence the kind of future policy decisions that might actually spur the public into deeper or more frequent engagement. Conclusion There seems to be a loose consensus in the literature that a modern, fit-for-purpose evaluation of the drivers, impact and value of the arts should embrace both intrinsic and instrumental outcomes and include both qualitative and quantitative methods. While there are clearly problems with the current state of evaluation in the sector, there are also countless examples of good, creative practice. There is undoubtedly a need for better knowledge transfer and exchange in arts evaluation and performance measurement, and this study advocates a holistic, balanced scorecard approach, where outcomes are judged against multiple objectives rather than on investment alone. The arts sector excels at telling stories, and it must not relinquish this right, this oldest of traditions, in communicating its value and impact. The key implication of this re-conception of value is that cultural policy should be articulated in a language that reflects artistic practice and speaks directly to audiences. Otherwise the risk is that a policy aimed to widen participation in the arts will only further alienate potential audiences and erect another barrier to engagement. It could be argued that we find ourselves at a turning point in history, where traditional economic models are being questioned and that this provides a historic opportunity for the arts sector to unite to reject the measurementobsessed dogma of public policy and develop a more creative approach, which puts the public back into value and the value back into evaluation. Evidence-based policy is all well and good, but evidence of cultural engagement cannot uniquely comprise un-contextualised quantitative data. Ultimately, “culture is concerned with the construction and transmission of meaning” (McGuigan, 2004, in O'Brien, 2010, p. 11) and if, as Sharpe (2010, p. 2) contends, “art is the currency of experience” in an “economy of meaning”, then the arts surely have a key role to play in elucidating the value of this meaning. The ultimate objectives of the CASE programme were to increase the public’s engagement in culture and sport and assess its likely impact on their wellbeing. It is unlikely that a database of facts and figures or a computer simulation of participation will accomplish the first aim; and the arts indubitably have a larger, wider purpose than quantifying happiness: as ever, they reflect our world back to ourselves and shape our very culture (Vuyk, 2010). References Arai, S., & Pedlar, A. (2003). Moving beyond individualism in leisure theory: a critical analysis of concepts of community and social engagement. Leisure Studies, 22(3), 185-202. doi: 10.1080/026143603200075489 Belfiore, E., & Bennett, O. (2007). Determinants of impact: Towards a better understanding of encounters with the arts. Cultural Trends, 16(3), 225–275. doi: 10.1080/09548960701479417 Boorsma, M., & Chiaravalloti, F. (2010). Arts marketing performance: an artistic-mission-led approach to evaluation. The Journal of Arts Management, Law and Society, 40, 297317. doi: 10.1080/10632921.2010.525067 Brown, A. S. (2006). An architecture of value. Grantmakers in the Arts Reader, 17(1), 18-25. doi: Brown, A. S., & Novak, J. L. (2007). Assessing the intrinsic impacts of a live performance. San Francisco: WolfBrown. Callon, M. (2007). What Does it Mean to Say that Economics is Performative? In D. A. MacKenzie, F. Muniesa & L. Siu (Eds.), Do Economists Make Markets?: On the Performativity of Economics (pp. 311-357). Princeton, NJ: Princeton University Press. Davern, M. T., Cummins, R. A., & Stokes, M. A. (2007). Subjective wellbeing as an affective-cognitive construct. Journal of Happiness Studies, 8, 429–449. doi: 10.1007/s10902-007-9066-1 Davies, W. (2012, 4 March 2012). Uneconomics: A challenge to the power of the economics profession. Retrieved from http://www.opendemocracy.net/ourkingdom/williamdavies/uneconomics-challenge-to-power-of-economics-profession EPPI Centre. (2010). Understanding the drivers, impact and value of engagement in culture and sport: An over-arching summary of the research. London: DCMS. Foster, D. (2009). The value of the arts and creativity. Cultural Trends, 18(3), 257–261. doi: 10.1080/09548960903086192 Fujiwara, D., & Campbell, R. (2011). Valuation techniques for social cost-benefit analysis: Stated preference, revealed preference and subjective well-being approaches. A discussion of the current issues. London: HM Treasury. Galloway, S. (2009). Theory-based evaluation and the social impact of the arts. Cultural Trends, 18(2), 125-148. doi: 10.1080/09548960902826143 Galloway, S., Hamilton, C., Scullion, A., & Bell, D. (2005). Quality of life and well-being: measuring the benefits of culture and sport: literature review and thinkpiece. Edinburgh: Scottish Executive Social Research. Garcia, B., Melville, R., & Cox, T. (2010). Creating an impact: Liverpool’s experience as European Capital of Culture. Liverpool: University of Liverpool and Liverpool John Moores University. Glinkowski, P. (2012). Artists and policy-making: the English case. [Article]. International Journal of Cultural Policy, 18(2), 168-184. doi: 10.1080/10286632.2011.577213 Gray, C. (2008). Arts Council England and public value: A critical review. International Journal of Cultural Policy, 14(2), 209–214. doi: 10.1080/10286630802106383 Hewison, R. (2006). Not a sideshow: Leadership and cultural value. London: Demos. HM Treasury. (2011). The Magenta Book: Guidance for evaluation. London: HM Treasury. Holden, J. (2006). Cultural value and the crisis of legitimacy: Why culture needs a democratic mandate. London: Demos. Holden, J. (2012, 5 January 2012). New Year, new approach to wellbeing? Retrieved from http://www.guardian.co.uk/culture-professionals-network/culture-professionalsblog/2012/jan/03/arts-heritage-wellbeing-cultural-policy Jennings, H. (2012, 12th December). Measuring wellbeing – no place for the arts and culture? Arts Professional Kaplan, R. S., & Norton, D. P. (1992). The Balanced Scorecard—measures that drive performance. Harvard Business Review, 70(1), 71-79. doi: 10.2307/1323539 Kelly, G., Mulgan, G., & Muers, S. (2002). Creating public value: An analytical framework for public service reform. London: Cabinet Office. MacKenzie, D. A., Muniesa, F., & Siu, L. (2007). Introduction. In D. A. MacKenzie, F. Muniesa & L. Siu (Eds.), Do Economists Make Markets?: On the Performativity of Economics (pp. 1-19). Princeton, NJ: Princeton University Press. Matarasso, F. (1996). Defining Values: Evaluating arts programmes. Stroud: Comedia. Matarasso, F. (2009). The Human Factor: Experiences of arts evaluation, Version 1.5. Retrieved from http://web.me.com/matarasso McCarthy, K. F., Ondaatje, E. H., Zakaras, L., & Brooks, A. (2004). Gifts of the muse: reframing the debate about the benefits of the arts (No. 0833036947 (pbk)). Santa Monica, CA: RAND. McMaster, B. (2008). Supporting excellence in the arts: From measurement to judgement. London: Department for Culture Media and Sport. Moum, T. (2007). A Critique of ‘‘Subjective wellbeing as an affective cognitive construct’’ by Davern, Cummins and Stokes. Journal of Happiness Studies, 8, 451–453. doi: 10.1007/s10902-007-9067-0 New Economics Foundation. (2008). Capturing the audience experience: A handbook for the theatre. London: New Economics Foundation. O'Brien, D. (2010). Measuring the value of culture: a report to the Department for Culture Media and Sport. London: Department of Culture, Media and Sport. O'Brien, D. (2012, 13 February). A year of measuring the value of culture. Retrieved from http://sotablog.artscouncil.org.uk/post/17547433925/a-year-of-measuring-the-valueof-culture Oliver, J., & Walmsley, B. (2011). Assessing the value of the arts. In B. Walmsley (Ed.), Key Issues in the Arts & Entertainment Industry (pp. 83-101). Oxford: Goodfellow. Rubin, H. J., & Rubin, I. (2005). Qualitative Interviewing: The art of hearing data (2nd ed.). Thousand Oaks, CA; London: Sage. Sanderson, I. (2000). Evaluation in complex policy systems. Evaluation(6), 433-454. doi: 10.1177/13563890022209415 Schalock, R. L. (2004). The Concept of Quality of Life: What We Know and Do Not Know. Journal of Intellectual Disability Research, 48(3), 203-216. doi: 10.1111/j.13652788.2003.00558.x Scott, C. A. (2010). Searching for the “public” in Public Value: arts and cultural heritage in Australia. Cultural Trends, 19(4), 273–289. doi: 10.1080/09548963.2010.515003 Sharpe, B. (2010). Economies of life: patterns of health and wealth. Axminster: Triarchy Press Simons, H. (2009). Case Study Research in Practice. London: Sage. Stevenson, D., Rowe, D., & McKay, K. (2010). Convergence in British cultural policy: The social, the cultural, and the economic. Journal of Arts Management, Law and Society, 40, 248–265. doi: 10.1080/10632921.2010.500926 Throsby, C. D. (2001). Economics and Culture. Cambridge; New York: Cambridge University Press. Vuyk, K. (2010). The arts as an instrument? Notes on the controversy surrounding the value of art. International Journal of Cultural Policy, 16(2), 173–183. doi: 10.1080/10286630903029641 Wallendorf, M., & Brucks, M. (1993). Introspection in consumer research: Implementation and implications. Journal of Consumer Research, 20(3), 339-359. doi: Walmsley, B. (2011). Why people go to the theatre: A qualitative study of audience motivation. Journal of Customer Behaviour, 10(4), 335-351. doi: 10.1362/147539211X13210329822545 White, T. R., & Hede, A.-M. (2008). Using narrative inquiry to explore the impact of art on individuals. Journal of Arts Management, Law and Society, 38(1), 19-35. doi: