OPEN SPACE INTERACTIVE DISCUSSION SUMMARY
Present: Roy Hanney (Southampton Solent University; Media production), Pune Pasafar (Middlesex University; Practice based research), Vesna Lukic (Middlesex University; film production), Djamila Boulil (independent researcher; social impact in arts and culture).
This open space discussion centred around the topic of how to measure impact in the arts. This is something that is increasingly demanded by funders and governments in UK and Europe and a hot topic in the university sector. Following Djamila’s video paper on the topic of impact in the symposium, we discussed several options for measuring the impact of creative projects. The focus was on two main challenges. The first was on how to measure impact (a long-term effect), when the only time available is during what are mostly short-term projects. The issue here is that once the funding is finished there is no additional support for longitudinal follow up evaluation.
Second, a preference for either qualitative or quantitative methodologies was discussed. The tension here being around the need for quantitative results from funding bodies, organisations, governments and so on. While creative practitioners feel that qualitative research is more informative, appropriate and of greater value. Importantly qualitative impact evaluation gives voice to the communities with whom creative practitioners are engaged which provides an opportunity for dialogue and potentially for empowerment.
Given the mismatch between project time and research time, we discussed the possibility that evaluation might focus on the intent for impact or for change by (co)creators. It was felt that this might go some way towards laying down a sense of a pathway for impact even if in fact, if there is no time available to really measure impact some way down the line.
Naturally, the distinction between creators, participants and audience was touched upon, all with different possibilities and approaches when it comes to impact. When talking about audiences, the conversation moved to the need for a focus on the social return on investment, rather than the need to measure only the economic return by counting the number of audience members. A way to do this is initiate creative projects with a Theory of Change (ToC) workshop that engages stakeholders in a discussion that sets out their intent for the project. thereby setting out exactly what the expected or desired effect of the project would be as a consequence of the project. When these objectives are clear, it was felt that behavioural science offers several options to quantify the qualitative nature of cultural projects.
We got side-tracked a little in the end. As all participants shared a love for film as artistic medium, the of a social action documentary became an example that was used to question some fundamental aspects of the balance between the social and the cultural. A documentary for instance can, possibly, communicate a certain ideal or goal. This makes that the political nature of art is unescapable; even when one tries to avoid it, doing nothing is also a statement. The discussion ended with the question, well a plea, to support pluralism, so that a range of voices might be heard.
In conclusion Roy, Pune, Vesna & Djamila would like to advocate to creatives to give some attention to your intent in the early stages of a project. In concurrence with the ToC: Make clear to yourself and your stakeholders what kind of effect you want to have on your environment (if your art project is one that has a social component). This also helps you identify what “math tricks” to use to quantify your social return and have that number to give back to some more market-oriented funders. Key is to always match your methods to your wanted impact, whether they are qualitative or quantitative (or something in between those or outside them).