With very special thanks to Pat Swell, CEO of Access Arts, for sharing their results and insights for this case study originally published on the Arts Queensland blog on 13 August 2015.
Pat Swell shares Access Arts’ experience of the Culture Counts platform, used to gather audience and artist insights from the Undercover Artist Festival.
Like me there are probably items on your job list that silently slip across to the next year, not yet done. For me, developing a methodology to measure the artistic vibrancy of our organisation was one such job. Access Arts has a strong track record of collecting feedback. We need this for our regular external audit under the Human Services Quality Framework. Turning this data into a useful measure for artistic vibrancy was my challenge, creating a lens that would moderate artistic innovation, quality of execution and strategic importance through customer and peer assessment.
And so I leapt at the opportunity to be part of Arts Queensland’s Culture Counts pilot. Not only would this deliver a framework for the future, but it would also give us external validation as it would be conducted by an outside body. With Culture Counts’ global connections, we might be able to compare our work with similar national and overseas organisations.
Working with Georgia Moore, Executive Director for Culture Counts, was an absolute pleasure. The brief was to survey Undercover Artist Festival, Queensland’s first disability arts festival. The Festival this June aimed to fill a gap in Queensland’s cultural infrastructure, a bold new departure for a small organisation like ours. The Culture Counts report would be a fundamental tool for us to identify how we performed and map the next steps.
How did we fare? At the end of the Culture Counts survey, people who attended the festival were asked to rate their experience overall, with a choice of five options – excellent, good, average, poor, very poor. 95% of respondents found the event to be better than average, with 79% having an excellent experience and 16% having a good experience. No one surveyed found the experience to be poor or very poor. Not bad then. We also learned that 62% of respondents had not had any previous involvement with Access Arts, suggesting we were reaching new audiences through the pilot.
People who attended the festival were also asked eight Culture Counts ‘dimension’ questions about artistic quality. These are related to captivation, rigour, relevance, meaning, challenge, imagination and distinctiveness. 97% of people agreed or strongly agreed that the Festival was absorbing and thought-provoking (captivation and challenge). Over 90% of respondents also agreed or strongly agreed that it had something to say about today’s world (relevance), was moving and inspiring (meaning), was well thought through and put together (rigour), helped them to feel connected to people in the community (connection) and explored a new point of view (imagination). The average scores were lower for distinctiveness, though 86% of respondents still agreed or strongly agreed that the Festival was different from things they’d experienced before. So far so good.
To the public’s evaluation, Culture Counts adds the lens of self-assessors and peer assessors, making a three-pronged evaluation process. Pre and post-event surveys measured expectation and actual experience of the Festival from the point of view of Access Arts as the event organiser and a selection of peers. The insights showed that expectations and actual experiences generally matched up, with strong ratings across all eight dimensions through both peer and self-assessment.
For our particular project which involved artists in a series of developmental workshops leading up to the festival, we also had the opportunity to trial some new metrics being developed by Culture Counts to evaluate participation experiences (in addition to evaluating audience/attendee experiences). In most instances, artists surveyed indicated that their experience of participation matched or exceeded their expectations. Results were particularly strong for the dimensions of belonging, skills, motivation and stretch, with all participants strongly agreeing they were made to feel part of the team, gained skills, were motivated to do more creative things in the future and did something they didn’t know they were capable of. Participants provided the following comments:
The experience has been life-changing. The workshop gave me the confidence to know I am heading in the right direction. Overall I was extremely pleased with my opportunity to attend this workshop. Worth its weight in gold!
Where to from here? It’s early days. Undercover Artist Festival only finished six weeks ago. We now have great data to mine and apply. I’m pleased Access Arts has been part of the Culture Counts pilot as it has given us hard evidence, rigorous results and impartial validation as we embark on this process of deciding our next directions.
Pat Swell is Chief Executive of Access Arts. She has worked in arts and cultural development in non-profits and the public sector in Australia and England. Previous posts include Strategic Director of Milton Keynes Theatre and Gallery Company and Senior Officer at Arts Council England. She holds a Master of Business Administration and serves on Griffith University Business School’s Alumni Group.
