The idea of changing what you do based on data can be scary. This is particularly true in the arts, as not all organisations are striving to reach the maximum number of people or put on the most popular works.
When we talk to organisations about using data insights to make evidence-based decisions, we encourage them to keep their big picture strategic goals front and centre. It’s important to consider how the information that’s been collected through the evaluation process can be used to reflect, learn and bring the organisation closer to achieving its own unique mission and vision.
Learning from evaluation data is different to learning about evaluation processes. As you become an experienced evaluator you will no doubt improve your data collection methods and processes. You’ll learn things like which survey distribution methods work best for your target audience and how many responses you require to generate a significant sample. While these are all useful learnings, it is the insights in the data that are key to successful evidence-based decision making.
Don’t let your evaluation data pile up on the shelf; it’s time to bring it into your internal reflection process.
Reflect for success
Reviewing your evaluation results should form part of your internal debrief process if there is one already in place. If not, it’s a great idea to carve out some time in your annual program of activities to reflect on what you’ve achieved, what worked and what didn’t. This is really important for morale and staff wellbeing. Don’t underestimate the power of a debrief to combat employee burnout, often brought on by feelings of ‘not being able to stop’ or ‘having to always be on’. With finite time and resources, it’s important to make sure the activities you’re investing in are having the greatest impact in line with your goals.
Know what questions to ask
The reflection process is a good time to take stock and check you’re asking questions that are useful. Is there anything you would do differently if you had the data to support a change of direction? For example, I might like to know if audiences who watched a talk via video stream experienced the same impacts as those seeing it live. If yes, I may invest in expanding digital content, if no, I could look at ways of improving. Collect research questions from different departments to make your evaluation is meaningful across the whole organisation. Make sure you have all the data you need for successful evidence-based decision making going forward.
Use comparison data to highlight areas for improvement
By aligning metrics with your key strategic goals and keeping measures consistent across the year, you will be able to determine which of your activities are achieving the biggest impact. Not every program needs to hit every goal, but in aggregate, you will build up a solid picture of how you’re tracking. You will understand which programs were most successful, and which can be built on or learnt from. Comparison data could also take the form of a peer or self-assessment to provide additional points of view. You could also bring various external data that helps to set the scene, such as weather scores for outdoor events or spend-map data to drill down into expenditure patterns in your area more generally.
So your event went badly, what next?
In the current funding climate, it may feel like you’re unable to acknowledge what didn’t go well. It may seem better to bury the results but this is not a solution if you want to set yourself up for long term success. If the data shows your program wasn’t having the desired impacts on your audience, this is not all bad news: you can do something about it. If you’re open to critical feedback, these insights can help take your work to the next level and bring you closer to your audiences by making it a two-way conversation. Create an organisational culture of being open to doing things differently.
Share with others
It’s a great idea to actively seek out opportunities to learn from others. These could be collegiate organisations that are willing to share insights and discuss what has worked well for them. Start practical and collaborative conversations about evaluation and data to strengthen the sector and your own work. Tap into global benchmarks and look to big data insights for ideas and trends.
Want to hear more about how you can become a learning organisation? Watch our latest webinar ‘Reflect for Success: Becoming a Learning Organisation’, hosted by Culture Counts Client Director Alison Lasek in conversation with Squirrel Main, Research and Evaluation Manager at the Ian Potter Foundation.
