Cultural managers and practitioners increasingly seek to understand and measure the impact of their projects on communities. This is important for best practice operation and ensuring the responsible use of public funds, and is increasingly demanded by funding bodies and decision-makers.
The intention to plan and evaluate responsibly can be challenging, with a lack of specialist expertise in evaluation common amongst those with demanding workloads. To date, there have been few specialised tools available to support the planning and evaluation of cultural and community projects. When evaluation happens, it often occurs as an after-thought, with experts brought in to measure impacts of activities that have not been considered in the planning stages.
Recognising that simpler solutions are required, the Cultural Development Network (CDN) and Culture Counts have developed two new tools – ZAPS (by CDN) for cultural project planning, and Culture Counts for the evaluation and reporting of cultural project impact. The two together offer a comprehensive solution to the problems of valuing cultural initiatives.
ZAPS is an online tool that supports the planning of projects; stimulating thinking about why the project is being undertaken, and therefore the change that is sought as a result. It encourages evidence-based work, by providing a bank of information that practitioners can draw on, as well as an iterative process by which project leaders can share findings of their evaluations with peers.
Historically, cultural organisations have counted only outputs, (the number of arts products created, event participants or tickets sold) as metrics of success. CDN has been working with the National Local Government Cultural Forum in Australia to establish an agreed set of definitions for inputs, outputs and outcomes of local government’s cultural activity. All outcomes fall within one of five public policy domains (cultural, social, civic, economic and environmental), and all publicly funded activities can generate outcomes across one or more of the domains.
Culture Counts complements this process by offering a technological solution to the collection, analysis and reporting of data. Since 2012, Culture Counts has been measuring emotional and intellectual responses from the public, peers and organisations and turning them into big data insights, generating hard evidence to help the arts attract investment and grow audiences. The use of a common language, collection methodology and analysis tool enables data to be interrogated, controlled, aggregated and globally benchmarked.
How Culture Counts Evaluation Platform Integrates with ZAPS Planning Framework
Culture Counts metrics connect to the CDN measurable outcome schema, enabling measurement across cultural, social, civic, environmental and economic domains. By using an agreed set of outcomes across these five domains, it is possible for all investments or programs to be evaluated using the same criteria, to identify where the greatest change is occurring and where best to direct limited funds to achieve maximum benefit. Outcomes data collected in Culture Counts can be fed back into ZAPS to demonstrate the success of the project or identify areas for improvement, forming an evidence base of learnings for the sector.
Both tools have been developed in collaboration with cultural organisations and local governments across Australia and overseas, seeking to meet the needs of the sector in practical and user-friendly ways. They are designed to be used by project managers and staff themselves, empowering designers and executors of arts projects to assess and evaluate as they go. The inclusion of outcome-focused planning and integrated evaluation into project design is expected to make a significant positive contribution to project impacts. The sector overall can be advanced by tools that enable practitioners to learn from each other.
Stay tuned for a practical case study about the City of Sydney’s involvement in this collaborative initiative.
Image: Alicia Frankovich, Outside Before Beyond, 2017. Installation view Kunstverein für die Rheinlande und Westfalen, Duesseldorf 2017. Photograph: Katja Illner.
