A case study from the Oldham Coliseum as a part of the Quality Metrics National Test in the UK. With very special thanks to David Martin at Oldham Coliseum for sharing his results and insight for this case study.
Oldham Coliseum has been involved with the Quality Metrics from the outset in the UK: along with a group of industry peers, a shared understanding of what artistic quality was found and much deliberation took place on refining a set of metrics that could be applied to a number of relevant contexts and then tested.
For the second phase, the Quality Metrics National Test, two evaluations have been run so far, with a third pending. The first, Pitman Painters, written by Lee Hall and directed by Kevin Shaw, is a piece of work about the Ashington Group – Northumberland miners who employed Robert Lyon, a Master of Painting at Kings College Newcastle, to teach them an evening class in Art Appreciation. The production was programmed as a definitive statement about the importance of art for everyone and innate creativity, with political reference. The second production evaluated was Our Gracie – a very different piece of work with a strong music score. This was a newly commissioned play written by Philip Goulding celebrating the life and times of Gracie Fields, a local and extremely popular entertainer.
Public Response
Surveys were sent via email to audience members at the end of each run (159 respondents for The Pitman Painters and 213 for Our Gracie). The Pitman Painters was uniformly liked with the lowest metric for Distinctiveness averaging 70% – all other metrics scored in the high 80s and low 90s. Our Gracie was similarly very popular with audiences, scoring lower for Distinctiveness, Challenge and Relevance – averaging between 58% and 69% – and scoring in the 80s for all other metrics. The dimension profile (the balance of scores across the metrics) were different for these two pieces of work.
Some Reflections
- Average audience response may be skewed, for example if someone likes the show then they may be more likely to say so rather than if they felt indifferently. Using tablets for intercept interviewing front of house may result in a different dimension profile as the sample won’t be as self-selecting.
- Peer and self review broadly followed the same dimension profile as that of audience members. Peers were generally more critical but the pattern of response indicates that the intentions of the work were understood by the peers, inferred by comparing the peer responses to that of the self assessors.
- There is a desire from both self and peer assessors to take a reflective approach when considering the work. Using the quality metrics in this test phase enables a very quick evaluation process – the onus for reflection is down to the respondent and not demanded by the process.
Is it useful?
The methodology and metrics have legs, particularly as a way of gleaning information from audiences if the biases in motivation to provide feedback can be addressed. It probably has a place in a portfolio of approaches and be developed alongside other methodologies of assessing quality. It is certainly something to stay interested in.
