Impact Assessment as a call to action and a better strategy
Guest post from Professor Simon Tanner
Why does digital strategy in the sector feel hard?
How about:
Organisational strategy doesn't always make best use of digital. This leads to digital having to make a case for its strategic importance. Which becomes a little circular if to do that it needs to reference the organisation strategy (which doesn't mention it).
We don’t know how to measure digital progress in a meaningful way. We have lots of digital data. But we’re not very good at relating this data to the organisation’s values.
It was while having thoughts like this that I came across Professor Simon Tanner’s recent book Delivering Impact with Digital Resources. In it, he comes up with a model for planning, measuring and evaluating the importance of digital projects. An organisation must pick a way of defining ‘value’, and from there it can derive how best to measure digital impact.
I’m very pleased to have Professor Tanner on Cultural Content as a guest poster, to take you through his book in 1,500 words. If you work in digital strategy or impact evaluation, this is for you. If you’re just interested in how to associate digital activity with value, come on board 🙌.
Over to Simon…
Memory institutions have the power to improve lives and life opportunities. Impact Assessment is the tool to foster an understanding of how strategic decisions may change behaviour within our communities and add value to them. This article will illustrate the beneficial connections between evidence-based decision-making, impact, and strategies to deliver a compelling call to action. Introducing the Balanced Value Impact Model (BVI Model, see www.bvimodel.org) will provide stepping stones for implementing these ideas.
Sustainability requires us to articulate the value and impact of our work powerfully. Moving beyond traditional evaluation measures is desirable to address our challenges. Primary measures have focused on evaluations of outputs (how much digitised), basic user statistics (how many people), or anecdotal information (feedback) rather than outcomes (value gained and change delivered). In short, introducing impact assessment responds to the demand for a strategic mechanism demonstrating that resources and service ecosystems are delivering a beneficial change to communities, expressed in economic and social terms.
Answering the Who Cares? question via the BVI Model
At its heart, Impact Assessment seeks to answer "who cares?" questions. My definition of Impact Assessment is the measurable outcomes arising from the existence of a service or resource that demonstrate a change in the life or life opportunities of the community. I used this as the basis of the Balanced Value Impact Model to put people at the centre of assessing impact and to raise expectations that cultural content strategy will aspire to make a beneficial change in people's lives.
As illustrated below, there are ten main viewpoints from which impact can be measured. With so many dimensions to impact, finding the most appropriate focal area is an important starting point.
Measure impact by focussing on the strategic change in services or content. Show how this improved life opportunities and benefited your audiences and communities.
Data gathering can use qualitative and/or quantitative methods. There is no shortage of tools and excellent guidance on data collection, metrics and usage figures, but knowing which one to apply to measure impact is not easy. Thinking about how you will use the intended resulting information and what it is worth to know this information will help guide choices to avoid expensive approaches that produce little actionable intelligence.
To ease choosing methods, it helps to structure the selection through the following Strategic Perspectives and Value Lenses designed specifically for cultural content within the BVI Model. The Strategic Perspectives and the Value Lenses guide the key impact decisions:
The BVI Model moves through the 4 Strategic Perspectives to select pairings with these Value Lenses to focus on the most productive aspects for measurement in that context:
The practitioner can use the Value Lenses to focus on the most productive aspects for measurement in that context. Every institution will have a different starting context (e.g. who makes up its community) which must be understood otherwise the impact narrative and results may be partial or lacking in actionable meaning. The goal is to narrow the innumerable impact measures possible to a manageable and meaningful subset.
Selection process pairing Strategic Perspectives with Value Lenses
The BVI Model is a cohesive model for impact assessment with a pragmatic framework for delivery. It is a means to structure activity to set strategic contexts, decide what to measure, and add purpose and a disciplined approach to the often-vague concept of impact. I argue in my book (Delivering Impact for Digital Resources: Strategies for the Attention Economy) that defining modes of value for digital culture is not solely driven by economics but contains indicators of other values, such as education, community, existence/prestige, inheritance/legacy, and utility.
This balanced approach in the BVI Model seeks to show that the services and cultural content demonstrably make the host organisation grow better, becoming more efficient and effective in reaching its goals, while its audiences and communities have become more satisfied, finding social, community and educative benefits of tangible worth that enhance society.
The BVI Model guides an institution to acknowledge and describe their values, objectives and stakeholders before agreeing on which aspects of itself and its services or content are included within the scope of the impact assessment. The act of asking what a community does with content or in response to a service and the impact of these upon their lives can be challenging to measure, but the benefits of this knowledge are strategic gold dust.
The following figure illustrates the measurement goals in a simple exemplar framework focused on the Value Lens of Utility for a digital collection.
This structured framework can be used to plan activity with clear objectives, monitor it whilst it's in progress and evaluate the success once finished.
The BVI Model also provides a natural structure for narrating results via the four strategic perspectives: social, economic, innovation, and operational. This narrative acts as a prompt to decision-makers and stakeholders to guide future direction and to justify current activities. New initiatives or strategic change usually require refocusing resources (especially time, money and infrastructure) from other operational activities. Understanding their impact allows for continuous assessment of whether innovation is working and how.
The stage of reflection and response vitally assumes that there is embedded continuous learning to allow for decisions made in every stage to be informed by the results and modified in response to change, feedback, and success criteria. Impact assessment also allows for the deliberate modelling of strategic planning. A pivotal realisation to accept is that the most critical outcomes of impact assessment may be to guide decisions about what to stop or pause doing now, plus where to refocus effort and resources in the future – as much as it is about what to shout about from the rooftops.
Europeana Foundation case study
The Europeana Foundation is the organisation tasked by the EC with developing a digital cultural heritage platform for Europe. Europeana has used the BVI Model to develop its own policy and strategic framework, the Europeana Impact Framework, as part of Strategy 2020.
Europeana has further led the sector by developing the Impact Playbook as a critical component in providing ongoing impact evaluation. This used the BVI Model as a foundational building block. An example of Europeana's impact assessment is the Workers Underground, which explored the social impact of a successful and long-running project: Europeana 1914–1918 collects, digitises and publishes personal memorabilia from the First World War. The following film provides an overview:
Through this impact assessment, they learned about how the Europeana 1914–1918 programme is greatly valued by the people who use it. The results also provided information on ways to improve and build on the existing activity. A significant insight – articulated through the education lens – was that users did not learn as much as they expected, and language was the main barrier to discovering new knowledge. This is an example of actionable information that Europeana was able to use to reset its ambitions and focus for future projects.
How measuring impact engages with strategy
Knowing how to start and plan are vital steps toward achievable, cost-effective and strategically driven impact. The BVI Model should guide thinking and planning, and the Europeana Impact Playbook is a great recipe book for getting started. Other resources, such as the Museums of Impact provide innovative playing cards to lead creative thinking about impact measures. Avoid the perception that gathering evidence of impact is something to do purely in response to external pressure or as a one-off event. We should embed evidence-gathering in our organisations so that continuous operational improvement becomes possible and the work's benefit becomes visible to our communities as partners in the process.
Impact connects special projects or new initiatives to the people most affected by the change; it either predicts or will show how well-served those people are. If we want our communities to engage, then understanding the worthiness of that engagement and investing time and other resources to enhance it will support active engagement. Make it easy – easy to understand the benefits, easy to use the services and easy to commit to action. Communities want to hear about the benefits available to them, not the underlying features or functions.
Participation by the public is critical to becoming a learning institution that responds to impact with action. Ask: are you willing to actively listen to your community? When done well, the institution integrates this mode of participatory engagement into its soul, resulting in it resonating far more profoundly than superficial consumption. Especially in the face of our attention economy, it is crucial to have a common understanding of the institution's core values and to deliver this hand in hand with its specific community.
Impact assessment is part of becoming a collaborative and learning organisation willing to change in response to evidence. Embedding impact into evidence-based management thinking and culture, avoiding comfortable metrics and celebrating success are all linked to my belief that the cultural sector plays a pivotal role in the future of society.
Like to hear more from Simon? He’ll be presenting at this year’s MuseTech (Museum Computer Group’s annual conference) on the Balanced Value Impact Model.
I may be biased (as it’s my first year as Chair of the Group) but this year looks like a FANTASTIC line up, and we’re starting ticket prices at the highly economic £44, but be quick, early bird tickets have only been on sale for a day and we’ve already sold a third of them!
Find out more about the event and sign up for tickets. Hope to see you there.
Are you planning on coming to the MCG conference? Drop me a line on Typeform if you’d like to say hi – being a remote worker I really want to maximise my face to face contact at big events like this.