How long is a piece of string? That was my first thought when someone proposed an unconference discussion about how the impact of Participatory Budgeting could be measured. To a degree it depends on the point of view that you take. For public services it usually involves a numerical target e.g. the number of crimes reported or the number of people presenting at hospital. For those pitching for money it may rest on their success on gaining funding, developing their project or achieving their aims in relation to an improved community. For those providing the financial support for the PB event, measurement may rest on the number of events held, the numbers attending or whether, in the case of elected representatives, whether they are re-elected? So do we mean evaluation for the funders, the planners, the bidders, or the community? All I would hope but most importantly the community.

The discussion initially focused on how we measure in silos and moved onto how academia can help (they may have resources in terms of students conducting research as part of their studies or established processes for measuring social science programmes). One example of this is a series of projects funded by the N8 group of universities (http://www.n8research.org.uk) focusing on Citizen Engagement. Durham University, also a member of N8, recently developed a process of measuring social capital and applied it to a programme being led by MutualGain with Durham Constabulary.

The cost of evaluation was raised: Durham Constabulary and many police forces have developed relationships with universities, usually at a local level, with a view to providing research opportunities. This was something that the academics in the room highly valued, and maybe something you could explore as you start to develop your PB process.

Internationally, measuring the value of PB remains a nut to be cracked. People from New York, Sweden and Italy all shared their views that the PB process differs across continents,something that Giovanni Allegretti, referred to in his keynote speech: the consistent factor, measurement, remains elusive.

All of those in the discussion group recognised the potential of measuring success through stories rather than target achievement. Stories are a powerful tool, especially when they relate to significant change to individuals, groups or communities and most of those present were able to share a story of success as a result of a PB process. This of course is unlikely to satisfy the bureaucrats and statisticians or indeed those academics who prefer quantitative research over qualitative research. So perhaps a middle ground needs to be found.

This may come from a focus on the social return on investment. This process examines the requirements of all stakeholders rather than separating them, and creates a value to the relevant stakeholders (specifically the end user or community). Activities are measured using a consistent method and outcomes and outputs are charted for all stakeholders. The result is a measurement on the social value achieved for all involved rather than a binary target.

Social return on investment is a dynamic process that once mastered can provide a far more holistic overview of success that is applicable to all stakeholders. There was a general agreement that this could be a way forward and many asked for further information (http://socialvalueuk.org/training/practitioner).

There was also a recognition that the UKPB Network3 has information and guidance in the shape of case studies and metrics developed in the USA in relation to measuring PB. This information provides a great ‘starter for ten’ and enables the process of measuring success to commence with some level of confidence.

So let’s get measuring and share the stories, the values and successes to enable us to learn across continents.

Written by Dr. Andrew Fisher http://www.pbpartners.org.uk/andrew-fisher/