The Innovation Barometer
The Innovation Barometer is the world’s first official statistics on Public Sector Innovation (PSI), now implemented in 5 countries, providing innovators and decision makers with systematic knowledge of what thousands of innovators 'actually do’. The barometer advances PSI as a tool for solving societal problems and is used in practice for: inspiring innovation work, policymaking, strategising, executive leadership development, teaching, research and consultancy services.
Private companies have been the subject of internationally comparable statistics on innovation for nearly three decades, giving private companies, scholars and public sector decision-makers essential guidance for business development, research and policymaking. For the public sector, however, anecdotes and opinions have been substitutes for statistical data on innovation. It has left public innovators and decision makers without a solid knowledge base. It has also left public sector innovation as such without data-based legitimacy.
Paradoxically, the need to implement, systematise, prioritise and scale public sector innovation has never been greater, as complex problems facing the public sector seem to continually grow in number and scale. Demographic changes, climate crisis, cybercrime, budget deficits, diminishing political legitimacy just to name a few. The widespread understanding that large pools of data, used in other contexts such as machine learning and artificial intelligence, has increased tremendously in value exacerbates the paradoxical lack of systematic data on PSI.
Objectives and solution
Determined to end the data deficit, in 2015 the Danish National Centre for Public Sector Innovation (COI), in association with Statistics Denmark, began separating myth from reality. One objective was to create a new ‘public good’, a pool of data provided to a diverse set of actors to use for their own purpose without reducing its availability to others. This would scale and diversify the impact on the public sector capacity for innovation much more effectively than simply adding the work hours of the (then) 4 COI employees to the existing 800.000 public sector employees.
The result was the Innovation Barometer, the world’s first official statistics on PSI. By 2018 Norway, Sweden, Iceland and Finland had all conducted one or more national surveys, utilising similar methodologies and definitions, though adapted somewhat to better serve national agendas. Their ongoing efforts have also contributed to methodological adjustments, improving the original survey design.
When development of the Innovation Barometer for the public sector began, in line with the desire to benefit from private sector experience, the team looked for guidance in the OECD’s Oslo Manual: Guidelines for Collecting and Interpreting Innovation Data, 3rd Edition, 2005. This focuses on innovation in the private sector, and recommends measuring innovation at the level of the smallest legal units with some authority. In the context of the public sector this means individual workplaces like kindergartens, nursing homes and schools.
Public sector workplaces were asked whether they had introduced an innovation over a two-year period. The team applied an adapted version of the definition of innovation used in the Oslo Manual, replacing, for instance private sector marketing innovation with public sector innovation in communication. Innovation Barometers define public sector innovation as new or significantly changed:
• processes or methods of organisation
• products or
The innovation must be new to the workplace, but the workplace does not have to be the original inventor. The innovation can also be copied from others or inspired by others’ solutions.
The innovation must have created one or more types of value, such as:
• increased quality
• increased efficiency
• citizen involvement or
• employee satisfaction
The Innovation Barometers cover a range of innovation topics, such as type of innovation, reusing innovation, collaborators, financing, spreading innovation, value created, evaluation, drivers and barriers, and workplace culture.
The Nordic Innovation Barometers show that large majorities of public workplaces are innovative, i.e. they have introduced one or more innovations over a two-year period. In Denmark, Iceland, Norway and Sweden approx. four out of five public sector workplaces are innovative, while 95% of Finnish municipal workplaces are. The findings also demonstrate a high capacity for collaboration, employee driven innovation, adapting and copying and points to solutions when it comes to challenges of evaluation.
The barometer is a public good, shown in practice to benefit innovators, decision makers, private consultants, HR people, cluster operators and interest groups. The barometer is used for inspiration for innovation work, policymaking, strategising, executive leadership development, teaching, studying, research, consultancy services and public affairs. The barometer gives insight into sector-specific differences between public and private sector, helping innovators in both sectors to learn more from each other.
Due to international interest, work on a “Copenhagen Manual” on how to make your own Innovation Barometer has begun. Delegates from 19 interested countries, most of the OECD members, participates in the work: https://www.innovationbarometer.org/cphmanual/.
What Makes Your Project Innovative?
The Innovation Barometer is the first fully representative national PSI survey, spanning all levels of government, in compliance with international quality standards for statistics.
The barometer differs from most statistics, as respondents are essentially end users - public innovators or non-innovators seeking to improve. This is achieved via a practical approach. Although the Oslo Manual was a starting point, every step is guided by the needs of those who will use the results. Questions are selected and tested to fit the interest and real life experiences of public employees rather than theoretical frameworks.
Following the user-oriented approach, 300 innovators were involved in interpreting and communicating results, using their own SoMe profiles, as the team helped them with infographics, photos and cases. Books, videos, conferences and a steady flow of new analyses combined with media strategies led to +200 news articles. Media coverage now occurs without COI doing anything.
What is the current status of your innovation?
Nordic data collection or planning thereof has reached 2nd (Norway 2019, Sweden 2021) and 3rd round (Denmark 2020), enabling comparison over time and across borders. Yet, the most significant development is the growing global interest and demand for practical guidance. To meet the demand, +50 actors from 19 countries are currently co-creating the Copenhagen Manual (www.innovationbarometer.org/cphmanual) - a user-oriented guide on how and why any country could benefit from making an Innovation Barometer. Besides the Nordics, the process has participators from Austria, Canada, Czech Republic, Germany, Greece, Ireland, Israel, The Netherlands, New Zealand, Poland, Portugal, Spain, Switzerland, Wales, OECD and the European Commission. Copenhagen Manual participators from Germany, Austria and Switzerland have tested an adopted version the Innovation Barometer - Innovationskompass - in 2019, planning for nationwide surveys in 2020. https://www.innovationskompass.net/hintergrund.html
Collaborations & Partnerships
1st version: COI, Statistics Denmark and University of Aarhus combined competencies in innovation, user involvement, survey design and statistics. Questionnaire development included 7 workshops, engaging +100 citizens, public sector employees and innovation specialists. Later COI assisted 7 Nordic adaptors - ministries, innovation authorities, municipal organizations (who have joined forces in The Nordic Public Sector Innovation Hub) – who further developed and improved the original design
Users, Stakeholders & Beneficiaries
A large variety of people and organisations use barometer data, applying them for their own purposes, e.g. practical inspiration for innovation work, policymaking, strategising, executive leadership development, teaching, research, consultancy services and public affairs. Or for legitimizing certain decisions and criticising others. In short, the Nordic Innovation Barometers are put to use as the public good they were intended to be, also in ways the developers and adaptors did not foresee.
Results, Outcomes & Impacts
A telling end user example: Ulla Egholm, the head of a municipal daycare centre, wrote in a review of one of the books based on the barometer: "The Innovation Barometer: New Together Better' allows you to dive into the different chapters according to what and where in the innovation work, or where you are interested, or need to read what others have succeeded - to be inspired in your own innovation process. There are many instructive practice descriptions that reveal how others have thought innovatively and imaginatively in solving their challenge. The book can be used as a reference in every part of the public sector, and for all kinds of innovation inspiration. There are plenty of useful tools. I think this book is going to be in my "work library" as a reference I use when we are working on innovative projects, or whenever I need to read more about where and how others have succeeded with new ideas".
Challenges and Failures
We underestimated the resistance to measurements. Many public sector employees find themselves overburdened with measurements and requirements for registrations for which they see no purpose. ‘And now we would introduce yet another one’. Various people feared that the public sector would be stigmatized as not at all innovative, that the survey would reveal that public employees were against change or that benchmarking would put some municipalities on display in a bad way.
The team reached out to our critics and learned from their concerns. More had to be done to avoid 'strategic answers', to invoke the interest of the individual respondent and to communicate the team's intentions. The survey was made voluntary and anonymous. Invitation emails were segmented so that each respondent got a personal email with her full name.
Later, when data showed that public workplaces are often innovative and that employee-driven innovation is widespread, critics became fond users of Innovation Barometer data.
Conditions for Success
You need to never stop thinking of your respondent as a potential user of the results. Why should a busy head of a school, library or nursing home answer? What should she be able to use the results for herself? Be ready to disappoint any innovation expert's dream of data if your user involvement tells you so.
You also need to set up a team or form a partnership where a mix of competencies for innovation, user involvement, survey design, statistics and strategic communication are continuously present.
You should expect resistance from strong and well-organized critics and be able to create a dialogue and understanding with them.
It is important to determine early what the specific purpose of your Innovation Barometer is. What agenda do you hope to set? Use your purpose to create interest and potential collaborations with other stakeholders.
To get the most out of your investment, make it easy and free for others to use your data for their own purpose.
The Innovation Barometer has been developed further by the Nordic countries using it also in subsequent rounds of the original Danish survey. Adding questions to the survey to focus more on topics important in a current national agenda is quite straightforward, although modifying essential questions from the original questionnaire complicates international comparisons. Recommendations on what questions to keep alike will be part of the Copenhagen Manual.
The most significant development made to the Innovation Barometer is that the second round of the Norwegian survey includes a parallel survey targeting local and regional politicians that play a crucial role in enabling local and regional public sector innovation. In turn, this has inspired Denmark to do a similar survey (February 2020) targeting Danish local and regional politician.
The interest in putting work into the co-creation of the Copenhagen Manual from 14 non-Nordic countries indicates a global potential.
The difficult part of the Innovation Barometer is to define and reach the respondents. The survey is conducted at a workplace level, but finding the workplaces can be hard. Reach out to a national statistics agency to explore whether registry information on public sector workplaces is available. If so, also explore the opportunity to merge data from the survey with existing registry data on workplace size, location, etc.
To reach a high response rate for the survey, the invitation email should address the manager of each workplace by name via the manager’s direct email address. When data collection have relied on the general administration within an organisation forwarding the invitations to the relevant managers, the response rate has been lower, as the forwarding practice involves a risk of the survey never reaching the intended respondent. Finding the contact information of all respondents is time consuming, but pays off in terms of a higher response rate.
Make sure to test the questionnaire on potential respondents. It is crucial that questions make sense to respondents in your national setting. Your time is better spent testing the questions than overthinking the them theoretically. Get a head start like this: Translate the exiting questionnaire into your own language using a free online translation service, email the document to 5 public employees you trust, and have them answer using a pencil. Call them up and ask if it makes sense. This will give you a lot of practically usable knowledge as you know have done testing.
Involve stakeholders from the beginning – especially those that might be opposed to the survey, as conflict might be an obstruction. Try your best to reach a compromise beforehand.
Finally, to make full use of the survey results a lot of communication efforts are needed. It is advisable to spend more time and resources communicating the results than conducting the survey, as there is no reason to do a survey that no one will know about.
Sharing your innovation with others takes time, but it multiplies the value of your work as others take it upon themselves to further develop your innovation and test its robustness in new contexts. It also makes your own work much more fun to be challenged by new insights, though it will mean mild annoyance like 'why didn't we think of that ourselves?'. Certainly, the value and global scaling potential of the Innovation Barometer would never have reached its current level had it not been for the adopting processes carried through by Association of Finnish Local and Regional Authorities (FI), Icelandic Association of Local Authorities (IS), Ministry of Finance and Economic Affairs (IS), Norwegian Association of Local and Regional Authorities (NO), Agency for Public Management and eGovernment (NO), Swedish Association of Local Authorities and Regions (SE) and Vinnova (SE).