Policy labs have emerged globally with a mission to support public policy development through innovative solutions. However, it is unclear to what extent policy labs provide new ways of addressing challenges.
At a glance
A paper in Policy & Politics explores the role of policy labs and assesses the extent to which their purpose, structures and processes address three challenges:
- establishing the causality and value of public interventions
- explaining mechanisms of change
- using research findings in public policy.
The article concludes that creating synergies between evaluation inquiry and policy labs can improve the design and implementation of public policy and programs.
Related research:
- More ethical, more innovative? The effects of ethical culture and ethical leadership on innovation
- The dark side of public innovation
- The intensity of innovation in public sector organisations
About the policy labs
The study focused on labs with a focus on:
- policy design
- solutions for citizens or social impact
- providing an open forum for new ideas and solutions to social problems.
The research covered 20 labs in 16 countries across Europe, North America, Latin America and Asia, including the NSW Behavioural Insights Unit and the Auckland Co-Design Lab.
Structures: who labs are
Half of the selected labs are exclusively owned by public sector entities representing different levels of governance. The labs operate at local, regional and national levels and some are also active at the international level.
The experience of the labs varies from two to 23 years. Labs’ regular partners include local and regional authorities, public administration institutions and private stakeholders. Civil society organisations are also a key partner for labs as they represent wider groups of citizens or specific social interests.
Functions: what labs do
Three main lab specialisations were identified:
- serving the government: improving public services/policies and training public sector staff
- serving the community: solving specific social problems such as health, poverty, urban development and gender equality
- serving both the government and the community.
The study found limited evidence on labs’ longer-term impact and the measurement of meaningful change. Instead, labs focused on the short-term and in some cases, process-related outcomes.
Processes: How labs do things
The processes and toolboxes of policy labs are heavily based on design practice including design thinking and industrial design. The study mapped labs’ activities and found five main areas accompanied by a variety of tools and techniques.
The five areas are:
- redefining a policy issue
- exploration to fully understand the problem
- analysis and hypothesising about the factors/contexts that help or hamper the desired policy change
- undertaking creative sessions to designing potential solutions
- providing a safe space for experimentation and testing the prototypes of designed solutions.
All labs use participatory orientation, involving representatives of various stakeholders in the exploration and co-creation of solutions. Methods include ethnographies, interviews, workshops and crowdsourcing. This human-centred approach welcomes citizens’ ideas and solutions, embraces ambiguity and identifies the conditions for co-design and implementation. Some labs also highlight the need to involve participants during the prototyping and testing stage to increase the take up of solutions and empower beneficiaries.
Behavioural insights was another methodology used. This uses insights about human motivation and decision-making to redefine policy problems. Exploration involves mapping behavioural bottlenecks – the factors that obstruct desired behaviours. Creation uses behavioural change strategies in the design of potential solutions. The testing stage relies on small, low cost interventions accompanied by experiments, usually randomised control trials.
Labs and the challenges of evaluation practice
1. The challenge of establishing what solution works
Looking at current lab practices, the study found labs have little to contribute to the ongoing debate addressing the challenges of valuing interventions. They could benefit from incorporating existing knowledge from evaluation into their practices as evaluation practice offers a high level of emphasis on obtaining strong causality.
2. Explaining why a solution works or not
Both evaluation practice and labs use a theory-driven approach that defines public intervention as a ‘theory’ – a hypothesis yet to be tested and verified in a real-life situation. The value of the theory-driven approach is its explanatory dimension. It reveals how a program works, with whom and under what circumstances.
The use of behavioural insights also helps explain why a solution works or not. The basic strategy is to break down the ‘journey’ of the policy actor throughout the intervention into a sequence of interactions and decisions, and then analyse behavioural bottlenecks. This approach:
- allows exploration of observed behaviours and their drivers
- clarifies diagnostic findings in the form of user profiles and their decision journeys
- links policy practice with the growing body of psychological and cognitive research on human decision-making.
3. The challenge of transferring research findings into policy actions
Labs’ scope of operations provides an opportunity to better utilise research by addressing the issue of timing. Most labs focus on working at the early stages of the policymaking process – problem definition and policy formulation. Within the main policy cycle, labs often try to build a smaller loop of design–testing–adaptation. This allows them to deliver meaningful feedback on ‘what works and why’ exactly when decision-makers need it – early in the policy process.
What this means
Well-established evaluation practices can support labs in tackling issues that require a more sophisticated understanding of the social dynamics and environments surrounding a certain policy intervention. In particular, evaluation brings together various perspectives of causality and complexity theories that can help labs measure their impact and understand how systems change over time as well as from place to place.
Lab practices can add to evaluation practice with their strong focus on the final users of policy. Some labs seem to be ahead of evaluation practice in terms of integrating behavioural insights into public policy. Policy labs seem to be better equipped than evaluation in addressing the timing issue of delivering results to knowledge users.
Want to read more?
Policy labs: the next frontier of policy design and evaluation? – Karol Olejniczak, Borkowska- Sylwia Waszak, Anna Domaradzka-Widła and Yaerin Park, Policy & Politics, Volume 48, Number 1, January 2020
Each fortnight The Bridge summarises a recent piece of academic research of relevance to public sector managers.
Recent Research Briefs include:
- Published Date: 8 September 2020