To improve the use of evidence in policy and practice, many organisations seek to promote research-policy engagement activities, but we know little about what works. An article in Evidence & Policy examines the impacts of research-policy engagement on decision making. It found these initiatives tend not to draw on existing evidence and are mostly unevaluated. They therefore fail to provide useful lessons for those wishing to improve evidence use, leading to wasted time and resources.
The role of engagement
For over four decades, researchers have written about how evaluation and research evidence is routinely ignored by decision makers. The perceived failure of decision makers to use evidence has led researchers to investigate barriers and facilitators of evidence use and conceptualising the ‘evidence-policy gap’, which includes bridging interventions.
Responding to this perceived failure to use evidence, organisations have sought to promote greater engagement between researchers and policymakers. Engagement is often taken to mean greater interaction at the interpersonal (networking events) or inter-organisational (secondment schemes) level. With many approaches available to promote to research-policy engagement it is a challenge to identify the most effective approaches to achieve particular outcomes.
About the research
The research involved a large-scale mapping exercise to identify:
- What research-policy engagement activities are being used with the goal of improving evidence use?
- What is known about the impacts of these activities?
Eight types of organisations were included from the UK (research funders, learned societies, universities, intermediaries, policy organisations and bodies, practice organisations and bodies, think tanks and independent research organisations, non-profits, and for profits/consultancies), and five from overseas (research funders, universities, learned societies, intermediaries and policy organisations).
The research identified 513 organisations globally who have been, or are currently, promoting research-policy engagement in over 40 countries. These organisations undertook over 1923 engagement activities across multiple policy and practice areas.
What the research found
Engagement activities included: disseminating and communicating research; facilitating access to research; building decision-maker skills; building researcher skills; building professional partnerships; and creating infrastructure and posts.
The majority of activities fell into the category of disseminating and communicating research with investment increasing since the late 1990s. This has meant attempting to increase the impact of one piece of research or pulling in evidence in direct response to a policy or practice need.
The research found total of 57 evaluations had been undertaken. These were of varied quality and most focused on one activity within one organisation. Some were independent and robust, but mostly evaluations took the format of annual or ‘end of project’ summaries, which described only selected aspects of research-policy engagement activities carried out by the organisation.
The research found a dual design failure: unclear aims, and a lack of appreciation of the policy and practice contexts within which the projects are attempting to operate. Most initiatives appear to address the assumption that decision makers do not listen to evidence, which is still widely held despite increasing evidence to the contrary. These initiatives are targeting a problem which may not exist. This means that when designing initiatives, many organisations may have an inappropriate or a poorly articulated goal.
Most initiatives refer to vague goals such as ‘research impact’ or ‘policy change’. This lack of specificity hinders quality evaluation and very few of these initiatives had evaluations available in the public domain. Because the existing evaluation evidence is so scarce, it is not possible to say which types of research-policy engagement activities will lead to which types of impact.
That does not mean that it would not be possible to design evidence-informed research-policy engagement initiatives. An obvious next step would be to more fully synthesise the evidence to tease out the context-specific lessons for each. There is clearly scope to further build the evidence base to capture the learning including building rigorous evaluation into proposals for engagement activities.
The bottom line
Overall, the picture is of a vast and increasing mass of rudderless activity, which is busy rather than effective. Without clear goals, and without strategic coordination, it is impossible to pick out any signal over the noise. Worse, without clearly collecting and building on existing evidence about these types of interventions or on a well-founded understanding of the decision-making context, there are almost certainly harms being inflicted. Harms are likely to include wasted time and resources, reduced goodwill and interaction, and increased inequalities in terms of participation in evidence production and use.
Want to read more
What works to promote research-policy engagement? – Kathryn Oliver, Anna Hopkins, Annette Boaz, Shannon Guillot-Wright and Paul Cairney, Evidence & Policy, February 2022.
Each fortnight The Bridge summarises a piece of academic research relevant to public sector managers.
Recent Research Briefs on policy development include:
- Published Date: 19 June 2024