ANZSOG Research Report examines how Public Opinion Data shaped policy during COVID pandemic

As the COVID-19 pandemic unfolded, the significant role of Public Opinion Data (POD) in shaping government responses became increasingly evident. 

A new ANZSOG report, Insights to Action: An Analysis of the COVID-19 Pulse Survey, co-funded by the Australian Public Sector Commission (APSC) and undertaken by Monash University’s Sustainable Development Institute, presents an analysis of the national Prime Minister and Cabinet (PM&C) COVID-19 Pulse Survey conducted from March 2021 to March 2022. 

The report is the second output from ANZSOG’s Research Model project Trust, Transparency and the Use of Data in Informing Policy Responses, which is developing  best practice guidance for public sector collection, evaluation, and use of Public Opinion Data (POD), for the purpose of developing public policy that integrates, and gives proper weight to, the opinions of those affected by it. 

It offers suggestions to ensure that POD obtained during future crises can be better integrated into policy, balancing the need for providing policymakers with timely and relevant information with retaining methodological credibility and trust. 

The Pulse Survey collected nationwide data on public attitudes, offering insights into the public’s response to the pandemic and the vaccine rollout, and providing a potential tool to help shape policy responses. 

The Report focuses on two main areas.  

First it examines the collected data to assess the evolution of vaccine hesitancy across various demographics over time. The data demonstrates the potential value of POD in informing government responses.  

Secondly, it explores how the insights from the Pulse Survey influenced policymaking in practice by examining the survey’s design process and its role in informing decisions or shaping policy. Interviews conducted with senior policymakers involved with the survey reveal that the survey was designed to quickly capture public attitudes during the COVID-19 crisis with no specific policy purpose in mind. The interviews show that the survey was designed to meet the crisis context, rather than for strict methodological robustness, and its impact on policymaking in this case could not be definitively established. 

The authors conclude that, when using POD, policymakers must balance agility and innovation against deliberate and methodical processes. Even in crisis contexts, there is a need for robust methods that: 

• Align POD instruments with the needs of policymakers
• Produce credible results
• Leverage POD findings as inputs into policymaking

Overall, in explaining the challenges of leveraging the Pulse Survey data for policymaking, the report finds that POD is a critical feedback mechanism during crises but that it also: ‘exposes the potential complexities involved in leveraging such data effectively. One challenge is ensuring the representativeness and appropriateness of the survey design while addressing evolving informational needs under time and resource constraints’.  

While the Pulse Survey data provides essential insights into the ‘what’ and ‘who’ of vaccine hesitancy, the report underscores the need to employ mixed methods to delve deeper into the ‘why’ – a dimension not fully captured by quantitative measures alone.  

The report finds that in the context of the COVID pandemic, the Pulse Survey’s anticipatory and agile approach to information gathering and identification of intended users aligns with the best approach to decision-making in chaotic situations. 

“However, our Pulse Survey interviews and data analysis suggest that these proactive measures did not sufficiently evolve to meet specific policy needs. While gathering broad data helped to understand the situation, it lacked the depth to significantly influence decisions, highlighting the need for more focused, actionable insights,” the Report finds. 

“Moreover, the feedback mechanisms were inadequate, as evidenced by the design team’s limited understanding of how insights were utilised. This disconnect suggests a need for clearer communication between decision-makers and survey designers to ensure data meets evolving needs.” 

The report suggests that key areas for future discussion to improve the usefulness of POD include: 

• Establishing robust feedback mechanisms: Ensuring that POD is aligned with decision-makers’ needs during crises. 
• Balancing methodical and agile approaches: Maintaining credibility and reliability in data collection while being responsive to urgent needs for actionable insights. 
• Strengthening relationships between researchers and policymakers: 
• Facilitating collaborative partnerships that leverage evidence for effective problem-solving 

The full report is available here. 

The first output from the POD Research Project, Use of Public Opinion Data to Inform COVID-19 Policymaking, a rapid evidence review of the utility of public opinion data, is available here.