Analysing policy success and failure

An article in the Australian Journal of Public Administration considers four dimensions in evaluating policy failure/success—process, goal achievement, distributional outcomes, and political consequences. It applies this framework to two Australian Commonwealth government programs and identifies the danger of ignoring subject matter expertise and poor policy/Cabinet processes as contributing to policy failure. It also acknowledges the danger of central agencies having too much weight in the policy-making process, as they seek to achieve their ministers’ political objectives.  

Evaluating success and failure

The article focuses on a success/failure framework with four categories: process; goal attainment; distributional outcomes; and political consequences. Identifying whether a program is a failure or success allows consideration of what caused that outcome. Elements include individual decision-makers; institutions/policy processes; and deeper societal values and power structures/interests. 

About the research

The article assesses the success/failure of two Australian government programs: 

• the Home Insulation Program (HIP) 
• the Digital Switchover Household Assistance Scheme (HAS). 

Both were developed around the same time with similar program designs and involved government intervention in a market in which the government would not usually be involved. Yet HIP resulted in four deaths, more than 200 fires and a Royal Commission. 

The research included an analysis of primary documents, a review of the existing literature and interviews with former public servants and political advisors. 

A case study of policy failure

HIP was an Australian Government response to the 2008 Global Financial Crisis. The program part of a broader economic stimulus package and it subsidised the installation of insulation in homes. 

The program was a failure across the four dimensions of the success/failure framework.  

The program’s dual objectives of stimulating the economy and reducing greenhouse gas emissions became competing objectives, with the stimulus objectives overriding all others, including health and safety. The stimulus objective meant that timelines were the key factor in most decisions on how the program was designed and this time pressure contributed to the policy failure.  

Other contributing issues were institutional/policy process centred. This related to the Cabinet and policy development process failures, as well as cultural issues within the Australian Public Service. This was a fear of fear of giving bad news to the minister. In the case of HIP, the public servants failed to inform the relevant ministers about the major risks associated with the timeframe.

A case study of policy success

Digital Switchover HAS was part of a broader program to switch from analogue TV to digital TV. The government faced the question of whether they would actively assist vulnerable cohorts to transition from analogue to digital. 

Having decided they would provide some form of assistance, the government opted for in-home assistance, with a trained installer coming to eligible people’s homes and installing a Set Top Box. The target group was people who needed technical assistance but could not afford to pay for it themselves—those on the maximum rate of certain Centrelink payments. 

The program had some level of success across most of the success/failure dimensions. 

The program was a failure across the four dimensions of the success/failure framework.  

Factors that contributed to the program’ success included a staged/phased rollout, installer/product regulation and training and demand-side controls. 

The bottom line

Institutional and policy process issues played a key role in both HIP and HAS. Both programs were developed under the same chaotic Cabinet processes but there were different outcomes. For HIP, the decision-making was rushed and disjointed, without any proper policy development process underpinning the program. For HAS, the internal policy development process had been careful, deliberate, and thorough which offset part of the risks associated with poor Cabinet processes. 

A key difference was in the public service leadership. The HIP was designed and implemented by career public servants, who may have bought into the ‘don’t tell the Minister’ culture. On the other hand, the head of the Digital Switchover Taskforce was external to the Australian Public Service (APS) and came without cultural baggage. 

Another comparative lesson is the danger of central agencies having too much weight in the policy-making process, as they seek to achieve their ministers’ political objectives. This is often at the expense of the subject-matter and service-delivery experts in line agencies. 

The problems associated with poor institutional culture, poor policy and Cabinet processes, and problematic leadership created risks across all programs. However, as with all risks, whether the risks result in problems depends partially on chance and partially on how effectively the risk can be managed and mitigated 

Analysing policy success and failure in Australia: Pink batts and set-top boxes – Daniel Casey, Australian Journal of Public Administration, August 2024 

Each fortnight The Bridge summarises a piece of academic research relevant to public sector managers. 

Sign up to The Bridge