ABS, 2016 Census and the same-sex marriage survey: an insider’s look at lessons learned

The Australian same-sex marriage survey was the centrepiece of one of the hottest debates in Australian politics in 2017, and the Australian Bureau of Statistics was tasked with its execution.

The Commonwealth Government gave the ABS 99 days to conduct the national survey, and made the request a year after the 2016 Census, which suffered from technology failures and confusion, had delivered a blow to the ABS’s stellar reputation.

But it was that Census which gave Duncan Young – ABS program manager for both the Census and the Australian Marriage Law Postal Survey – the insights needed to successfully implement a postal survey which sent forms to 16 million people.

Mr Young is an ANZSOG graduate, and he recently provided insights into the Census and same-sex marriage survey at an ANZSOG function.

“Exactly one year on from Census night, a number of Census colleagues and I had planned to get some quiet drinks,” he said.

“The Prime Minister had different plans. “We had 99 days to design, develop, conduct, process and publish results from the survey, in an environment that was high-profile and highly political.”

Census Lessons Learned

The ABS delivered the postal survey accurately and on time, and the 80% participation rate was higher than almost anyone predicted. 

In addition, the survey came in $40 million under budget.

The acclaim the ABS received was a world away from the technical issues that saw the Census website shutdown for 43 hours in 2016, and briefly saw the #Censusfail hashtag trend world-wide.

Mr Young says that the lessons learned from the Census were crucial in ensuring the postal survey went off without a hitch.

“There were real issues: our defence against denial of service attacks didn’t work, public concern was raised. We received 3.5 million phone calls, and weren’t able to deal with all of them so the service we provided to customers was compromised.”

Mr Young said three focal points emerged from the Census:

stakeholder engagement
public trust
and risk management.

Engagement led to improvement

Mr Young says the successful implementation of the postal survey in the 99-day timeline highlighted that they’d learnt from the Census.

“We had to get everything right the first time. Engagement began literally on day one. We went to stakeholders, such as Council of the Ageing, Federation of Ethnic Councils Australia, Vision Australia and many others, to contact them about the design process.

“It made it clear that the stakeholder’s views were important, led to improvements in design and built a stronger relationship.”

The ABS also went to the AEC to learn how they would have conducted a plebiscite, and the ABS also added 10 staff from other agencies to bolster the ABS core team of around 90 full-time staff.

The design of the survey, like the Census, leveraged behavioural economics to maximise and encourage participation. A total of 99.84% of respondents provided a clear response, and more than half the nation are estimated to have responded to the survey within eight days. 

The ABS also had an advertising campaign live within 36 hours and an information line with a supporting call centre operating in four days.

Building Trust

To improve public trust in the survey process, and its implications for privacy, the ABS engaged the services of former Australian Privacy Commissioner Malcolm Crompton to review the design and conduct of the survey.

Mr Young said there were numerous benefits to having respected, independent sources of information to validate ABS’s processes, particularly when the organisation was trying to strike a balance between efficiency, integrity and privacy in the way it conducted the survey.

“From a communication point of view, if you don’t trust an organisation, listening to that organisation’s spokesperson won’t reassure you. But this is not just a protection exercise, it gave us opportunities to refine what we were doing and improve our practice.”

Being agile and responding to risk

The postal survey required the ABS to work with agility and collaborate with 30 agencies.

They adopted twice-daily stand-up meetings for the entirety of the program, including all key decision-makers, and a ‘Kanban’ board to visualise the work. Importantly, these techniques allowed activities to be coordinated and progressed simultaneously, while escalating issues that required immediate attention. 

Given the Census experience, there was a renewed focus on identifying and dealing with risks, which Mr Young said resulted in a constant emphasis on residual risk exposures and how to reduce them.

“This risk register, containing around 50 risks, was used at least weekly by program leadership to guide where work should be undertaken to try to mitigate or reduce the impacts of risks. 

“When you have 16 million forms to send out, even a one-in-a-million possibility is going to happen 16 times. I think we were prepared for everything that happened – whether it was lost forms, or forms allegedly being sold online, which we were able to shut down quickly by working with online retailers,” he said.

Mr Young was part of the 2010 cohort of ANZSOG’s Executive Master of Public Administration, and said that the course had equipped him well for the challenges of his position.

“The frameworks that you learn become part of your weekly, if not daily routine, and your way of making sense of the world you operate in.”

“The course also gives you a strong network, and I used a lot of my contacts to help me with our work on the postal survey”

He says that the experiences of the last two years have strengthened the ABS and given staff confidence in their ability.

“We are working towards another digital census in 2021, with a more user-centred approach, and a focus on collecting information quickly and accurately. And if the government asks us to do another postal survey, then we have the capacity to do it.”