James Veysey/Shutterstock
The UK’s Office for National Statistics (ONS) privately discussed concerns about the risk of its flagship covid-19 infection survey offering a biased picture of the country’s epidemic, according to documents from between May and July.
The survey is considered one of the best insights into the prevalence of the disease, as it samples random households in the UK’s four nations. However, New Scientist has previously reported that the response rate has fallen as the survey has expanded.
Of the households in England invited to take part, the response rate dropped from 51 per cent in April to 5 per cent in October, raising fears that if only a certain group of people were completing the survey, it may become less representative of covid-19’s real prevalence in the community. In October, the ONS told New Scientist that the survey was weighted and continued to offer reliable estimates of the virus’s spread.
Advertisement
Now documents released under freedom of information rules show that the government body was sufficiently concerned about the risk of bias from falling response rates that it discussed action to address the issue.
“Lockdown has eased and public attention has begun to shift onto returning to work and ‘the new normal’. Within this context, both household response rates and the average number of respondents in a household have fallen, increasing the risk of non-response bias,” says an annex to the contract between the ONS and IQVIA, its contractor for the survey.
The annex discusses the need for action to reverse the falling response rate and reduce the bias from failures by households to respond. “Particularly interested in immediate quick wins,” its says. Meetings between ONS and IQVIA were proposed for late July and August to discuss measures such as nudge letters and texts, given what officials called “the urgency” of the matter.
Contacted by New Scientist about the documents, the ONS said only some initial work was undertaken because the risks didn’t materialise, as shown by an October target of 150,000 unique fortnightly tests being reached early. After reaching a low of 5 per cent in October, the response rate has climbed in England to 12 per cent by 11 December.
That is still exceptionally low for an ONS survey, as response rates ideally hit greater than 80 per cent, says Sheila Bird at the University of Cambridge. She says the survey is a “huge undertaking” but the ONS should be “a lot clearer” about how it is addressing the danger of bias when reweighting results to make them reflect the wider population.
Elsewhere, the documents list key performance indicators for IQVIA, including “time to book appointments”, “completion of booked appointments” and missed calls being returned within one working day.
But people taking part in the survey deluged IQVIA with complaints in September and October, with participants unable to get through to call centres, calls not being returned and no-shows for booked appointments. Those issues appear to have continued into November and December, with people taking to social media to report ongoing problems contacting IQVIA and missed appointments.
The ONS said the survey continued to provide reliable estimates of the epidemic. “The Coronavirus Infections Survey has met or exceeded all of its key performance targets and remains a vital source of timely data on Covid-19 infections and antibodies across the UK,” it said in a statement.
The documents also reveal when the ONS forecasts its survey will be rendered unnecessary as the country’s epidemic recedes. Tables show the body expects to have zero unique subjects per fortnight by March 2022, two years after the survey began.
Sign up to our free Health Check newsletter for a round-up of all the health and fitness news you need to know, every Saturday
More on these topics:
