Behind the scenes of the coronavirus dashboard: a record 76.5m views, 2bn rows of data and 100 computers

On 15 January, as Covid-19 cases, hospitalisations and deaths were reaching the peak of the third wave, the government’s coronavirus dashboard was viewed a record 76.5 million times in a 24 hour period.

What started in April as a simple page displaying only case numbers and deaths, attracting just a few thousand visitors, has turned into the most-viewed government website ever since gov.uk was created in 2012. On its busiest days, it handles 1.2 million unique users.

Every day at 4pm, when the latest figures are published, coronavirus.data.gov.uk accounts for 50 per cent of all of the government’s online traffic. As many as 70,000 people are online at that time refreshing the page, waiting for good – or bad – news.

Lighthouse in the darkness

The key to the dashboard’s popularity seems to be, amid all the uncertainty about what will happen next with lockdown, in the regular publication of charts and graphs showing national and local trends of the UK’s experience of the pandemic – a sort of lighthouse showing the way over often very dark seas.

While there are heated debates about policy, when lockdown should end and whether ministers have got things right or wrong, there is no arguing with cold, hard data.

In their first newspaper interview, two of the key people behind the dashboard have revealed to i the extraordinary human daily effort it takes to produce the charts that have become part of our daily routine.

Clare Griffiths, head of profession for statistics at Public Health England, is in charge of a team of five analysts who collect data from 26 different sources and feed it into a “pipeline”, while technical and development lead Pouria Hadjibagheri and one other colleague are responsible for converting the data into the maps, graphs and tables that we see on the dashboard.

Democratised data

While ministers and scientists are able to see individual data sets before the public, the dashboard itself is an example of truly democratised, open-access data: the latest graphs and someone sitting at home in Newcastle sees the latest trends and graphs for the first time at 4pm, the same moment as Boris Johnson in his office in Downing Street does.

Asked why she believed it was so popular, Ms Griffiths says: “It’s about being consistent, saying you will publish at 4pm every day and we are committed to that… and that is what makes it trustworthy.

“Everyone knows what they’re looking at and everyone gets equal access at the same time at 4pm. The Secretary of State for Health gets access to the dashboard at 4pm.”

It is an extraordinarily small staff for such a huge amount of data: besides Ms Griffiths’ and Mr Hadjibagheri’s teams, there are around 15 others working on things like design and engaging with users to improve the dashboard.

12 hours a day

Members of the team work – all remotely from their homes – seven days a week, even on Christmas Day, to update the dashboard. Despite working closely for months, Ms Griffiths and Mr Hadjibagheri have never met face to face.

He has worked an average of 12 hours a day since the start of the pandemic in March, without a day off, and once worked two days straight without sleep, fixing issues with the system.

After starting in April with just four metrics, there are now 177, including an interactive map showing how many cases there are by local ward area.

Every day the pipeline handles more than 600 million raw records which produces an average of 40 million figures. The dashboard database now has more than two billion rows of data, stored on a server of 100 computers.

Pipeline

Ms Griffiths’ team start at 8am, when the latest hospitalisation data from the devolved administrations comes in. At 9am, cases data for England arrives. At lunchtime, figures on deaths in England are processed. Last month, daily vaccination figures for each UK nation were added to the pipeline.

At 2pm, remaining figures from the devolved administrations arrive and by 3pm Ms Griffiths’ team is ready to “push the data” along the pipeline to Mr Hadjibagheri for data visualisation on the dashboard.

He has an hour before publication – with colleagues checking for errors. Sometimes – like last Thursday – things go wrong and there is a delay, leading to hundreds of people on Twitter and via email asking where the data is.

Does the dashboard get more views when the pandemic is in advance or retreat? Mr Hadjibagheri explains the site is viewed most when major announcements are made.

Unsurprisingly, however, the public are more interested in the “good news” metric of vaccinations – this section of the dashboard is now the most popular, followed by cases, healthcare, deaths and testing.

‘We want to maintain its integrity’

The dashboard is not just used by ministers, scientists and journalists to track the UK epidemic’s progress. Mr Hadjibagheri says: “People are using the service to plan their daily activities, which is quite interesting.”

The owner of a nationwide restaurant chain told him, before the latest lockdown, they used local and regional trends from the dashboard to calculate how much stock they needed to order, based on expected restrictions and likely fall in demand. “He told me: ‘we shouldn’t order more stock for certain areas in England, and that reduced our loss’.”

How long do they expect the dashboard to continue? Mr Hadjibagheri says: “We are hoping to keep it on for as long as it is needed… It is something we are committed to. We have been doing our very best and investing our personal lives into it, so it is something we really care about, in maintaining its integrity.”