Data dashboards have been an important part of pandemic response and planning. What have their developers learned about communicating science in a crisis?
With a Postscript : “This is no time to stop tracking COVID-19”
From my COVID-19 research team (no, it ain’t over and we are still following it)
12 April 2022 – Data dashboards have been an important part of pandemic response and planning. What have their developers learned about communicating science in a crisis?
In March 2020, Beth Blauer started hearing anecdotally that COVID-19 was disproportionately affecting Black people in the United States. But the numbers to confirm that disparity were “very limited”, says Blauer, a data and public-policy specialist at Johns Hopkins University in Baltimore, Maryland. So, her team, which had developed one of the most popular tools for tracking the spread of COVID-19 around the world, added a new graphic to their website: a colour-coded map tracking which US states were – and were not – sharing infection and death data broken down by race and ethnicity.
They posted the map to their data dashboard – the Coronavirus Resource Center – in mid-April 2020 and promoted it through social media and blogs. At the time, just 26 states included racial information with their death data. “Then we started to see the map rapidly filling in,” says Blauer. By the middle of May 2020, 40 states were reporting that information. For Blauer, the change showed that people were paying attention. “And it confirmed that we have the ability to influence what’s happening here,” she says.
COVID-19 dashboards mushroomed around the world in 2020 as data scientists and journalists shifted their work to tracking and presenting information on the pandemic – from infection and death rates, to vaccination data and other variables. “You didn’t have any data set before that was so essential to how you plan your life,” says Lisa Charlotte Muth, a data designer and blogger at Datawrapper, a Berlin-based company that helps newsrooms and journalists to enrich their reporting with embeddable charts. “The weather, maybe, was the closest thing you could compare it to.” The growth in the service’s popularity was impressive. In January 2020 – before the pandemic – Datawrapper had 260 million chart views on its clients’ websites. By April that year, that monthly figure had shot up to more than 4.7 billion.
Policymakers, too, have leaned on COVID-19 data dashboards and charts to guide important decisions. And they had hundreds of local and global examples to reference, including academic enterprises such as the Coronavirus Resource Center, as well as government websites and news-media projects.
The New York City Department of Health was among Datawrapper’s clients. And Blauer notes that she has hosted regular webinars with several US mayors, walking them through her team’s metrics. She is confident, she says, that the “data informed policy”.
The architects of these dashboards put in long hours and faced considerable challenges, including incomplete and inconsistent data, misconceptions and misunderstandings about how the information was collected, and efforts to twist the messages that the dashboards present. As these data wranglers continue to try to inform individuals and public-health officials, they are learning lessons that will help to navigate the next stage of the pandemic, as well as other social and public-health issues — from crime to climate change.
Hard data
The Johns Hopkins dashboard originated at a meeting between Lauren Gardner, who studies civil and systems engineering at Johns Hopkins, and her PhD student, Ensheng Dong. In early January 2020, Dong began closely following cases of a type of pneumonia emerging in his home country, China. “He was hearing things directly from his friends and family,” says Gardner.
Dong was concerned for their well-being, and began pulling data from a Chinese website, DXY.cn. He and Gardner spent days and nights tracking the information on a Google sheet. Then they built a map to go alongside that dynamic spreadsheet and made both available to the public. “We literally decided this one afternoon and built the initial version of the dashboard that night,” says Gardner. “It seemed like a manageable, simple task, given the scale of the problem at the time. Of course, we didn’t know the scale that this would grow to.” Just weeks later, the website had upwards of 4 billion queries a day.
Gardner and Dong eventually moved the data to a GitHub repository, a cloud-based-data-storage-and-management space that maintains a file history. Their initial global map, with its recognizable red dots proportional to case counts, is still updated every hour. Blauer and others joined the effort early and expanded it with a multi-layered, interactive dashboard to help people digest the data.
Ideally, data that are this important for public health should be freely available, machine-readable and standardized. From the start, the team realized that they were not. Compiling complete and consistent COVID-19 data was “very manual and very messy”, says Gardner. “We were scrambling, collecting and validating reported data as fast as we could.” Because COVID-19 data were not yet provided on any public-health agency’s website, they looked elsewhere, including on Facebook and Twitter posts and in one-off news and media announcements. Even after agencies launched official data pages, both sourcing and formatting remained an issue. Gardner says that some of the data the team collects are still not machine-readable. “There should be a standardized way in which the data is provided and shared publicly, as well as what is shared,” says Gardner. “That would’ve made our job a lot easier.”
Blauer has been vocal, blogging about the need for greater accessibility and standardization of data, including the use of consistent categories and naming conventions for age, gender, race and ethnicity. “Demographic data is a complete mess,” she says. Racial and ethnic categories are tricky because they are regarded differently in different countries. But even in a single US state, Blauer found category definitions changed depending on whether they were linked to vaccination rates, cases or deaths. She has made creative moves to fill in the blanks,such as when her team revealed which states were and weren’t collecting race and ethnicity data. Blauer and her team have confirmed that the pandemic has had inequitable impacts. As of September 2021, for example, Black residents of Washington DC made up 45% of the population, but 76% of COVID-19-associated deaths.
The Johns Hopkins team was not alone in its struggles. Hannah Ritchie, head of research at the non-profit organization Our World in Data in Oxford, UK, recalls her efforts to copy data from PDFs. She also points to some of the consequences of incomplete and inconsistent data. For example, differences in access to COVID-19 testing data can result in inaccurate comparisons. “It can often lead you to conclude that some countries have not been touched by the pandemic,” says Ritchie. “That is just not true.”
Ritchie also fears that the gains that have been made in data collection and visualization could easily be lost before the global pandemic is over. “A lot of these data projects are seen as one-off things,” she says. “As rich countries start to get more back to normal because of high vaccination rates, for example, will they turn around and just let these projects die?” Some dashboards have already stopped their efforts. And government efforts to collect and display data in real time are slowing in many parts of the world.
Raising ‘graphicacy’
Whether they have realized it or not, people around the world have essentially been enrolled in a two-year mathematics course. Core to this curriculum is ‘graphicacy’, or the ability to understand data presented in graphs and figures. “I think the pandemic helped to bring the graphicacy of the general public to a higher level,” says Maarten Lambrechts, a data-visualization consultant based in Diest, Belgium. “A lot more people now have a better understanding of how a chart works and how they should interpret it.”
COVID-19 dashboards were vehicles for this public education. Data scientists took to mainstream news outlets and social media to explain and contextualize the visualizations they posted on dashboards. Soon, terms such as ‘flattening the curve’ and ‘log scale’ became dinner-table topics for many people.
Among the shared themes for the dashboards were simplicity and clarity. Whether you are producing visuals and analytical tools for policymakers or for the public, Blauer says, the same rules of thumb apply. “Don’t overcomplicate your visualization, make the conclusions as clear as possible, and speak in the most basic of plain-language terms,” she says.
Yet, as other data scientists point out, presenting data simply might not be enough to ensure viewers get the message. For one thing, attention to detail matters. Ritchie recalls how she and her team spent hours focused on the titles and subtitles of charts, “because that is ultimately what most people will look at”. And in those titles and subtitles, the analysts made sure to specify ‘confirmed’ deaths or ‘confirmed’ cases. “An emphasis on ‘confirmed’ is really important because we know that it’s an underestimate of the total,” says Ritchie. “It might seem very basic, but it’s really crucial to how you understand the data and the scale of the pandemic.”
The best data visualization might also not be the one that is most pleasing to the eye. Data scientist Pouria Hadjibagheri led the team working on the UK COVID-19 dashboard run by the UK Health Security Agency, formerly known as Public Health England, in London. His team gathered and processed nearly one billion records a day from 26 different data sources. His audience includes individuals with visual impairments, epilepsy, attention-deficit hyperactivity disorder or other conditions. “All of these people have equal rights to see the data,” says Hadjibagheri, who was the lead software developer for the agency until June 2021, but continues to work on the dashboard.
His team regularly surveys dashboard users to see how well the visualizations are understood. In February 2021, for example, he asked users which of the three vaccination visuals they liked the best. Importantly, the team also quizzed the participants on their interpretation of those graphics. People tended to prefer a bar-chart option that looked “simple and very nice”, says Hadjibagheri, but they couldn’t accurately identify the proportions it represented. Ultimately, the dashboard team opted for a ‘waffle’ chart design that offered a more granular view of the data, because the polling showed that users understood it better. See following chart:
Hadjibagheri and other dashboard architects have also been engaged in a two-year crash course of sorts, on how best to present information to the public. “Science communicators have had about two years now to sort of stress-test all of their methods of communicating to a general audience,” says John Burn-Murdoch, chief data reporter at the Financial Times in London. “And I think there have definitely been improvements.”
Trust and transparency
Other common lessons learned: let people see behind the scenes, share the data sources, and teach people what data methodologies were used and why. And always be transparent about any gaps or errors that could lead viewers astray.
The UK dashboard got some fine-tuning in the aftermath of a mishap on 7 March 2021. That day, death data from England had not been processed by the agency’s usual deadline. The team decided to go ahead with an update and to post a prominent explanation for why there were no deaths listed. “We thought it was so bizarre a number at that point in time that people would be forced to look at the website,” says Hadjibagheri.
They were wrong. Within minutes, several news agencies, including the BBC and Reuters, posted stories with the new data — and without the added caveat. Hadjibagheri quickly logged on to Twitter and corrected their mistakes. “We knew people were paying close attention. But we weren’t expecting broadcasting agencies to not apply a sensibility check,” he says.
The team subsequently set up automatic feeds so that organizations could subscribe to their announcements directly. It also decided to delay posting data with big gaps in the future. Hadjibagheri had been increasing his social-media activity since about November 2020. He says that trust in his service, based on user surveys, subsequently rose by nearly 35%.
Engagement is important. In parallel with his widely shared data visualizations, Burn-Murdoch took to social media as well. In March 2020, he posted a six-part Twitter thread explaining the log scale and why he chose to use it to show the rise in case numbers in different countries. “I’ve always seen my job, fundamentally, as communication,” says Burn-Murdoch, who regularly responds to people’s questions about charts and concepts.
Social media has also given data scientists a chance to draw back the curtain around what they do, answer questions, correct misunderstandings and provide context for their data — promoting transparency and trust. But social media can rapidly spread inaccurate messages and potentially degrade trust in dashboards.
Take, for example, the wave of misinformation that rippled worldwide after a guest of US podcaster Joe Rogan said that COVID-19 case rates were higher among vaccinated people. Brazilian President Jair Bolsonaro reported the same thing on social media and included baseless claims that vaccinated people also had a higher risk of contracting AIDS. They each referenced data published by the UK Health Security Agency, which did show that SARS-CoV-2 infection rates among fully vaccinated people aged 40 years and older were higher compared with unvaccinated people in the same age group.
The numbers, however, were based on an inaccurate estimate of the pool of unvaccinated people. When a more appropriate estimate was used, case rates among unvaccinated individuals were shown to be higher than for vaccinated people in nearly all age groups. The agency was sharply criticized for releasing the misleading figures. Recognizing early on that the data interpretation is not straightforward, Hadjibagheri says that his team never published visualizations of the breakdown of cases and deaths based on vaccination status.
Indeed, the COVID-19 pandemic has shown how easy it can be to twist data to fit a narrative. The dashboards have provided a lot of information. “There are so many numbers, and so many sources and tables and papers to pick from,” says Lambrechts. “It’s a real threat.”
Bill Hanage, an epidemiologist at the Harvard School of Public Health in Boston, Massachusetts, warns that people can also “fall prey to motivated reasoning”.
Dashboards are a great place to start thinking about data, he says, but people shouldn’t stop testing their assumptions. “Look at all the people who trumpet the low reported mortality in sub-Saharan Africa, without asking about testing capacity.”
He points to a lesser-known category of data dashboards that has proved powerful — and perhaps less prone to manipulation. Wastewater samples provided one of the earliest signals that Omicron was emerging in South Africa, he notes. Many US municipalities, including several counties in Massachusetts and North Carolina, have created public dashboards for wastewater data, helping to track infections and predict new outbreaks. “Wastewater can’t lie,” he says.
Beyond COVID-19
Data scientists are hopeful that the time and money invested in COVID-19 data dashboards, and the global education they provided, will pave the way for similar efforts to address other important issues. “Creating standards that are easily adoptable, like measuring cases and deaths for COVID, will be really important when we’re trying to do the hard work of eradicating poverty and improving climate conditions,” says Blauer. “The big question is, are policymakers willing to do it?”
For example, the data hosted on US dashboards such as Mapping Police Violence, run by the non-profit US police-reform advocacy group Campaign Zero in New York City, suggest that racial bias plays a part in police violence. “The data is telling us that there are kinds of officers who are going to engage in an officer-involved shooting,” says Blauer. Yet incident reporting is piecemeal, voluntary and not standardized across the country.
And the data are complex. Researchers have yet to reach a consensus on how to account for differing rates and types of police encounter — from traffic incidents to active-shooter situations — for different races and ethnicities. “How do you attract better, more empathetic officers? Getting that data is almost impossible,” says Blauer.
Much like COVID-19, climate change is a global problem that no country can tackle on its own. Also, as for COVID-19, crucial information on the subject can be difficult to access. The International Energy Agency (IEA) in Paris, an intergovernmental organization established as part of the Organisation for Economic Co-operation and Development, is the world’s most authoritative and comprehensive source of global-energy data. Yet much of the IEA’s data remain locked behind a paywall. Currently, many people rely on information provided by the energy firm BP, which is missing data from many countries and lacks key metrics. “So we have people trying to understand climate change based on data published by a petrochemical company,” says Ritchie, who is among many scientists pushing the IEA to become more forthcoming with its data. The agency has shown signs of relenting. It has started offering free reports and providing more free data, according to Jad Mouawad, an IEA spokesperson. “We are now exploring options to further increase the amount of data that is available for free to more users while at the same time maintaining the financial stability of the agency,” says Mouawad.
Meanwhile, improvements in technologies, such as remote sensing using satellites, will show in near-real time and in greater detail “how we are changing our planet”, says Lambrechts. An increasing number of climate data dashboards are popping up, including one from the UK government that shows changes in metrics ranging from temperature and rainfall to the area of woodland in the United Kingdom. “The thing about climate change is that we think it’s not happening in the present, when it is actually happening in the present,” says Angela Morelli, co-founder of the InfoDesignLab in Oslo, which produces data visualizations for the Intergovernmental Panel on Climate Change.
Hadjibagheri, who was recently approached by the Ministry of Justice in the United Kingdom regarding a potential dashboard for sex crimes, suggests that the world can build on the data lessons and achievements from COVID-19. “The infrastructure built, the services built, the culture built — those can all be adopted,” he says. Specifically, he argues that every country needs an independent data-science office, with the expertise and ability to handle and publish large volumes of data. “This is data that we can use to our advantage.”
* * * * * * * * * * *
This is no time to stop tracking COVID-19
From the way political leaders in many high-income nations are talking and acting, it would be easy to think that the COVID-19 pandemic is no longer worth keeping track of.
The pandemic might have taken upwards of 18 million lives, disabled many more than that and gut-punched the global economy, yet surveillance and reporting of the virus’s movements are starting to slow just at a time when a highly infectious subvariant of Omicron, BA.2, is spilling out across the world and case rates and hospitalizations are creeping back up.
These cutbacks are not based on evidence. They are political, and they could have disastrous consequences for the world. Maria Van Kerkhove, technical lead for COVID-19 at the World Health Organization (WHO), says it’s crucial that “the systems that have been put in place for surveillance, for testing, for sequencing right now be reinforced, that they are not taken apart”.
Around the world, the frequency of national reporting has slipped below five days a week for the first time since the early months of the pandemic, according to the publishers of the website Our World in Data. In the United States, the Centers for Disease Control and Prevention (CDC) is still reporting nationwide data, but there’s less real-time reporting of death and infection figures at the local level. All but eight states have scaled back to reporting data five or fewer days per week. Florida announced last week that it will now be reporting only fortnightly.
The UK government’s COVID-19 tracking dashboard, one of the world’s most comprehensive, is stopping its weekend updates of infections, mortality, hospitalizations and vaccinations, lumping Saturday and Sunday figures into Monday’s. Prime Minister Boris Johnson says this is part of plans to “live with COVID”.
The downward trend in reporting is subtle, but it mirrors other signs of complacency about COVID-19. The United Kingdom, for example, will no longer provide diagnostic tests free of cost. Several of its data-collection programmes are also ending. REACT-1, a long-running random-testing study, will lose its government funding at the end of this month. And ZOE, a mobile app that residents can use to log their COVID-19 symptoms, has lost its public funding, too. Both have been invaluable to research and policy.
The United States and United Kingdom aren’t alone. In many countries, political sentiments are shifting towards adopting a ‘new normal’. Of course, national budgets are being stretched thin as governments look to increase public expenditure on subsidizing fuel and food as the world plunges from dealing with the pandemic to tackling the global impacts of war in Ukraine. But scaling back virus surveillance at this time is short-sighted. It’s like stopping a course of antibiotics at the first sign of symptoms easing: it increases the risk that the infection will roar back. A study published last month says the next variant could well be more dangerous than those circulating now.
Public-health decisions need to be informed by the best available data. Cutting the ability to track and respond to the virus while most of the world remains unvaccinated makes these decisions less reliable. It will also reduce people’s ability make decisions about their own safety.
This is all the more infuriating given that roll-backs of public-health interventions have often come with messages that people should now decide for themselves what measures to take. The CDC, for example, recommends that people at risk of serious complications from COVID-19 “talk to their healthcare provider” about whether they should wear a mask or respirator during “medium” community transmission levels — just when data on transmission are becoming less accessible.
Researchers have worked hard to make disparate sources of data about the pandemic available to the public through several celebrated dashboards. Tools such as the WHO Coronavirus (COVID-19) Dashboard, Our World in Data and Johns Hopkins University’s COVID-19 Dashboard have empowered governments, businesses and individuals to use the best available evidence to make decisions. By reducing the data streams that power these dashboards, governments are shutting their eyes to the danger. If this trend continues, the new normal is going to look a lot like the false comfort of ignorance.