Helen Johnson, mathematical modeller at the European Centre for Disease Prevention and Control (ECDC), explains how modelling teams around the world are working together to develop accurate and timely predictions of Covid-19 cases and deaths in Europe. These forecasts have become an invaluable tool for making public health decisions.
How did the European forecast hub come about?
Helen Johnson: Since the start of the pandemic there has been great interest in tracking the number of Covid-19 cases and deaths to try to forecast what is going to happen in the future and to understand the effects of different restriction measures. This is done by mathematical modelling. You can just look at the numbers to predict what will happen if they continue in the same trajectory, or you can take into account other factors related to the population and how the virus spreads.
There are always limitations with this because you have to make a lot of assumptions: about what is important, who has been tested, changes in behaviour… To look at any one model is always somewhat limited. A really good approach is to combine multiple models together. This is often done in climate modelling.
About eight years ago, our counterparts at the CDC (Centers for Disease Control and Prevention) in the US started collating forecasting models for influenza. We’ve been working closely with them and were planning on doing something similar in Europe for West Nile Virus. However, when Covid-19 emerged we realised that this would be the first virus that we would study using this approach.
How does it work?
Helen Johnson: Each week, modellers from all over Europe and beyond (we have contributions from the US, Canada and Australia), submit their forecasts for the number of confirmed Covid-19 cases and deaths over the coming 4 weeks in their country and in some cases, their neighbours or all EU Member States. They each use very different methods to do this.
Combining the forecasts, in what is known as ‘ensemble forecasting’, gives you a more accurate model than almost all single models. This has been demonstrated in climate forecasting and has been observed consistently in the US forecasting hub over different viruses. The pooling of knowledge is highly effective because it takes out some of the biases of the assumptions and of the data.
Around 30 teams contribute to the European forecast hub every week. At our weekly meetings different teams have the opportunity to present their model and discuss the assumptions, the data and epidemiological situation in their country. This helps to build the network of infectious disease modellers in Europe.
When it comes to the use of infectious disease modelling to inform policies, European Member States are in very different stages. Some countries, like Belgium, the Netherlands and Finland have been doing this for many years, so their Health Departments and governments are really familiar with it; they know how to interpret and communicate the uncertainties. However, there is a whole raft of other countries where there is some or no experience of using modelling to inform public health policies. Bringing them all together is one of the great strengths of the hub.
What data does the European forecast hub use?
Helen Johnson: First, we need to ensure that the models reflect reality; we don’t want to be looking forwards with a model that can’t tell you about today. We have to start the model running sometime back and make sure that its estimates for last week or yesterday fit with the data that we’ve seen already. Only then you can have some confidence in what it says about tomorrow or next week. This process, known as model fitting, uses standardised data on case numbers and deaths from the ECDC and John Hopkins University.
To hit these targets, we can draw from all sorts of data. Some models use telecoms data on mobility or vaccination uptake figures. Some account for variants of concern, the introduction and lifting of restriction measures… you could include anything. If you think transmission is associated with good weather and people going out more, you could look at correlations with ice cream sales.
Other models don’t take any of these things into account and simply look at current trajectories. These can perform extremely well but will be slow to pick up a change in direction.
Combining these models with ones that make implicit assumptions based on different types of data is very helpful for identifying the core truth.
What are some of the challenges when forecasting Covid-19 cases and number of deaths?
Helen Johnson: The delay in death notifications varies a lot between countries and can increase the lag between case numbers and peaks in deaths. Also, if testing rates change dramatically, if tests are suddenly more accessible or there is a change in testing policy, you may see an increase in the number of confirmed cases that might not necessarily indicate an increase in transmission.
The changes in testing rates can vary a lot between countries and are not well documented. There are also very big differences in the way that hospital and intensive care unit admissions are classified and reported between countries, so we aren’t currently forecasting those.
How are the hub’s forecasts reaching policy makers?
Helen Johnson: Many of the teams that are contributing to the hub are already informing policy makers in their own countries. The ECDC also sends out regular forecasts to Member States and the European Commission and the press contributes to increase their visibility.
It is a bit of a long game. When I started working on infectious disease modelling in the UK in 2006, the Department of Health was getting used to the concept of using mathematical models and how to be ‘good consumers’ of modelling analysis, how to be critical of it, how to think of them in terms of uncertainty, different scenarios…By 2020, having been through swine flu and Ebola, the UK Government was very happy to be calling on and referring to their modelling teams.
It takes time for people to become comfortable and confident with what modelling can and can’t do. In the last year, many Member States have increased the extent to which they conduct, think about and call upon modelling.
What will happen post-Covid-19?
Helen Johnson: The beauty of this approach is that it is open, collaborative and applicable to other diseases. We are completely transparent. All our models and results are available. It is positive for European science and positive for European public health. We will continue to work closely with the forecasting hub in the US as they move on to scenario forecasting to explore what would happen in different potential situations such as if vaccination uptake increased by 50% in the next 3 months or if a vaccine-resistant mutation emerges.
The pandemic has led to some really strong projects such as the CoMix study, part of the EpiPose project funded by the EU, in which people in 20 European countries are being interviewed every two weeks by the market research company Ipsos MORI. Over one thousand people in each country are asked about their awareness, attitudes and behaviours in response to Covid-19.
The data from this study will be extremely valuable for working out the differences in social behaviour between countries. We will be able to learn things such as: “What is the effectiveness of closing schools?”, or: “How does behaviour change after vaccination?”. This can improve modelling not just for Covid-19 but, potentially for other infectious diseases as well.
