Unforeseen challenges: An insight into volunteering on an IFoA pandemic working group


Analysing and understanding the COVID-19 pandemic introduces a host of unforeseen challenges that are very different to traditional actuarial work. Andrew Robinson, Naliaka Wafula, Sukrita Singh and John Branford from our PAN4 COVID-19 workstream share their experiences and ask for your thoughts and insights as to how the challenges can be best addressed.

Whilst volunteering for the IFoA, our pandemic working group has rapidly developed an appreciation and respect for the work put in by others when conducting research. In this blog we share those lessons with you and ask you for your views on how to tackle research in an ever evolving pandemic. 
As actuaries we have a broad range of skills useful in understanding the COVID-19 pandemic, derived from our industry specific knowledge, including data handling and statistical analysis. The COVID-19 pandemic has given rise to a whole host of challenges unforeseen to our volunteer group as they differ from those in traditional actuarial work.
Our volunteer group is looking at the impacts of the pandemic on inequality, and we have started by testing the hypothesis that the level of equality (fairness) in a country rather than the level of absolute wealth has had the greater impact on the health and economic outcomes from coronavirus. More to follow on the outcomes of that study in another blog but for now we share some of the obstacles we have faced so far.
The health outcome - studying the death rate
We quickly concluded that deaths rather than cases were the most reliable health measure as the latter depend crucially on testing levels which lack consistency across countries and the former were apparently well-defined and easily available. Although different countries use different methods for reporting coronavirus deaths we have used the published figures as the (perhaps better) alternative of excess deaths is available for far fewer countries.   
Our next problem was considering how one assesses the spread of coronavirus through a country whilst comparing to another. The most simple metric would be deaths per million but would it be reasonable to compare this data at a static point in time if one country didn’t see its first case until much later than others? Should one look at the number of deaths in the ‘first wave’? Where though does a wave begin and end? Do these timing problems discredit the outcome of the study for those countries well into their second wave?
We had considered measuring the impact of the virus since the date of the 100th case or 100th death, but in measuring the total impact over the course of the pandemic and not the rate of spread, deaths per million was chosen. The economic data being compared to could not be broken down into small time periods, adding additional complexity to any subsequent correlation testing. 
The economic outcome - the quest for data
In our working lives, our profession, employers or clients tend to provide us with established data sets in standard, familiar formats from which we conduct our analysis. The challenge faced in studying the economic impact of the pandemic came in finding data sources for outcomes that were reliable, up to date and had data on a sufficient number of countries to aid us in performing reliable statistical tests.
With the pandemic changing constantly, finding the best economic indicator has proved a challenge. Many countries are yet to produce their second quarter GDP figures and we are already in the fourth quarter. We have used some experience data on economic growth and unemployment figures. Separately we are using forecasts for these items available for all countries.
The multivariate nature of the pandemic
With any complex analysis, a univariate view of the world is dangerous. You wouldn’t build a car insurance pricing algorithm using one variable  for a host of reasons, one being that a single variable can never explain something as complex as a human’s ability to safely navigate a vehicle. Similarly inequality is just one of many variables that may impact a country’s death rate or economic output in a global health crisis. Without building a full predictive model of the pandemic, our attention must turn to appreciating which variables will have proved significant in studying the pandemic. With the coronavirus these range from government actions, to quality of healthcare to levels of tourism, age distribution and many others. 
Time will also be one of the variables and we have become conscious that any conclusions we make may well be provisional. Indeed we expect to see improvements, whether through reduced infection, improved treatment, vaccination or even eradication which might change any assessment of how different countries have ‘performed’ in the pandemic.   
Consequently it is difficult to quantify the actual impact of the pandemic since it is still evolving and its course cannot be predicted. Some of its effects will be felt for years to come and what we believe or find now may not be true next year or the year after. It may be that our hypotheses are not supported by the data. We have realised that this doesn’t represent failure but rather, brings with it a deeper understanding. We believe that as actuaries we bring an objective approach to this research. 
How do we interpret results - statistics
It is an established fact that the mere existence of a relationship between two variables does not by itself indicate a cause and effect relationship. It takes some careful consideration to interpret results and keeping this in mind, we know that any conclusions we come to may involve a certain level of subjective judgement. For example, if analysis shows that countries with female leaders have fared better through the pandemic than countries with male leaders - do we conclude that women are inherently better leaders than men? Or does it suggest that ‘mature’ economies, economies capable of handling stress situations better are the kind of economies that are more likely to appoint female leaders in the first place? This opens our minds to the possibility of the same statistics being interpreted in different lights depending on the lens through which we look. 
One may be tempted on reviewing this blog to question why so much was unforeseen; I see a willingness to lend a hand during a global pandemic as being more prevalent in the mind as an answer to this question rather than naivety on behalf of the participants. We have all developed a much greater appreciation for the endeavours of others in producing good research. The reader may also be tempted to point out how many questions this blog poses instead of answers to you. I say, welcome to the problem and we would love to hear your insights.