Relying on Unreliable Forecasts

Carter Mecher MD -- Medical Advisor
September 24, 2021

During a pandemic, what should be the role of government in a free society? Public health edicts including mask mandates, limits on the size of public gatherings, closing of restaurants, bars, gyms, schools, and places of worship, as well as vaccine mandates are viewed by many as being coercive. As a consequence, a number of state legislatures are looking at limiting the authority of state and local public health officials. A number of Americans believe that mask  and vaccine mandates cross a line. They believe those decisions are best left to individuals, not the government. That then implies that individual Americans have the capability to make an informed decision regarding the actions that they need to take to protect themselves, their families, and their community.

Whether we have mandates or not, the collective actions of all of society (the sum of individual actions in response to individual perceptions of risk) determine the course of a pandemic.

One core issue is risk assessment. How do 330 million Americans assess what is happening during the pandemic? Are they looking at data on what is happening locally, at a state level, nationally? Are they following news reports, social media posts, or interactions/conversations with family and friends? Where are they getting their information? How is that information being filtered? How do they decide on the best course of action? 

Consider how we think about weather and weather forecasting. The government does not mandate that we wear a raincoat or carry an umbrella when it is raining. We all have access to weather reports. Those reports for a specific area are consistent no matter where you seek information on current weather conditions. There is no political slant placed on the data. At any moment, you could check on the current weather conditions for just about anywhere on the globe. One could also simply look out a window or walk outside to immediately confirm that data (at least for your present location). The weather data is timely (current), accurate, and easily confirmed by anyone with absolutely no meteorological training or expertise. As a result, weather reports are trusted. Forecasting is a little more complicated, but short range forecasts (<72 hrs) are fairly accurate. We look at weather forecasting to make decisions on how to dress, whether to wash a car, paint the house, have a picnic, etc. Hundreds of millions of us are making all sorts of decisions and taking all sorts of actions based upon weather data. And at any moment, we have instant feedback on current conditions with our own eyes. We even have a bit of commonsense forecasting by looking up at the sky (and the clouds). We can see an approaching storm. We don’t need government mandates to compel us to act in a certain way. We just need reliable information.

But there are some special circumstances when we need a little extra help (and sometimes even an extra push). We get weather warnings (many receive them on their iPhones, much like an Amber alert, and we also have a siren in our communities for alerting us to tornados). We can get warnings of heavy thunderstorms, potentially damaging winds, damaging hail, potential for local flooding, risk for mudslides, risk of a tornado by diagnostic Doppler radar signals, an approaching hurricane, heatwaves, dangerously cold temperatures, heavy snowstorms and blizzard conditions, etc. Those warnings are really threat assessments based upon the expert analysis of the weather data--something most of us cannot do on our own (even with access to real time information on the weather). And in some circumstances, the government does coerce people to act—to immediately seek shelter, to evacuate. And in some cases, the government does implement mandates—mandatory evacuation for hurricanes. That is a form of coercion that we accept under those life-threatening conditions.

So where are we in terms of the state of art of pandemic assessment and forecasting? If the displays on the Johns Hopkins site or CDC or in the New York Times are like weather reports, they are providing the equivalent of weather a week or so ago for cases and lag even farther behind for deaths. The most current data we have is hospitalization data, but that lags behind cases (the real measure of community transmission). So imagine trying to make decisions based on weather from a week or so ago. Even worse, our awareness of cases is limited by case ascertainment. During the spring wave of the 2009 H1N1 pandemic, we estimated that we were aware of about 5% of true cases (meaning that we were unaware of 95% of those who were infected during that spring wave). We estimated a similar case ascertainment rate in the early days of this pandemic. I suspect that even today, we may only know about 20-30% of the true infections. Could you imagine a weather report that was only aware of 20% of the rainfall or snowfall amounts and how that might impact your personal assessment and behavior? To make matters even worse, we can’t “walk outside” and see things with our own eyes to assess accuracy. Depending on where you live, if you walk outside today or walk into a store or restaurant, one would hardly know there is a pandemic happening. You find pre-pandemic behavior with only a small number of individuals wearing masks, and pretty much no social distancing. That social proof is extremely powerful. Just like the weather, we overlay our everyday perceptions with the data we hear reported. We also filter those perceptions very differently depending upon political beliefs (for the most part). Without realizing it, we are hampered by our own confirmation bias.

Except for hospital staff, inpatients and their families, the serious consequences of this pandemic are nearly invisible. In a population of 330 million, there are currently less than 80K inpatients and 22K ICU patients. Accordingly, the population directly impacted on an average day as the result of a family member fighting for his or her life in an ICU is relatively small. Given that hospitals have limited visitors, the daily war being waged in our hospitals is only visible to the families of those patients and the 6.5 million people employed in US hospitals. As a result, only 2% of the US population is keenly aware of what is happening in our hospitals. https://www.bls.gov/opub/ted/2020/number-of-hospitals-and-hospital-employment-in-each-state-in-2019.htm  For most of us, the sun is shining and things don’t seem that bad. You can travel where you like within the United States. You can go to a bar, restaurant, store, gym, school, and place of worship much like people have always done. In many parts of the country, there are some people wearing masks, but for the most part nobody is going to make you wear one. Friends and family are attending weddings, birthday parties, sporting events, and they all seem to be ok.

To make things even more complicated, how do we then take all the objective and subjective information we gather from these various sources and estimate risk?

The other difference between weather and COVID is this—whether you wear a raincoat or use an umbrella, it only affects you. At worst, if you disregard the weather and don’t take appropriate actions, you get soaked and perhaps ruin a pair of shoes or an article of clothing. If you disregard the risks associated with COVID and become infected, you potentially become a link in a chain of transmission and potentially trigger a cascading outbreak that can impact your community and the lives of others.

In the case of natural disasters, emergency managers do not issue evacuation orders lightly. Ordering people to leave is a tough call, considering the logistical complexities of mass displacement, the increased vulnerability of stabilizing support systems, and loss of public trust if these forecasts flop. For those who choose not to evacuate, their decisions do not stop us from helping in the event of a crisis. It certainly complicates the response, but people should not be abandoned based on making the wrong risk assessments


Join our Team

We are actively hiring and would love to connect with talented individuals that want to be on the forefront of change. Contact Us.