Introduction

Context

On March 11, 2020, the World Health Organization (WHO)’s declaration of COVID-19 as a pandemic caused widespread panic and alarm across the United States. One of the biggest shocks about the results of the pandemic was that the U.S. fared worse than other countries, “with more than 29 million cases and nearly 530,000 deaths” (Scientific American). As a result, there is an urgent need to understand the mistakes that were made which led to where we are today. One factor that health experts and scientists have widely pointed to is the fact that the United States’ response to the COVID pandemic was not uniform, and much of the pandemic response was left up to state and local leaders. Without a unified national strategy, states implemented largely uncoordinated policies. Among these, a crucial point of investigation is the discrepancy in mask mandates among different states.

Research Questions

We explore a variety of research question, each about the effectiveness of mask mandates at the state-level in the United States.

  1. How varied was mask mandate policy at the state-level?

  2. How long does it take after a mask mandate intervention to see an impact on COVID cases?

  3. Does effectiveness of mask mandates vary by season?

  4. Mask Mandates in the Delta Wave

  1. Were mask mandates in the Delta Wave effective?

  2. How does vaccination rate interact with mask mandate effectiveness?

Heterogeniety in Masking Policy in the United States

We observe from the map above that between March 2020 - May 2021 (total of 455 days), states with state-wide mask mandate policies had on-time durations ranging from 97 to 408 days, being Wyoming and New York respectively.

Moreover, the state abbreviations in red represent regions that has implemented multiple restrictions, with potential combinations of easings and liftings as well. We noted and separated the 7 states with multiple restrictions into strict and loose categories:

Loose: Mississippi - 2 restrictions with 2 liftings soon after Kansas - 1st restriction and a soon after lifting

Strict Oregon - 2 restrictions and then 1 easing Wisconsin - 2 restrictions and then 1 easing Michigan - 2 restrictions and then 2 easings New Mexico - 3 restrictions and 1 easing in between Hawaii - 2 restrictions only

To look into whether having multiple restrictions helped to lower the rate of change in case counts, we look at the following lag plots.

From the 4 subplots above, we can see that after 3 weeks, apart from the 2 states with “loose” multiple restrictions (red dots), the rate of change for rest of the states definitely all stayed below the identity line. However, this does not necessarily mean that putting on multiple restrictions result in lower case count after 3 weeks, since there are also many other states that consistently stayed below the identity line despite only have 1 restriction.

Definitions

Success Criteria

E.g. Duration: 30 days, Percentage = 90% How many days does it take for us to see a lower case rate than the start rate for 30 days in a row at least 90% of the time? In other words, if at least 27 days within a 30-day timeframe has a lower case rate, then we say the intervention is successful and record the number of days it took to get there.

Lag Day vs Success Criteria

Duration vs Percentage

We tried out different combinations of duration and percentage on the top right corner, and calculated the average number of days taken to meet each criteria. As we can see, the average number lag days is within one to two weeks. This indicates is generally takes one to two weeks to see an effect from the mask mandate.

In the above plot, each bar represents a state and the y-axis represents the number of days it took to meet the success criteria. Most states reach a lower rate with a week.

Mask Mandates by Season

Does Season Have an Impact on COVID Case Rates?

We also seek to understand whether the time of year a mask mandate intervention is placed has any impact on the COVID case rates. To accomplish this, we labeled all the mask mandate interventions based on the season they were implemented.

The time ranges for each season were referenced from https://www.timeanddate.com/calendar/aboutseasons.html

  • Fall (9/1 - 11/30)
  • Winter (12/1 - 2/28)
  • Spring (3/1 - 5/31)
  • Summer (6/1 - 8/31)

Data

The dataset includes 44 mask mandate interventions in total, placed by different states in the United States during the pre-Delta COVID wave.

Distribution of Mask Mandates by Season

First, we’ll explore the distribution of mask mandate interventions by the season during which they were implemented.

Overall, we see that mask mandates were most often implemented by states during the summer, and much less frequently during the spring and fall.

How long does it take after a mask mandate to see lower COVID case rates?

From the above plot, we do not observe a clear separation between mask mandates that happened in the fall versus spring versus summer. We do see that there is a larger number of summer mask mandates which take a shorter number of days after a mask mandate to observe lower COVID case rates – however we also have to keep in mind that our dataset contained more summer mask mandates than fall or spring. As a result, it is inconclusive whether summer mask mandates truly performed better than those in the fall or spring.

Another note we must consider when viewing the plots is that the first few bars (where the number of days after mask mandate to see lower case rate is a very small number, e.g. 2) must be taken with a grain of salt. Mask mandate interventions that fall into this category most likely were already seeing a decrease in COVID case rate when the mask mandate was placed. In other words, the mask mandate itself most likely should not be attributed for the lowering in case rates.

In general however, we do notice that states that did see a lower COVID case rate after a mask mandate intervention observed the change around 1 to 2 weeks after implementing the intervention. We also find promise in the fact that a majority of these mask mandates were able to sustain their lower COVID case rate for up to 30 and 60 days.

Effectiveness of Mask Mandates in the Delta Wave

The following seven states implemented state-level mask mandates in July and August 2021 as the Delta variant swept across the United States: California, Connecticut, Illinois, Louisiana, Nevada, New Mexico, and Washington. Studying these mask mandates has two main advantages. First, these mask mandates tended to be the only major actions these state governments took to fight the spread of COVID during this time; earlier in the pandemic, states implemented multiple restrictions at once or within short periods. A lack of other state-level actions reduces one (of many) areas of potential confounding. Second, varying levels of vaccination around the county allows us to study how the rate of vaccination in a state interacts with the effectiveness of mask mandates.

The plot below shows the lag time in six of the seven Delta Wave mask mandate states compared to the lag time in all other states without active mask mandates. Here, lag time is defined as the number of days it takes for the rate of change in cases to stay below the the rate of change in cases on the day of the mask mandate for 10 days, 90% of the time. The bars are ordered from longest lag time to shortest, where a lag time of -1 indicates that the state never succeeded in terms of our definition of lag time before 11/23/2021. The plots are ordered from highest vaccination rate in the mask mandate state at the time of the mandate (Connecticut) to lowest (Louisiana).

We see that, according to this metric of success, lag times for mask mandate states vary from 3 to 27 days. We also see that the mandates vary in terms of how quickly they succeed in comparison to benchmarks without mask mandates. Also, we see that more highly vaccinated comparator states tend to have lower lag times even without mask mandates.

Realistically, 3 days is too short of a lag time to attribute to a mask mandate, so it is likely that a better metric for lag time is needed to properly measure the relationship.

Mask Mandates Are More Effective in Lower Vaccinated States

Again, one of the advantages of studying mask mandates in the Delta wave is the potential for exploring the interaction between mask mandates and vaccination rates. Specifically, we ask the following two questions:

  1. Do states with active mask mandates see better COVID case count trajectories compared to similarly vaccinated states without mask mandates?

  2. Is (1) different for more highly vaccinated states compared to lowly vaccinated?

The graph below shows the normalized rolling average of cases for one week before and seven weeks after a mask mandate in six of the seven Delta wave mask mandate states, compared to the four states with the closest vaccination rate. The plots are ordered from most vaccinated mandate state (Connecticut) to least (Louisiana) at the time of the mandate. In each graph, the red dashed line is the state that implemented the mask mandate. Any other dotted lines are states that have an active mask mandate, but it has been “eased” to a less restrictive level. All other lines have no active mask mandate.

We see different levels of mask mandate effectiveness across vaccination levels. Louisiana sees an extreme reduction in case counts about 3 weeks after the implementation of the mask mandate. It should be noted that cases in Louisiana already peaked at the time of the intervention, but the speed and magnitude of the decrease in cases seems to outpace similar states who also saw reductions in cases. In comparison, we see less success in Connecticut and Washington, although a major confounder is the level of cases in the state at the time of the intervention. However, case counts in Nevada and New Mexico stay low after their mask mandates compared to similar states, much more than Connecticut and Washington.

Conclusions

In summary, we investigate the effectiveness of mask mandates in a few contexts: via lag times with case rates, by season, and in the Delta wave. With our measure of lag time, it takes on average 1 week to see lower case rates but it’s subject to change based on different success criteria. We see that seasonality did not have a significant impact on the number of days it takes for a mask mandate to result in lower case rates during the pre-delta wave. Further, there is conflicting evidence for effectiveness of mask mandates during the Delta Wave when we measure using lag time; however, there is striking visual evidence for mask mandates being effective overall in the Delta Wave and particularly effective in lower vaccinated states.

Discussion

Due to conflicting visual and statistical evidence for the effectiveness of lag time in the Delta Wave, we think that there are better ways to capture the idea of lag time than we are currently using. Specifically, our measure of lag time relies on an underlying rate of change in case counts; however, we see convincing visual evidence of the effectiveness of Delta Wave mask mandates based on case counts and less so with a rate of change. As a result, we hypothesize that a measure of lag time based on visual evidence, using a combination of case counts and case rate, can better capture the idea of lag time, and aid the visual evidence.