Counterfactual time series analysis of short-term change in air pollution following the COVID-19 state of emergency in the United States Lockdown measures implemented in response to the COVID-19 pandemic produced sudden behavioral changes. We implement counterfactual time series analysis based on seasonal autoregressive integrated moving average models (SARIMA), to examine the extent of air pollution reduction attained following state-level emergency declarations. We also investigate whether these reductions occurred everywhere in the US, and the local factors (geography, population density, and sources of emission) that drove them. Following state-level emergency declarations, we found evidence of a statistically significant decrease in nitrogen dioxide (NO2) levels in 34 of the 36 states and in fine particulate matter (PM2.5) levels in 16 of the 48 states that were investigated. The lockdown produced a decrease of up to 3.4 µg/m3 in PM2.5 (observed in California) with range (− 2.3, 3.4) and up to 11.6 ppb in NO2 (observed in Nevada) with range (− 0.6, 11.6). The state of emergency was declared at different dates for different states, therefore the period “before” the state of emergency in our analysis ranged from 8 to 10 weeks and the corresponding “after” period ranged from 8 to 6 weeks. These changes in PM2.5 and NO2 represent a substantial fraction of the annual mean National Ambient Air Quality Standards (NAAQS) of 12 µg/m3 and 53 ppb, respectively. As expected, we also found evidence that states with a higher percentage of mobile source emissions (obtained from 2014) experienced a greater decline in NO2 levels after the lockdown. Although the socioeconomic restrictions are not sustainable, our results provide a benchmark to estimate the extent of achievable air pollution reductions. Identification of factors contributing to pollutant reduction can help guide state-level policies to sustainably reduce air pollution. There is consistent evidence that short- and long-term exposure to fine particulate matter (PM2.5) and nitrogen dioxide (NO2) increases the risk of mortality, hospitalization, and other adverse health outcomes1,2,3,4,5,6,11,12. Furthermore, several studies have provided preliminary evidence that short and long-term air pollution exposure increases the risk of hospitalization and death among individuals with COVID-194,5,6,7,8,9,10.The United States mitigates air pollution through a combination of federal, state, and local air pollution regulations13. For example, the federal government sets emissions standards and the NAAQS. They also require states to prepare State Implementation Plans (SIPs) that detail emissions reductions strategies for areas that are not in compliance with the NAAQS (non-attainment areas). SIPs use air quality models to demonstrate how regulating local emissions sources helps a non-attainment area meet the NAAQS. Geographically heterogeneous regulations, emission sources, and meteorology, results in varying air pollution concentrations by geographic location13,14.Several studies have examined the impact of a sudden intervention on changes in air pollution (see15 for a review). For example, researchers used interrupted time-series designs to quantify the impact of the 1990 Dublin coal ban16 and regression discontinuity to identify the arbitrary spatial impact of the China Huai River Policy17. An important feature of these studies is that they investigated abrupt and localized changes across a relatively short time span (Dublin coal ban) and spatial scale (Huai River policy)18. Because of the abrupt nature of these interventions, defining a hypothetical experiment in these studies was straightforward.Similarly, we examined the effect of the abrupt lockdown measures implemented in response to the COVID-19 pandemic, which produced sudden and significant changes in how society functions, with decreases in road traffic, air traffic, and economic activity19. This provided us with an unprecedented opportunity to implement a quasi-experimental design with a well-defined control condition (no pandemic) to estimate the changes in air pollution because of the implementation of these extreme measures. In a quasi-experimental design, the researcher compares outcomes between a treatment group and a control group, just as in a classical experiment; but treatment status (in our context the COVID-19 related intervention) is determined by politics, an accident, a regulatory action, or some other action beyond the researcher’s control (in our context the start of the pandemic). See20 for a discussion of strengths and limitations of a quasi-experimental design. Furthermore, the spatial heterogeneity in the extent to which air pollution levels changed because of the lockdown measures allowed us to identify factors contributing to these changes.A number of recent studies have investigated the effect of the COVID-19 pandemic on the levels of different air pollutants in the US21,22,23,24,25,26,27,28,29,30,31,32,33,34, globally35,36,37,38,39 and for several cities around the world40,41,42,43,44,45,46,47,48,49,50,51. Table 1 summarizes studies that have estimated changes in air pollution levels by comparing air pollution levels during the COVID-19 pandemic period to historical data both in the US and globally.Table 1 Summary of published studies examining changes in air pollution attributable to COVID-19 related interventions in the US and globally.Regardless of the emerging literature on this topic, these studies for the most part do not simultaneously account for autocorrelation, time trends and seasonality, and meteorological factors. To our knowledge, none of these studies attempt to identify state-level factors contributing to heterogeneity in the air pollution declines across states for both PM2.5 and NO2.In this study, we had several scientific objectives that distinguish this paper from existing contributions in the literature. More specifically, we 1) develop and implement state-of-the-art time series approaches for counterfactual forecasting to predict weekly state-levels of PM2.5 and NO2 from January 1, 2020, to April 23, 2020, under the hypothetical scenario that the pandemic did not occur. These models account for measured confounding (e.g. meteorological factors), unmeasured confounding (e.g. seasonal variation and time trends) and residual autocorrelation; 2) properly validate the accuracy of the model fitting and account for the uncertainty in the counterfactual forecasts via bootstrap; 3) estimate the weekly state-level deviations and 95% CI between counterfactual (e.g., absent the pandemic) and observed levels of PM2.5 and NO2 from January 1, 2020 to April 23, 2020; 4) assess whether the deviations between the counterfactual values and the observed values start to deviate in correspondence to key interventions implemented as a result of the pandemic; 5) assess within each state, changes in both PM2.5 and NO2; and finally 6) investigate which state-level characteristics, including emissions sources, contributed the most to these changes, while adjusting for geography and population density.We gathered and harmonized data from several databases (Table S1). We obtained historical daily monitor data of PM2.5 and NO2 concentrations for January 1, 2015 to August 31, 2019 from the US EPA Air Quality System52. We obtained current levels of these air pollutants for August 31, 2019 to April 23, 2020 from the EPA AirNow application programming interface53. We linked historical and current monitor data within each state. These data were available for 48 states for PM2.5 and 36 states for NO2. We obtained daily temperature, humidity, and precipitation data from the University of Idaho’s GRIDMET project, which were then aggregated to the state level using Google Earth Engine54.We obtained state-level source emissions totals from the National Emissions Inventory for 201455, and gathered information on population density and geographic region classification of the states from the United States Census Bureau56,57. Finally, we accessed the COVID-19 US State Policy Database58 to extract information regarding the dates of COVID-19 related state interventions, including state-level declaration of emergency, shelter-in-place orders, and non-essential business closures for each state. All the data sources are publicly available, they are summarized in Table S1, and also available on GitHub along with all code necessary to conduct the analysis; https://github.com/NSAPH/USA-COVID-state-level-air-pollution-SARIMA-analysis.Statistical methodsCounterfactual forecasting of air pollution levels starting January 1, 2020SARIMA models are autoregressive models often used to forecast time series where future observations are correlated with past observations59,60. They have the advantage of accounting for the time trend, seasonality, confounders (e.g., meteorological variables), and residual autocorrelation. We fitted SARIMA models to historical data using weekly state-level air pollution levels (from January 1, 2015, to December 31, 2019) accounting for time trend, seasonality, autocorrelation and also accounting for the effect of weather by including temperature, precipitation, and humidity as covariates in the model.The basis of the SARIMA model is a linear regression of a response variable Yt at time t against the past values (Yt-1, Yt-2, ….) of Y and the past forecast errors (ɛt-1, ɛt-2, …). A detailed example of this analysis for NO2 in California is provided in the supplementary materials, including model validation measures (Figures S1-S5).We conducted the following analyses separately for PM2.5 and NO2 and for each state. The algorithm of the model construction and prediction is presented below. 1. We created 1,000 time series bootstraps using Box-Cox and Loess-based decomposition61 to separate the time series into the trend, seasonal, and remainder part. The remainder is then bootstrapped. We used historical data from January 1, 2015, to December 31, 2019 (see Figure S2 for an example of NO2 in California). 2. For each bootstrapped time series, we: Fit SARIMA models59,60,61 adjusting for meteorological factors, namely temperature, precipitation, and humidity (see Figure S3 for an example of NO2 in California). From the fitted SARIMA models, we predict air pollution counterfactual levels (absent the pandemic) during a 16-week period from January 01, 2020, to April 23, 2020 (see Figure S4 for an example of NO2 in California). 3. For each state and for each week, we average the predicted air pollution counterfactual levels across all bootstrap replicates. We denote these averages by (C_{i,j}^{pred} ,;where; i = 1,2, ldots ,16), and j indicates the state (see Figure S4 for an example of NO2 in California). 4. For each state j and for each week i, we estimate the weekly differences(delta_{i, j} = C_{i,j}^{obs} – C_{i,j}^{pred} , ;i = 1,2, ldots ,16), between the observed values (under pandemic conditions) and the predicted (assuming that the pandemic did not occur) (see Figure S5 for an example of NO2 in California). The quantification of the statistical uncertainty of these weekly differences using the bootstrap replicates is called ‘bagged SARIMA’ (see Figures S4 and S5 for an example of NO2 in California). The data and code for the analysis is available at https://github.com/NSAPH/USA-COVID-state-level-air-pollution-SARIMA-analysis.Model assessmentTo assess the overall predictive performance of the SARIMA model, we repeated the same procedure of model building and prediction as described in the algorithm above, this time training the model based on the data from January 1, 2015 to December 31, 2018, and predicting for a 16-week period from January 01, 2019 to April 23, 2019. This allows us to assess model fit and evaluate our modeling approach absent the pandemic. The main goal of implementing this assessment is to find out the model’s performance in prediction absent the pandemic and compare its predictive performance using the average prediction error as defined below during the pandemic.Average prediction error (APE) for state j:$${text{APE}}j = 1/16sumlimits_{(i = 1)}^{16} {delta_{(i,j)} } ,;where;delta_{(i,j)} = C_{(i,j)}^{obs} – C_{(i,j)}^{pred}$$as defined in Step 4 of algorithm above.We used the R package auto.arima to select model coefficients with the best predictive capability based on bias-corrected Akaike Information Criterion (AIC)62,63 and then used the mean absolute scaled error (MASE) to evaluate the fit of the model64.Estimating air pollution changes attributable to state-level emergency declarationsIn step 2 described above, we start the counterfactual forecasting for the period January 01, 2020, to April 23, 2020 without any consideration regarding the date of the intervention (such as the declaration of the state emergency). After the forecasting was complete, we then chose the declaration of the state of emergency as the intervention because it most closely visually aligned with the onset of deviations from the forecasted pollutant concentrations. Other interventions, including the timing of non-essential business closures and shelter-in-place orders, were considered visually (see Figures S7 and S8 in the supplementary material, the differences between these interventions are less than two weeks).We use Tint, j to denote the date of the state intervention (declaration of the state of emergency) for each state j. For each state and for each of the two pollutants (PM2.5 and NO2), we estimated the parameter (Delta_{j}) denoting the change in pollutant concentrations following the state intervention compared to before by calculating:$$Delta_{j} = Delta_{before,j} – Delta_{after,j}$$where (Delta_{before,j}) is the median of the weekly deviations, ({delta }_{i,j}), (as defined in step 4 above) for the weeks before the date of the declaration of the state emergency (Tint,j) and (Delta_{after,j}) is the median of these weekly deviations, ({delta }_{i,j}) , for the weeks after Tint,j. Because of the good fit of the SARIMA model to the historical data (Figure S6), and because the counterfactual forecasting is agnostic to the date of the state level emergency (see Figures S4, S5 for an example of NO2 forecasting in California), we argue that negative estimated values of ({Delta }_{j}) indicate that air pollution levels declined because of the state-level emergency. We note that since the state of emergency was declared at different dates for different states, and the total length of the prediction period was 16 weeks in 2020, therefore the period “before” the state of emergency in our analysis ranged from 8 to 10 weeks and the corresponding “after” period ranged from 8 to 6 weeks.To identify the states with the most pronounced discrepancy between the pattern of change in PM2.5 and NO2, we calculated the ratio (ρj) for each state j, defined as:$$rho_{j} = Delta_{{NO_{2} ,j}} /Delta_{{PM_{2.5} ,j}}$$I f ({rho }_{j} <0) the two pollutants changed in opposite directions (i.e., one increased while the other decreased), and the larger the magnitude of ({rho }_{j}) , the larger the discrepancy between the pollutants' patterns of change.Regression modeling to identify state-level factors contributing to heterogeneity in the air pollution across statesIn this part of the analysis, our goal is to quantify the associations between the change in pollutant concentrations during the forecasting period January 01, 2020 to April 23, 2020 and several sour
https://www.nature.com/articles/s41598-021-02776-0
Counterfactual time series analysis of short-term change in air pollution following the COVID-19 state of emergency in the United States
