Retrospective Analysis of Environmental Regulations

Policy leaders in the regulated community, Congress, and academia, as well as some in the broader public, continue to express an interest in credible retrospective analyses that can help document the extent to which the expected benefits of regulation have been realized and at what cost. Such studies can illuminate any unintended consequences, such as adverse distributional outcomes or enhancement of market power. And they can support innovation in regulatory design and help guide the reform of poorly performing rules.

Although prospective, ex ante studies—known as regulatory impact analyses (RIAs)—are now routine for major new rules, retrospective measurements of actual outcomes, based on quasi-experimental or other modern methods, are rare. Recall Michael Greenstone’s often-quoted observation that RIAs are developed at the “point when the least is known and any analysis must rest on many unverifiable and potentially controversial assumptions.” Michael Greenstone, “Toward a Culture of Persistent Regulatory Experimentation and Evaluation,” in New Perspectives on Regulation, eds. David Moss and John Cisternino (Cambridge, MA: Tobin Project, 2009), 111–26. The Evidence Based Policymaking Act of 2019 is an important step forward, and the pending Smart Act could further advance ex post studies of federal regulation. Nonetheless, many barriers remain.

This paper describes the results of seven new quantitative analyses of major federal environmental regulations plus a paper that assesses a range of institutional design issues to support future retrospective studies, including operation of the Evidence-Based Policymaking Act (EBPA) of 2019 and a legislative bill, the Setting Manageable Analysis Requirements in Text (SMART) Act. The quantitative studies focus on specific outcomes of regulation rather than broad-brush results and differ from studies that simply update the ex ante assumptions used in pre-regulatory analyses.

The paper is divided into five sections. Section II provides background on retrospective analyses of federal regulations.

Section III reviews the seven new quantitative studies developed for the project, including a description of the process used to select these studies and the overall rationale for the cases. This section also presents analytic lessons from the new studies, including findings about the accuracy of any ex ante analyses, and assessments of the distributional effects of the rules, whether studied ex ante or not. Additional lessons concern new evidence on the interactions and strategic behavior of state and local officials in response to major federal initiatives. Some new categories of benefits not covered in the original ex ante analyses are revealed. Further, several studies identify new or improved methodologies and data sources that could be used in future ex ante or ex post studies. Section III also offers insights into both the analytical and the policy aspects of the individual studies. Examples of the former include impacts of the ZEV program, potential limits on emissions trading, reform of auto registration fees, and reform of the boutique fuels program. Examples of the policy-relevant implications involve potential reform of the RFG and boutique fuel programs, and consideration of damage-based vehicle registration fees in lieu of the current state and local registration fee regimes, and potential reform of the industry assistance elements of California’s cap-and-trade program.

Section IV focuses on institutional issues related to both the Evidence-Based Policymaking Act and the pending SMART Act. It discusses the institutional challenges identified in a conference sponsored by RFF in March 2022. Section V presents overall conclusions and a framework for further work in this area.