Using Prices, Automation, and Data to Shape Electricity Demand and Integrate Renewables into the Grid

To realize the ambitious clean electricity goals of many states and the Biden administration, variable renewable energy sources will need to be effectively integrated into the electric grid. This report presents a summary of an online workshop convened by Resources for the Future (RFF) in December 2021. RFF convened a group of economists, industry officials, policymakers, data aggregators, and regulators to discuss the role that time-varying pricing, device automation, and high-frequency data can play in shaping electricity demand and aiding renewables integration effort. Existing regulatory, technological, and economic barriers have hindered progress in these efforts. These barriers include regulatory inertia, fears of retail bill volatility, and potentially less-effective rebates for reducing peak consumption. Recent research provides reason for optimism, however, as consumers appear to be cognizant of prices and willing to cede control of some types of electricity consumption to automated processes. Advances in machine learning may also help utilities effectively evaluate the consumption impact of different types of electricity rates. Moving forward, there are ample opportunities for researchers to partner with smart thermostat and other smart device companies, nonprofits, and data aggregators for insights on effective strategies to engage flexible demand from the vast amount of high-frequency consumption data made possible by smart meters.

To realize the ambitious clean electricity goals of many states and the Biden administration, variable renewable energy sources will need to be effectively integrated into the electric grid. In December 2021, Resources for the Future convened a group of economists, industry officials, policymakers, data aggregators, and regulators to discuss the role that time-varying pricing, device automation, and high-frequency data can play in shaping electricity demand and aiding in the effort to integrate renewables into the grid (i.e., accommodate variable generation from renewables in grid operations and load balancing). To date, existing regulatory, technological, and economic barriers have hindered progress in the efforts to use these tools to effectively shape electricity demand and integrate renewables into the grid. The goals of the RFF workshop were to discuss the nature of these barriers, learn what the latest research has to say about solutions in this space, and identify opportunities for partnerships among academics, utilities, and private-sector companies and data aggregators for future research.

The need for demand-side management and time-varying prices to aid in renewables integration is often illustrated through the classic “duck curve,” which charts net electricity load, or demand, that system operators must meet throughout the day

(Figure 1). In a system with a relatively large penetration of renewables, the amount of generation from nonrenewables necessary to meet electricity load rises dramatically as the sun sets and intermittent solar generation ceases to produce. This is particularly problematic because this loss of renewable generation coincides with times of peaking electricity demand in the evening as customers return to their homes and fire up electric-hungry appliances. Utilities have historically attempted to turn to time-varying electricity prices that try to discourage electricity use during these peak times and demand-side management programs that offer payments or other incentives to reduce electricity consumption during peak hours.

Another challenge posed by the duck curve is that as solar generation grows, the level of net load during hours of abundant renewable generation continues to decline, bringing with it a decline in the value of additional renewable energy, a fall in the daytime wholesale price, and an increased likelihood that renewable generators could be curtailed as a result of insufficient demand for power during certain hours in certain locations. Passing price signals that reflect these variations in cost across the day to consumers could facilitate demand shifting toward periods of lower generation costs, raising the value of renewable generators during hours of operation and helping flatten the ramp up in demand at sunset.

Coincident in time with growth in variable and intermittent renewables is a growing desire to electrify energy end uses typically powered by fossil fuels, such as vehicles and space and water heating. Opening remarks at the workshop highlighted that those new sources of electricity demand may be more flexible than others to the extent that electricity consumption and energy service consumption can be (or in some cases necessarily are) separated in time. Electrifying these end uses is an important part of the energy transition ahead and a larger strategy to decarbonize the economy through reduced fossil fuel use. To the extent that electricity rate design affects incentives to invest in electric vehicles (EVs) and electric heat pumps, these considerations will also play a role in larger questions of rate design to engage flexible demand. Allocation of fixed charges for grid access to customers and end uses could play a role in creating incentives for electrification.

As the opening speaker said, perhaps the question for potential EV owners and for grid operators is, who owns your driveway and is responsible for the operation of the EV charger that resides there? Might subscription services that couple the device with the service be a way of saving consumers from having to actively manage vehicle charging and realize associated cost savings in the presence of time-varying energy rates? The speaker also stressed that distributional impacts on consumers need to be part of the conversation around retail rates. The energy transition that will be needed for decarbonization cannot be completed cost-effectively until the demand side is fully engaged.

As the workshop began, audience members made note of several points that helped guide the discussion over the course of the afternoon. These included questions about the extent to which customers need to be engaged and attentive to electricity consumption and demand-side programs in order to reshape energy use. Additionally, another participant noted that advances in renewable technologies and distributed generation have muddied the traditional idea of separate demand and supply sides of the electricity market, with “prosumers” that both produce and consume electricity gradually playing a larger role in the future of electricity markets.

Engaging electricity consumers through demand-side management programs and electricity prices that vary throughout the day to accurately reflect social marginal costs can greatly reduce the cost of renewables integration. To date, multiple barriers have prevented active demand-side management programs and time-varying rates from being implemented. The first workshop panel focused on identifying these barriers and how they can be overcome.

One barrier to widespread implementation of time-varying pricing identified by the panelists is the difference in policy and regulatory frameworks across states. The largest example of this is the difference in the adoption and deployment of smart meters over time by utilities in different states. Smart meters provide the electric utility with the ability to track electricity consumption at a granular timescale throughout the day, rather than the old approach of having a meter reader read a customer’s aggregate consumption once per month. Smart meters are a necessary first step, as demand-side management programs and time-varying rates are generally difficult or impossible to implement without the ability of the utility to track household consumption over time. Figure 2 shows substantial heterogeneity in the proportion of electricity customers serviced by smart meters, with at least five states having fewer than 15 percent of homes serviced by smart meters as of the end of 2019. Until all states can get closer to universal smart meter deployment, it will be difficult for demand-side management programs and time-varying rates to become the default electricity pricing approach in the United States.

Inertia in the policymaking and regulatory process was identified as another important barrier to effective implementation of time-varying pricing. In many jurisdictions where time-varying prices have been adopted, the process has taken several years from initial proposal of the rates to actually charging customers time-varying prices. Once such rates have been proposed and implemented, utilities have generally been reticent to change them. Ontario, Canada, provides one such example, as less than 10 percent of customers opt out of a time-varying price for a more traditional regulated rate (Faruqui and Bourbonnais 2020). The most popularly adopted time-varying electricity pricing scheme is the time-of-use (TOU) rate, in which a typical day is split into discrete periods (such as on- and off-peak periods), and a different price is charged for electricity consumed in each period. TOU prices are typically easy to understand and have the benefit of being set in advance, so they are known to customers. However, the panelists generally agreed that TOU rates are not granular enough, nor are the prices often differentiated enough to send the correct price signal to customers to reduce electricity consumption at the times when it most beneficial to the grid. The panelists preferred rates that allow utilities to call special demand events and implement critical-peak prices, or fully dynamic time-varying rates, but these rates are not often observed in practice, and given the existing inertia, it may be difficult to get policymakers to move beyond TOU rates.

On a related note, the idiosyncratic nature of the electricity ratemaking process in general was also mentioned by several panelists as another barrier to implementing the types of time-varying electricity prices that would help integrate renewables into the grid effectively. When designing electricity rates, utility officials are generally charged with balancing multiple goals, none of which are directly concerned with prioritizing time-varying electricity prices or demand charges. These goals include adequate recovery of revenue to cover costs, efficiency concerns, simplicity of the rate structure, and increasingly, concerns about fairness, equity, and affordability of the rate structure for low-income households. For example, time-varying rates can be complex and difficult for the average electricity consumer to understand.

Another common barrier identified was utilities’ reticence to adopt rate structures that may introduce a large degree of bill volatility for their residential customers, a particular concern for real-time prices. Especially during peak demand periods, the social marginal cost of the marginal kilowatt-hour (kWh) of electricity consumed is much higher than at other times of the day, and time-varying prices that accurately reflect these social marginal costs may result in much higher electricity prices than residential customers are used to paying (Borenstein and Bushnell 2018). Panelists and audience members stressed the need for more creative types of electricity contracts offered to residential customers that could help reduce the potential for bill volatility, such as hedging instruments. These could come in the form of subscription-style contracts, in which a household contracts for a specific amount of power and then only pays the difference if it goes over its allotted power for the month (and similarly is rebated if it does not use the entire allotment).

One panelist explained that regions with retail competition in electricity supply, such as Texas, offer more flexibility in rate design. In Texas, NRG customers are offered a subscription program in which they get a flat bill that does not vary with kWh use, and this service can be combined with a smart thermostat that the provider can control. Such services need to be priced in a way that accounts for potential adverse selection by high users and for moral hazard of the free marginal kWh resulting in demand increases. Provisions that limit free additional energy consumption up to a point should also be considered. These types of contracts have the potential to provide significant savings to customers who are willing to surrender control of their thermostats within agreed parameters.

Finally, panelists noted that some existing demand response programs operated by the utilities and the regional transmission organizations may be paradoxically hindering the renewables integration transition through the perverse incentives created by payments under these existing programs. Many existing demand response programs offer big incentive payments to heavy electricity users in the industrial and commercial sectors for being available to curtail their electricity use if called to do so; these can be thought of as business attraction rates for the utilities. Some companies build their entire business models around these demand response programs and, as a result, lobby to keep them in place, thereby preventing further experimentation with dynamic pricing and other forms of consumer engagement. Paying for demand response introduces a moral hazard problem in that the electricity system operators cannot directly base financial rewards on customer effort to reduce demand. Reductions are not observable, only levels, so there is no way to tell whether customers are making an effort to reduce demand. Indeed, sometimes programs create incentives to increase demand during peak hours on noncritical peak days in order to increase the baseline against which payments for demand response are assessed. An adverse selection problem also arises in that these demand response programs are often opt-in by nature. This means that the type of firms that opt in to the programs may be those that already have relatively low electricity use during system peak periods, so the consumption levels they exhibit when called do not necessarily reflect true “additional” reductions in energy use necessary to reduce peak load.

The panel did note opportunities moving forward in this space. One prominent solution discussed was energy storage technologies. While technologies like batteries have largely been too expensive to scale, advances in household devices such as electric vehicle charging technology and smart water heaters are changing the game and allowing for energy storage in ways that have not been possible before. For example, water heaters can store thermal energy and keep water hot for many hours after the initial electricity use for heating. In essence, this allows the water heater to act as a battery, shifting electricity consumption away from peak hours by heating water overnight and during other low-cost periods of the day. Figure 3 illustrates how this load shifting works. As more households install electric water heaters and drive electric cars, it will be crucial to exploit the energy storage opportunities that these technologies offer.

After considering the barriers to using demand-side management, workshop participants moved into a discussion of recent research on electricity pricing, economic incentives, and smart technologies. Presentations focused on the role that such smart technologies can play in shaping demand and helping overcome the barriers discussed in the previous panel. Panelists largely agreed that informed electricity consumers do respond to prices. However, they also agreed that nonprice factors, such as inattention and complex rate structures, play a critical role in potentially limiting the effectiveness of prices in shaping demand. Our understanding of how these factors affect demand and how they can be addressed is incomplete, and good research designs informed by institutional knowledge are important to improving that understanding.

If addressing customer inattention is likely an important factor in shaping demand, what is the proper role for automation? Will customers cede control of the thermostat and allow for automatic heating and cooling adjustments within the home? One study addressed this question through the use of a randomized experiment in which existing smart thermostat customers were encouraged to activate a feature that partially automated customer cooling patterns in response to time-varying prices. Customers who were only sometimes or rarely at home saw reductions in central air conditioner run times without correspondingly large increases in household discomfort, as measured by deviations in realized indoor temperature from desired temperature, indicating that households in the experiment were willing to trade small monetary energy savings for small increases in discomfort. These trends are illustrated in Figure 4. Further experimentation with automation programs such as these paired with time-varying rates could prove promising in shifting electricity demand in time and further aiding renewables integration efforts.

Other research presented in the session yielded promising results for the future of automation and time-varying prices in shaping electricity demand. Evidence from simulation-based modeling shows that TOU pricing for both energy and demand charges for commercial load can lead to large operating cost savings for electric bus fleets by providing incentives to shift times at which they charge away from peak periods. Another study focuses on new methods for evaluating the effect of peak pricing policies on electricity consumption. While randomized controlled trials are often held up as the gold standard in the evaluation of public programs, they can be difficult and costly for utilities to conduct. The presented research demonstrates that machine learning techniques can accurately predict counterfactual electricity consumption (i.e., what consumption would have been if a policy had not been introduced) and thus can replicate experimental findings on how pricing and other policies affect consumption. Machine learning techniques paired with high-frequency consumption data from smart meters will enable utilities and others with access to meter data to evaluate new pricing mechanisms and other policies more easily, and without the time and monetary costs of running a full randomized evaluation.

Broader discussion among the panelists and attendees focused on several areas where additional research could be particularly valuable. These include commercial and industrial buildings, which have been generally understudied by economists. The heavy-duty trucking industry is another fruitful area for research. Contrary to popular perception, most heavy-duty trucks are privately owned by smaller owner-operators and not by large corporations, and thus getting the owners to both electrify their vehicles and employ smart charging will take significant effort. Research about distributional impacts of rate designs is also lacking, due in part to limited information about household socioeconomic characteristics in metered data. Having access to this information could enable a better understanding of heterogenous impacts of various pricing approaches and other measures to shape electricity demand. Lastly, the next frontier for research in this space could be to study energy demand management by a utility or other intermediary that is not the customer. Would customers be open to a program that sets a monthly service price and, with their permission, allows an intermediary to manage air conditioner operation in the home (or other end uses) to take advantage of energy price fluctuations? Or would customers opt out of such a service?

Several key insights ultimately emerged over the course of the workshop. While customers, particularly well-informed ones, do respond to prices, inattention and inconvenience can limit responses to frequent price changes, and smart devices and automation may potentially play a large role in enabling price responsiveness and shifting consumer demand to help integrate renewables into the grid. The workshop highlighted research suggesting that automated responses to price can save customers money in ways that do not diminish their experiences of energy service provision, such as comfort provided by HVAC systems. Abundant data from devices and smart meters enable the use of new machine learning methods to study the effectiveness of pricing and other approaches in shaping demand. Existing opportunities for load aggregators to profit from shaping electricity demand tend to focus on competing in traditional supply-side markets and displacing the need for generation during peak hours. These programs may not incentivize sufficient demand flexibility for renewables integration and may also be a barrier to further adoption of more granular time-varying prices going to consumers or new types of demand aggregators that manage electricity use on behalf of customers.

Workshop participants also identified several areas where additional research would be valuable. Understudied aspects of the problem include distributional concerns and the characterization of different groups of customers to identify where opportunities for demand flexibility may be greatest. Much high-value research remains to be done on how best to influence and shape demand in the commercial and industrial sectors as well. While monthly subscription offerings coupled with provider control of smart home devices are starting to appear in some places, more formal research on consumer acceptance of such models and their costs and performance could help identify potential opportunities in this space. As the penetration of smart meters and smart devices in homes and businesses across the United States is only expected to rise in the coming years, opportunities to implement new types of dynamic pricing and demand-side management programs will grow as well. Research can help identify which combinations of prices, policies, and new service offerings will most be most effective in shaping demand on the grid of the future.

Click here to view the full report with references.