Integrating Renewable Generation into Electricity Markets

Electricity supply industries in many parts of the world are undergoing disruptive change because of policymakers’ desire to reduce energy-sector greenhouse gas (GHG) emissions. Electrifying energy services such as transportation and space heating can significantly reduce these emissions globally. The transportation, electricity, and space-heating sectors currently account for 28, 27, and 9 percent of US GHG emissions, respectively. It follows that reducing GHG emissions from these sectors by significantly increasing wind and solar energy production is likely to be the most economically viable pathway to sizeable reductions in global GHG emissions.

Reliability of Supply with an Increasing Renewable Generation Share

Managing an electricity supply industry with a large share of wind and solar generation capacity involves many new operational challenges, as demonstrated by the rolling blackouts of August 2020 in California. By replacing natural gas-fired and nuclear generation capacity with solar and wind generation capacity, California substituted on-demand generation capacity with generation capacity that only produces when the underlying resource — wind or sunshine — is available. Figure 1 shows the declining shares of natural gas and nuclear generation and increasing shares of wind and solar generation in California since 2013, the first compliance year of the state’s 33 percent by 2020 renewables portfolio standard goal.

In an empirical analysis of the behavior of the hourly output of more than 50 wind and solar generation unit locations in California, I found a high degree of contemporaneous correlation in the hourly output of individual solar and wind facilities.¹ This implies a “feast or famine” distribution of aggregate wind and solar energy production, which can make it extremely challenging for system operators to meet the difference between the hourly system demand and hourly renewables production, typically called the net demand.

Net demand must be met by natural gas, coal, or other generation capacity that has controllable instantaneous output. These kinds of generation units are called “dispatchable,” to distinguish them from intermittent renewable units. As the amount of wind and solar generation capacity in a region increases, during the majority of hours of the year many dispatchable units are no longer needed to meet the net demand. However, because of the feast or famine nature of wind and solar energy production, there are still likely to be hours during the year when each of these units is required to meet the net demand. Figure 2 presents a graph of system demand, the hour-ahead forecast of system demand, and net demand for August 15, 2020, when 470 MW of load shedding occurred 6:25 pm to 6:47 pm. The rapid disappearance of solar energy production in the evening implies a rapid increase in net demand.

The intermittency of wind and solar energy production also implies that these dispatchable units will have to be switched on and off more frequently. Turning on a natural gas or coal generation unit incurs a significant up-front cost because fuel is burned without injecting electricity to the grid. These dispatchable generation units also have minimum safe operating levels, maximum safe operating levels, and minimum up-time and minimum down-time constraints. In addition, these units have ramping constraints that restrict how fast they move from one output level to another. Finally, transmission network capacity constraints can restrict the ability of all generation units to supply energy to where it is needed.

An increasing the amount of intermittent generation in region makes the non-convexities and indivisibilities in production described above increasingly relevant. In addition, the requirement to deliver all electricity through a transmission network with finite transfer capacity between locations in the grid becomes increasingly important because of the substantial increase in volatility in net demand versus system demand. Christoph Graf, Federico Quaglia, and I demonstrate that electricity market designs that employ simplified models of the transmission network operation that ignore many transmission and generation unit operating constraints (such as the electricity market designs that existed in many regions of the US and currently exist throughout Europe) are increasingly costly to operate, particularly in regions with a growing share of intermittent renewables.² A general conclusion from this work is that the market designer must make the market model that is used to set prices and dispatch generation units match as closely as possible the model that system operators use to operate the system in real time.³

The increased energy supply risk from a larger share of wind and solar resources also increases the expected benefits from risk-management services. Akshaya Jha and I find that the actions of purely financial participants who do not own generation capacity or serve demand can reduce the cost of serving demand, particularly during high-load conditions when all of these operating constraints are likely to be most relevant. The actions of these purely financial participants make the generation schedules that emerge from the day-ahead market closer to how these generation units actually operate in real time, thereby reducing the need for costly increases or decreases in their output in real time.⁴

Strategies for Active Participation of Final Demand in the Wholesale Market

As the share of energy from wind and solar resources increases, system operators have fewer supply options to deploy to maintain real-time system balance at all locations in the transmission network. Consumers can no longer be passive participants in the wholesale market. By shifting the demand for grid-supplied electricity from hours when the wind and solar resources are not producing to the hours when they are allows system operators to maintain system balance with fewer dispatchable resources, thereby reducing the cost of serving demand. Laura Andersen, Lars Gårn Hansen, Carsten Jensen, and I performed a field experiment involving residential consumers in Denmark to investigate the willingness of consumers to shift consumption away from or into certain hours of the day if prodded at short notice using cellphone text messages.⁵

This experiment provided Danish residential consumers with dynamic price and environmental signals aimed at causing them to shift their consumption either “into” or “away” from certain time periods. We found that the same marginal price signal caused substantially larger consumption shifts “into” target hours compared to consumption shifts “away” from target hours. We also found that consumption is reduced in the hours before and after the “into” target hours. There is weaker evidence of increased consumption in the hours surrounding the “away” target hours. The same “into” versus “away” results hold for the environmental signals, although the absolute size of the effects is smaller. For both the price and environmental treatments, the same qualitative results are obtained, but with uniformly smaller quantitative magnitudes. We use these estimates to perform counterfactual experiments in which all of an electricity retailer’s residential customers are assumed to face these dynamic price signals. We find substantial wholesale energy cost savings for the retailer from declaring “into” events designed to shift consumption from higher to lower demand hours within the day, which suggests that such a pricing strategy could significantly reduce the cost of increasing the share of wind and solar electricity generation.

The declining cost of electronic devices enables automated customer-level demand response actions. A number of companies have designed machine learning-based technologies that use Wi-Fi-enabled plugs to control electricity use on the customer’s premises. Understanding how individual appliances are used throughout the day and which uses are flexible can provide important input into determining the efficient deployment of these technologies. Jiyong Eom and I used a field experiment involving commercial customers in South Korea to measure the typical pattern of appliance-level electricity use and the appliance-level responsiveness of these customers to dynamic prices.⁶ We find an important difference between the how commercial versus residential customers respond to dynamic prices. Rather than reducing their consumption in response to individual dynamic pricing events, commercial customers appear to reconfigure their mode of operation in response to facing dynamic prices. Consistent with our reconfiguration hypothesis, small businesses primarily curtailed their electricity usage during peak periods of the day during all days of the experiment period. Appliances not critical to a positive customer experience were the major sources of the energy savings from these reconfiguration actions. Figure 3 shows a representative difference between the mean daily load profile for an appliance for customers in the treatment group and customers in the control group during the dynamic pricing experiment, what we call the “Campaign effect.” The profile labelled “Full Event-day effect” is the difference in the mean daily load profile between customers in our treatment group and customers in the control group during a dynamic pricing event day. It measures the combined impact on appliance use of participating in the experiment and a dynamic pricing event day.

Rooftop versus Grid-Scale Solar Generation Investment

Rooftop solar systems are an alternative source of renewable energy that many customers find attractive because of how the sunk cost of the transmission and distribution networks have historically been recovered. A per-kilowatt-hour (kWh) charge is typically assessed on all energy withdrawn by the customer to recover these sunk costs. This raises the customer’s opportunity cost of consuming grid-supplied electricity, which makes an investment in a rooftop solar system more attractive, despite the fact that utility-scale solar units produce electricity at a significantly lower average cost than a rooftop solar system. Currently in Northern California, the average cost of electricity to residential consumers is close to 20 cents per kWh, despite the fact that the average marginal cost of grid-supplied electricity in 2019 was less than 5 cents per kWh. This creates an incentive for households to install a rooftop solar system that produces electricity at an average lifetime cost of 15 cents per kWh in order to avoid consuming more expensive, grid-supplied electricity. Although this decision is privately profitable for the household, it increases the total cost of supplying electricity to all customers, including those that install rooftop solar systems, because the sunk costs of the transmission and distribution network must now be recovered over a smaller quantity of grid-supplied electricity.

Because installing a rooftop solar system requires substantial up-front costs and a house for the customer to install it on, these systems tend to be clustered in wealthier neighborhoods. During many hours of the day, rooftop solar systems produce more electricity than the customer consumes. This introduces reverse energy flows, which can require expensive distribution network upgrades to accommodate. Using data from the three major California distribution utilities, I find that distribution network prices for each of these utilities more than doubled between 2003 and 2019.

Using time series data on utility-level quarterly rooftop solar capacity, I find that virtually all of the increase in these distribution network prices can be explained by distribution network upgrades to accommodate the distribution network flows associated with rooftop solar investments. The mechanical effect of fewer withdrawals of grid-supplied electricity to recover the same sunk costs of the distribution network explains only a small fraction of the distribution network price increase.⁷ This paper concludes with a description of a distribution network pricing scheme that eliminates the incentive for economically inefficient bypass of grid-supplied electricity.

Carbon Pricing and Electricity Supply

Carbon pricing is an important part of any climate policy for reducing GHG emissions from fossil fuel generation units. Although a carbon tax and a cap-and-trade market can be shown to be equivalent under certainty, they can lead to different outcomes under uncertainty. Over the past 10 years, Trevor Davis, Mark Thurber, and I have developed a web-based Energy Market Game (EMG) in which students own a portfolio of thermal and intermittent renewable generation units and compete to sell electricity in an offer-based wholesale market with uncertain demand with either a cap-and-trade market or a carbon tax. Using the EMG, we compared the performance of three matched carbon-tax/cap-and-trade pairs with equivalent emissions targets, mean emissions, and mean carbon prices respectively.⁸

Across these matched pairs, the cap-and-trade mechanism produced much higher wholesale electricity prices (38.5 to 52.6 percent higher) and lower total electricity production (2.5 to 4.0 percent lower) than the “equivalent’’ carbon tax, without any lower carbon emissions. Market participants that forecast a lower price of carbon in the cap-and-trade games ran their generation units more frequently than those that forecast a higher price of carbon, which caused emissions from the dirtiest generating units — coal and natural gas-fired units with high heat rates — to be significantly higher (15.2 to 33.0 percent higher) than in the carbon tax games. This highlights an important advantage of the carbon tax as a policy. With a carbon tax, the carbon is a known input to the supplier’s production process and there is no disagreement among market participants about the price of this input. Under a cap-and-trade mechanism, market participants can hold different beliefs about the price of carbon, and these differences will typically result in higher wholesale electricity prices.

Directions for Future Research

There are many difficult remaining economic and engineering challenges associated with reducing GHG emissions from the electricity sector. These increase rapidly as the share of wind and solar generation rises above 50 percent. Addressing them will require more active involvement of consumers in the wholesale market, an increasing range of financial tools to manage supply risk, investments in both short-term and long-term storage technologies, spatial and temporal pricing of access to distribution networks, and new protocols for operating the transmission and distribution network.