How Good is the EIA at Predicting Henry Hub?

Natural gas power plants are a key component of bulk electrical systems in North America. In the U.S., natural gas power plants made up the largest portion of installed capacity, 42%, as of December 2016 and contributed more to generation than any other source. In Mexico, natural gas power plants supplied 54% of the required electricity in 2015 and are a key component of the capacity additions in development of the national electrical system. Natural gas is also likely to be the primary energy source in the U.S. due to increased regulation on coal units, uncertainty around the future of nuclear generation, and low natural gas prices.

Natural gas prices are a critical driver of electricity prices and a key input variable in electric power models. Due to the large amount of natural gas power plants in North America, and because fuel costs are the largest cost component of a thermal power plant, wholesale electricity prices are tightly coupled with natural gas prices. There is also an important feedback loop, in that natural gas demand, and price, is tightly coupled to the operation of natural gas power plants. Understanding the interplay between gas and power markets, and uncertainties in forecasts, is critical for forecasting either.

The U.S. Energy Information Administration (EIA) provides natural gas price short-term forecasts through the Short-Term Energy Outlook (STEO) and long-term forecasts through the Annual Energy Outlook (AEO). For the purposes of this article, we will focus on the STEO. The STEO is a monthly report with, among other items, a natural gas consumption and price forecast for 13 to 24 months in the future depending on the month published. The model predicts consumption and prices for three sectors (commercial, industrial, and residential) in the nine U.S. census districts. To do this, the model calculates natural gas consumption and supply levels to build an inventory. Prices are derived from a regression equation using the inventory and heating and cooling degree days, and analysts then make adjustments for final prices. Detailed information on each equation and method is provided by EIA Natural Gas Consumption and Prices document.

How good is the EIA at forecasting natural gas prices from a month to a year out?

To evaluate the STEO forecasts of natural gas prices, we downloaded each monthly STEO report from January 2012 to December 2016 to allow for at least a full year of analysis with historical prices. This period was selected because it is representative of the current trend of low natural gas prices (relative to historical). The mean absolute error (MAE) and mean absolute percent error (MAPE) were calculated for each forecasted value. Prices were then evaluated for the first forecast in each year and a subset of forecasts from consecutive months during a price spike. The mean absolute percent error was also evaluated for each report year and across all reports.

For the period analyzed (2012 to 2016, shown in orange below), the wholesale Henry Hub gas price averaged $3.30/mmbtu with a high price of $6.19/mmbtu in early 2014 due to the extreme Northeast weather (i.e., the polar vortex) and a low price of $1.78/mmbtu due to warm weather conditions and large amount of storage late in 2016. This period is representative of relatively low natural gas prices as compared to the previous five-year period with high prices exceeding $10/mmbtu driven by high oil prices and an average of $5.63/mmbtu despite the sharp decline due to the financial crisis in 2008-2009.

hh1

Figure 1. Historical Henry Hub natural gas prices. The yellow period denotes the study period used for this analysis. Source: EIA.

We started by looking at the longest-term forecasts (24 months) that are delivered in January of each year, and saw an inability to capture rapid fluctuations in prices in the study period:

hh2

Figure 2. Historical Henry Hub gas prices with 24 month forecasts from the January STEO of each year starting in 2012 and ending in 2015 using the base case data. Source: EIA STEO.

The January 2012 forecast missed the sharp reduction in prices from the winter to summer that were driven by high storage volumes. Less volatility occurred over the first part of the January 2013 forecast, however this forecast missed the large increase in prices to over $6/mmbtu which were driven by extreme weather conditions. The January 2014 forecast also missed the weather-driven high price for this period and then was high-biased in the later months of the forecast. The January 2015 forecast was high-biased the entire forecast period and missed the lower prices which were driven by a combination of mild weather and high storage volumes.
The STEO forecast is very sensitive to the initial conditions or starting month’s price. For example, plotting each month’s forecast during the increase from $3.74/mmbtu in November 2013 to $6.19/mmbtu in February of 2014 shows the impact of the rapid change in initial condition (last known price) on the first month forecasted value:

hh3

Figure 3. Historical Henry Hub gas prices with forecasted values from the months leading up to the rapid price spike in February 2014.

Presumably the long-term fundamental drivers of the STEO do not change as much as the initial conditions, and thus the longer-term forecast is much less sensitive to initial conditions.
Despite missing the fluctuation events, on average across the years analyzed the STEO is within 8% of the price in the first month of the forecast, 25% of the price out to eight months and 33% of the price out to 13 months:

hh4

Figure 4. Mean absolute percent error calculated for each forecasted month of STEO reports. Data are averaged over a report year, as well as over all of the report years. Maximum and minimum percent error is calculated over all STEO reports.

On average, the trend has increasing error with forecast length, however, this does not occur in the 13-month 2012 or 2013 STEOs. The expected error growth with time does appear in the 2014 and 2015 STEOs, reaching nearly 60% in the 2014 STEO. The maximum percent error in any given forecast grows rapidly from 26% in the first forecasted month to 75% in the fourth forecasted month, and reaches a high of over 100% 12 and 13 months out.
In absolute terms, the error ranges on average from $0.25/mmbtu in the first forecasted month to $0.88/mmbtu 13 months out. Maximum and minimum errors range from less than a penny up to $2.45/mmbtu.

hh5

Figure 5. Mean absolute error calculated for each forecasted month of STEO reports. Data are averaged over a report year, as well as over all of the report years. Maximum and minimum absolute error is calculated over all STEO reports.

Is the STEO forecast good enough? Unfortunately, as with many answers, it depends. More importantly, however, is understanding the limitations and uncertainties in their gas forecasts. If relying on EIA forecasts, you must realize the sensitivity to initial conditions and the typical error growth in the first months to year of the forecast. With this information, sensitivity studies can be formulated to capture possible fluctuations in gas prices. Taken together with other uncertainties such as demand, transmission outages, and plant outages, you can begin to form an ensemble of forecasts.

Filed under: Natural Gas, UncategorizedTagged with: , , ,

Uncertainty for ERCOT Markets

AURORAxmp data is ready to take on the unpredictable nature of ERCOT’s markets.

The ERCOT reserve margin is by no means certain in 2016. According to the latest NERC 2015 Long-Term Reliability Assessment, ERCOT is showing a healthy reserve in the summer of 2016. However, NERC and others have had a tendency to miss the target in regards to reserve margin in this region. Reviewing projections for the 2015 summer period, the NERC Summer Reliability Assessment showed anticipated reserves in ERCOT of 16.24%, and the final Seasonal Assessment of Resource Adequacy (SARA) from ERCOT agreed that the region was expected to have sufficient capacity to meet peak demands with a 14.26% margin. Interestingly, the final forecast was an abrupt change from the preliminary forecast issued only 2 months prior which anticipated an 11.45% margin, or a 2% shortfall of the NERC reference margin of 13.75%. According to the final report:

The ERCOT Region is expected to have sufficient installed generating capacity to serve forecasted peak demands in the upcoming summer season (June – September 2015)… The primary reason for this change is the summer weather forecast, which generally indicates milder conditions than the 12-year normal forecast used in the Preliminary Summer SARA. As a result, the demand forecast for summer has decreased…

However, a few months later, ERCOT announced in a press release that it experienced its highest peak demand on record, “For the first time in this grid operator’s history, hourly demand within the Electric Reliability Council of Texas (ERCOT) system today broke the 69,000 MW threshold…”. Days later, in another press release, ERCOT reported the record peak was broken again by over 800 MW. Ultimately ERCOT missed the mark in its final, more optimistic report, and this shows how volatile projections can be. Not to say that NERC, FERC, ISO or RTO assessments aren’t excellent tools for understanding some of the fundamentals of a market, it’s just important to remember how significantly reality can differ from constantly changing expectations and how important it is to do analysis around the key fundamental drivers.

Once again, ERCOT has released its latest demand forecast. Has it overstated its margin once again? That question is enough to make one pause. Compliance extensions filed in 2015 for over 5 GW of Mercury and Air Toxic Standards (MATS) forced retirements, expire in 2016. Will they all be compliant and stick around or were they just hoping to operate one more year before finally deciding to retire? A lot of unknowns, but certainly the situation in ERCOT could be much tighter than some of these assessments suggest. With so much going on in 2016 for ERCOT, this year could be a pretty wild ride.

We are in the midst of a large ERCOT update of resources and demand that will be coupled with the latest ERCOT nodal case. Our data is net up in supply, but this is accompanied by an increase in the demand forecast. We have also added demand response units to capture their paramount importance to proper modeling of the system.  This database release is due out in Q1 2016 and is ripe for 2016 summer analysis.  Couple this with AURORAxmp’s risk analysis and you’ll be prepared for the market’s uncertainties.

For additional information, please contact us

Filed under: Power Market InsightsTagged with: , ,