Follow

http://epis.com/powermarketinsights/index.php/2017/05/18/how-good-is-the-eia-at-predicting-henry-hub/

Natural gas power plants are a key component of bulk electrical systems in North America. In the U.S., natural gas power plants made up the largest portion of installed capacity, 42%, as of December 2016 and contributed more to generation than any other source. In Mexico, natural gas power plants supplied 54% of the required electricity in 2015 and are a key component of the capacity additions in development of the national electrical system. Natural gas is also likely to be the primary energy source in the U.S. due to increased regulation on coal units, uncertainty around the future of nuclear generation, and low natural gas prices.

Natural gas prices are a critical driver of electricity prices and a key input variable in electric power models. Due to the large amount of natural gas power plants in North America, and because fuel costs are the largest cost component of a thermal power plant, wholesale electricity prices are tightly coupled with natural gas prices. There is also an important feedback loop, in that natural gas demand, and price, is tightly coupled to the operation of natural gas power plants. Understanding the interplay between gas and power markets, and uncertainties in forecasts, is critical for forecasting either.

The U.S. Energy Information Administration (EIA) provides natural gas price short-term forecasts through the Short-Term Energy Outlook (STEO) and long-term forecasts through the Annual Energy Outlook (AEO). For the purposes of this article, we will focus on the STEO. The STEO is a monthly report with, among other items, a natural gas consumption and price forecast for 13 to 24 months in the future depending on the month published. The model predicts consumption and prices for three sectors (commercial, industrial, and residential) in the nine U.S. census districts. To do this, the model calculates natural gas consumption and supply levels to build an inventory. Prices are derived from a regression equation using the inventory and heating and cooling degree days, and analysts then make adjustments for final prices. Detailed information on each equation and method is provided by EIA Natural Gas Consumption and Prices document.

How good is the EIA at forecasting natural gas prices from a month to a year out?

To evaluate the STEO forecasts of natural gas prices, we downloaded each monthly STEO report from January 2012 to December 2016 to allow for at least a full year of analysis with historical prices. This period was selected because it is representative of the current trend of low natural gas prices (relative to historical). The mean absolute error (MAE) and mean absolute percent error (MAPE) were calculated for each forecasted value. Prices were then evaluated for the first forecast in each year and a subset of forecasts from consecutive months during a price spike. The mean absolute percent error was also evaluated for each report year and across all reports.

For the period analyzed (2012 to 2016, shown in orange below), the wholesale Henry Hub gas price averaged $3.30/mmbtu with a high price of $6.19/mmbtu in early 2014 due to the extreme Northeast weather (i.e., the polar vortex) and a low price of $1.78/mmbtu due to warm weather conditions and large amount of storage late in 2016. This period is representative of relatively low natural gas prices as compared to the previous five-year period with high prices exceeding $10/mmbtu driven by high oil prices and an average of $5.63/mmbtu despite the sharp decline due to the financial crisis in 2008-2009.

Figure 1. Historical Henry Hub natural gas prices. The yellow period denotes the study period used for this analysis. Source: EIA.

We started by looking at the longest-term forecasts (24 months) that are delivered in January of each year, and saw an inability to capture rapid fluctuations in prices in the study period:

Figure 2. Historical Henry Hub gas prices with 24 month forecasts from the January STEO of each year starting in 2012 and ending in 2015 using the base case data. Source: EIA STEO.

The January 2012 forecast missed the sharp reduction in prices from the winter to summer that were driven by high storage volumes. Less volatility occurred over the first part of the January 2013 forecast, however this forecast missed the large increase in prices to over $6/mmbtu which were driven by extreme weather conditions. The January 2014 forecast also missed the weather-driven high price for this period and then was high-biased in the later months of the forecast. The January 2015 forecast was high-biased the entire forecast period and missed the lower prices which were driven by a combination of mild weather and high storage volumes.
The STEO forecast is very sensitive to the initial conditions or starting month’s price. For example, plotting each month’s forecast during the increase from $3.74/mmbtu in November 2013 to $6.19/mmbtu in February of 2014 shows the impact of the rapid change in initial condition (last known price) on the first month forecasted value:

Figure 3. Historical Henry Hub gas prices with forecasted values from the months leading up to the rapid price spike in February 2014.

Presumably the long-term fundamental drivers of the STEO do not change as much as the initial conditions, and thus the longer-term forecast is much less sensitive to initial conditions.
Despite missing the fluctuation events, on average across the years analyzed the STEO is within 8% of the price in the first month of the forecast, 25% of the price out to eight months and 33% of the price out to 13 months:

Figure 4. Mean absolute percent error calculated for each forecasted month of STEO reports. Data are averaged over a report year, as well as over all of the report years. Maximum and minimum percent error is calculated over all STEO reports.

On average, the trend has increasing error with forecast length, however, this does not occur in the 13-month 2012 or 2013 STEOs. The expected error growth with time does appear in the 2014 and 2015 STEOs, reaching nearly 60% in the 2014 STEO. The maximum percent error in any given forecast grows rapidly from 26% in the first forecasted month to 75% in the fourth forecasted month, and reaches a high of over 100% 12 and 13 months out.
In absolute terms, the error ranges on average from $0.25/mmbtu in the first forecasted month to $0.88/mmbtu 13 months out. Maximum and minimum errors range from less than a penny up to $2.45/mmbtu.

Figure 5. Mean absolute error calculated for each forecasted month of STEO reports. Data are averaged over a report year, as well as over all of the report years. Maximum and minimum absolute error is calculated over all STEO reports.

Is the STEO forecast good enough? Unfortunately, as with many answers, it depends. More importantly, however, is understanding the limitations and uncertainties in their gas forecasts. If relying on EIA forecasts, you must realize the sensitivity to initial conditions and the typical error growth in the first months to year of the forecast. With this information, sensitivity studies can be formulated to capture possible fluctuations in gas prices. Taken together with other uncertainties such as demand, transmission outages, and plant outages, you can begin to form an ensemble of forecasts.