Data: Timing is Everything

Staying ahead of the curve by staying on top of industry data

Keeping data current, and applicable to your modeling needs, is not a simple task. It is a known theme within the power industry to expect that as soon as you input data, there will be a need for another update. Much of this has to do with today’s markets being much more transparent than in previous years and more data being available than ever before. Deregulation has played a large role in this transformation; with its need for open markets and transparent pricing came the introduction of a slew of new market products.

Fifty years ago when deregulation began, what are now fundamental market drivers (e.g. sub-hourly, capacity auctions, demand response, energy efficiency, etc.) were unheard of at that time.  The rise of various market data available can be attributed in part to deregulation, or simply to the evolution of technology and the markets. Couple this with the increase in computing speed, server technology advancements, and society’s current “instant gratification” attitude, and you have an industry that demands the right data right now. The growth of available data inputs has led to the need for checks and balances and transparency to the underlying fundamentals. There are a lot more moving parts in today’s power industry which culminates to where we are today: professionals with an enormous amount of data to keep up with and incorporate into simulation models.

In an effort to help integrate posted data in a timely fashion, EPIS has summarized some of the major release dates for data across the U.S., that when considered as a whole, can help your annual planning. The data releases below are grouped by subject type and then further color coded by region. Depending on your modeling needs (large region, day-ahead, capacity expansion, nodal, etc.) you will care about different data releases. However, making sure the data is available when you need it is a significant part of the process that applies across all modeling endeavors.



Figure 1: Some of the key market data releases and the time frame they are typically available

An Excel version of this information is also available for download from our website.  When you filter by region you can see a clearer picture of data availability and start to form regional timelines for your own updates based on the available data.

In today’s transparent power markets, staying current can be a difficult task. Knowing when the data is available is an important first step to planning your update schedules in order to most effectively forecast power markets.

Filed under: Data ManagementTagged with: , ,

European Power Market Reporting Changes

Data Transparency Doesn’t Always Mean Ease of Use

The ENTSO-E Transparency Platform has increased the amount of European power market data publicly available in recent years.  While not completely comprehensive, it does help consolidate a vast amount of information in a single location.  ENTSO-E (European Network of Transmission System Operator for Electricity) was established in 2009 for the purpose of “further liberalising the gas and electricity markets in the EU.”  ENTSO-E represents 42 TSOs from 35 countries, including EU countries and non-EU countries like Iceland, Norway and Turkey, among others.

Diverse Levels of Compliance

Unfortunately, the various TSO’s have diverse levels of compliance in reporting data completely, or in some cases on a regular basis, as they follow their own time schedule and level of detail.  Some appear to only report units with installed capacity above 10 MW, while others also report smaller units.  ENTSO-E provides data in two different levels of detail: by unit and by country.  The by-country values are totals for the entire country for units above 1 MW.  By unit, ENTOS-E only asks that its members report details on units above 100 MW; but the actual minimum size for unit detail reported varies by country, as well as the fuel type.  Some countries identify the fuel explicitly, while others simply identify units as thermal, which might be coal, natural gas, fuel oil, or a combination of fuels.  When comparing old data sources to each TSOs publicly-released data, a complete and exact unit-by-unit match with ENTSO-E reported data nearly impossible.

Reviewing ENTSO-E Data by Country

For example, EPIS recently performed an update to resources in Italy.  While gathering data from ENTSO-E at the country level, we found this year over year comparison provided by ENTSO-E.


Figure 1: ENTSO-E: installed capacity by fuel type, by country Source

Note that the 2014 total of 102,547 MW is only a five percent variance to the 2015 total of 97,794 MW. But the interesting values in this report are the variances reported in the different fuel categories.  For instance, there are a number of Production Types that are relatively close year-over-year, but notice that the “Other” category in 2014 was ~37k MW, while 2015 was ~14k MW, resulting in a 63% decrease for that fuel type.  Another set of values also should jump out at the casual observer: “Fossil Hard coal” increased from 1,360 MW to 6,386 MW.  Was Italy introducing new coal units?  No. They were simply modifying their reported fuel type to be more in line with ENTOS-E reporting policies.

Differences in ENTSO-E Data by Unit

Next we reviewed the ENTSO-E data by unit, which is required above 100 MW.


Figure 2: ENTSO-E 2015: installed capacity by fuel type, by unit Source

In this analysis, the item that is most unique is that while the data is now at a finer granularity of detail (i.e. by unit), the “Other” category has now grown larger, to ~43k MW, than the reported values by country of ~14k MW and ~37k MW in 2015 and 2014 respectively.

In other words, their own by unit data is not matching their reported country level totals. What is going on here?  Primarily, when researched further, we found a large number of units, that can rely on multiple fuels, are categorized as “Other” in the by-unit report.  When we then condensed the Production Type detail a little further and compared 2014 and 2015 by country to the 2015 by unit data, we found this:


Figure 3: ENTSO-E: capacity differences reported by country or by unit

After reviewing these summaries, we saw that the renewable fuels are fairly close when comparing by-unit to by-country totals: wind is comparable, GST is also very close, but solar does not compare well since many units are under 1 MW and not included in the by unit report.  This comparison also showed that the totals of thermal and “Other” fuels together are fairly similar and make up over 60% of the total installed capacity in each report.

Moving Forward & Cross-checking

So where to go from here in making sense of reporting variability?  ENTSO-E is currently compiling data submitted by TERNA, the TSO in Italy, and we took a look at what data is available in that report.


Figure 4: TERNA 2015: installed capacity by fuel Source

Two things to note here are that the TERNA resource database only reports units 100 MW and larger, and they have an even smaller set of Production Type groups.  Again we noticed the total capacity reported by unit is very different at ~73k MW versus the ~93k MW from the previous report, but explainable due to renewable sources generally having smaller installed capacity values and therefore not included in this report.  Of note, no solar is reported here, only two wind units that total 243 MW are included, and the reported hydro is approximately 60% of the total MW reported to ENTSO-E.  However, the thermo electric total matches fairly well with the ENTSO-E data at ~60k MW.

So, what have we seen in reviewing these 3 sets of data from these two sources?  ENTSO-E and TERNA have come a long way in providing transparency with their data, but as the details here show, there is still a long way to go before the data can be easily adopted without a lot of scrubbing.

Filed under: European Power Market, Power Market InsightsTagged with: , ,