Forecaster Report

Transcription

Forecaster Report

AWS Truewind’s Final Report for the
Alberta Forecasting Pilot Project
Submitted To:
Alberta Electric System Operator
2500, 330 – 5th Avenue SW
Calgary, Alberta, Canada
T2P 0L4
Attention: Darren McCrank, P. E.
Submitted By:
AWS Truewind, LLC
463 New Karner Road
Albany, NY 12205
Tel: 518-213-9944
Fax: 518-213-0045
Submitted On:
June 25, 2008
Table of Contents
1
INTRODUCTION ....................................................................................................................................................... 5
2
BACKGROUND INFORMATION .......................................................................................................................... 6
2.1
2.2
3
AWST’S PILOT PROJECT FORECASTING SYSTEM ................................................................................................ 6
OVERVIEW OF THE FORECAST EVALUATION PERIOD ........................................................................................... 8
LESSONS LEARNED AND FINDINGS ............................................................................................................... 13
3.1 COMPARATIVE FORECAST PERFORMANCE ......................................................................................................... 13
3.2 THE I MPACT OF FORECAST SYSTEM COMPONENTS ON FORECAST PERFORMANCE .......................................... 14
3.3 THOUGHTS ON REGIONAL GROUPING OF SITES .................................................................................................. 17
3.4 IMPACT OF W EATHER REGIMES AND EVENTS ON FORECAST ERROR................................................................ 18
3.4.1 Change in the Mountaintop Cross-Mountain Wind Component.............................................................. 18
3.4.2 Shallow Cold Air Masses ............................................................................................................................ 21
3.4.3 Strong Cold Surges...................................................................................................................................... 26
3.4.4 Nocturnal Mountain Outflow...................................................................................................................... 29
3.5 IMPACT OF WIND FARM DATA ............................................................................................................................ 30
3.6 FORECASTING OF RAMPS ..................................................................................................................................... 37
3.6.1 Ramp Forecasting Considerations............................................................................................................. 37
3.6.2 Frequency of Ramps During the Project Year .......................................................................................... 38
3.6.3 Analysis of Ramps for Quarter 3 (Nov-Jan) by Weather Regime. ........................................................... 40
3.6.4 Performance of a Simple Ramp Forecast System ..................................................................................... 42
3.7 AWST FORECASTING ISSUES .............................................................................................................................. 48
3.7.1 Shallow Cold Air Cases .............................................................................................................................. 48
3.7.2 Problems with Rapid Update Cycle (RUC) forecasts ............................................................................... 51
4
RECOMMENDATIONS.......................................................................................................................................... 54
4.1
4.2
4.3
5
CENTRALIZED VS. DECENTRALIZED FORECASTING SYSTEM ............................................................................. 54
TECHNICAL REQUIREMENTS ................................................................................................................................ 55
METHODOLOGY AND APPROACH ........................................................................................................................ 56
NEXT STEPS IN ALBERTA .................................................................................................................................. 58
5.1 ESTABLISH FORECASTING PRIORITIES ................................................................................................................ 58
5.2 UPGRADE QUALITY OF DATA FROM WIND GENERATION FACILITIES ............................................................... 59
5.3 DEVELOP AN OFF-SITE REAL-TIME METEOROLOGICAL DATABASE .................................................................. 59
5.4 ESTABLISH AN ALBERTA-ORIENTED FORECASTING R&D PROGRAM ............................................................... 61
5.4.1 Customization of the Regime-Based Forecasting Concept....................................................................... 61
5.4.2 Use of Flexible Regimes.............................................................................................................................. 62
5.4.3 Expanded Use of the RUC Approach......................................................................................................... 62
6
NEXT STEPS IN INDUSTRY................................................................................................................................. 63
6.1
6.2
6.3
6.4
6.5
ESTABLISH FORUMS FOR COMMUNICATION ....................................................................................................... 63
ESTABLISH FORECAST PERFORMANCE BENCHMARKS ....................................................................................... 63
IMPROVE DATA QUALITY FROM WIND FARMS................................................................................................... 64
ADVOCATE/LOBBY FOR RELEVANT N EW MEASUREMENT SYSTEM TECHNOLOGY .......................................... 64
SUPPORT WIND POWER PRODUCTION FORECASTING R&D............................................................................... 65
1
Executive Summary
AWST used its eWind forecasting system to produce 1 to 48-hour ahead wind and power
production forecasts for 12 wind generation facilities (WGFs) and 5 aggregates of those WGFs
for AESO’s Pilot Project. The forecasts were delivered on an hourly basis from May 1, 2007 to
April 30, 2008. An extensive evaluation of forecast performance was done by ORTECH.
The interpretation of results from a forecast evaluation experiment should consider the
representativeness of the meteorological regimes during the project’s forecast evaluation period
and the pre-project period for which data was provided for potential use in training the statistical
components of a forecast system. Wind speed anomaly data that were available for the first 9
months of the project indicated that there were no monthly wind speed anomalies greater than
about 15% of the 10-year average speeds in any of the project’s regions. However, there were
major positive monthly wind speed anomalies of 20% or more in the project’s regions in three of
the 12 months (all in the winter) preceding the start of the evaluation period. The large anomalies
in some months of the pre-project year suggest the possibility that the pre-project data may not
have been representative of typical Alberta weather regimes in certain seasons.
The eWind system produced forecasts with an ensemble of physics-based (also called Numerical
Weather Prediction or NWP) and statistical models. The system was configured to employ 5
different physics-based simulations that were generated by 2 different physics-based models
from 4 different initialization datasets. The grid point output data from these physics-based
simulations along with recent data from WGFs was used as input into an ensemble of statistical
models that employed different statistical methods, input data, training sample sizes and training
time periods and update frequencies. This suite of physics-based and statistical models produced
an ensemble of forecasts for each 1-48-hour-ahead forecast hour. An additional statistical model
was used to construct a composite forecast (called the optimized ensemble forecast) by optimally
weighting each individual forecast according to its recent performance to minimize the root
mean square error of the forecasts. A statistical wind plant output model, based on observed
meteorological and power production data from each site, was used to convert the forecast of
meteorological variables to a prediction of power production.
AWST’s experience indicates that the overall magnitude of the forecast error, as measured by
parameters such as the RMSE or MAE, for individual Alberta WGFs for the Pilot Project period
were, in general, substantially higher (20% to 30%) than the magnitude of error achieved for
many individual WGFs in other parts of North America such as in California or Texas. There are
a number of factors that combined to make wind power production forecasting for the Alberta
sites in this project more difficult. The most significant factor is that the forecasting of nearsurface winds in the typical weather regimes that affect the Alberta wind generation regions is
more difficult than forecasting the near-surface winds in the typical weather regimes in other
North American locations.
A number of lessons were learned while monitoring the weather conditions and forecast
performance during the forecast evaluation period and through a post-project analysis of forecast
performance. A basic set of knowledge gathered from this process was a better understanding of
how forecast error was linked to the Alberta-specific weather regimes. The more significant
weather regimes and events included (1) changes in the strength of the cross-mountain flow at
the mountain top level; (2) the onset and termination of shallow cold air (SCA) masses; (3) cold
air surges; and (4) nocturnal cooling. Each of these has unique factors that significantly impact
2
forecast performance. Power production changes during SCA episodes were especially difficult
to forecast. Further investigation of each of these regimes has a high probability of yielding
improved forecast performance.
The analysis of AWST’s forecast performance data also indicated that the quality of
meteorological and operational (e.g. turbine availability) data from the WGFs substantially
impacted forecast performance. A manual examination of the location of the meteorological
towers and turbines at each facility indicated that facilities that had a more representative
placement of a meteorological tower had less scatter in their wind speed vs. power production
relationship. Furthermore, an analysis of data for a portion of the project year indicated that 62%
of variation in the forecast MAE among the WGFs could be explained by the degree of scatter in
the relationship between measured wind speed and power production.
During the course of the project, there was considerable interest and discussion in the forecasting
of large ramp events. ORTECH analyzed the performance of implied ramp events in the hourly
forecast data delivered by the three forecast providers. In this analysis a large ramp event was
defined as a 1-hour change in power production of more than 20% of capacity. The evaluation
presented in ORTECH’s quarterly reports suggested that all three forecast providers had very
little skill in forecasting ramp events that were evaluated in this manner. One point that was
immediately noted in the discussion is that neither AWST, nor the other forecast providers, had
configured their forecast systems to optimize performance for ramp events.
AWST performed a post-project investigation of the large ramp events that occurred in the
aggregate production of the existing WGFs during the project’s 3rd quarter. The 20 largest
upward and the 20 largest downward ramps were analyzed. The focus of this investigation was
on the identification of the meteorological processes that caused these events and the estimation
of how much information about these was present in the eWind forecast dataset. This analysis
indicated that a large majority of these events were associated with either (1) changes in the
cross-mountain component of the wind at the mountain-top level and concurrent changes in the
stability of the low-level atmosphere; (2) the advance or retreat of a shallow cold air mass that
caused significant changes in the low-level atmospheric stability; or (3) cold surges (i.e. strong
cold fronts) from the north or northeast. Upward ramps were more common with significant
changes in cross-mountain flow and downward ramps were more frequent in cold air surges.
An examination of AWST’s delivered (optimized ensemble) forecast and forecasts directly from
the output of the physics-based model simulations indicated that the majority of ramp events
were captured by the forecast system but that their amplitude was much too low. The negative
bias in the ramp amplitude forecast was larger in the optimized ensemble forecast than in the raw
physics-based model forecasts. This is most likely due to the fact that the minimization of
RMSE that is implicit in the statistical methods used to construct the optimized ensemble
forecast tends to “hedge” in large ramp forecast cases by lowering the amplitude and extending
its duration in order to minimize the hourly errors. The large negative bias in the forecasted
amplitude explains why ORTECH’s evaluation, which required an amplitude between 80% and
120% of the observed amplitude indicated very poor performance. If a 40% of actual amplitude
threshold within a +/- 3-hour window is used as the hit (i.e. successful detection) criteria, the hit
rate is about 60%. This indicates that signatures of these events are present in the forecasts but at
a much lower amplitude. Interestingly the hit rate was about 70% for upramps and 50% for
downramps, which suggests that downramps are more difficult to forecast than upramps. This
analysis suggests that there is a considerable amount of information about large ramps within the
3
NWP model output data and that there is potential to produce useful large ramp forecasts on
Alberta through the use of statistical tools that effectively extract this information.
AWST identified several specific areas in which the performance of its eWind system during the
Pilot Project could have been improved. One area was the more effective mapping of the
differences in the characteristic NWP model forecast errors between weather regimes. A postproject analysis indicated that the error characteristics of the NWP models were quite different in
the shallow cold air (SCA) regime and the non-SCA regimes. A rerun of the forecasts using a
modified forecast system configuration that more explicitly accounted for this characteristic,
resulted in a 10% to 15% improvement in AWST’s overall MAE for the existing sites for 6 to 48
hour-ahead forecasts during the third quarter. This demonstrates how an analysis of forecast
errors and Alberta-specific forecast system research could improve forecast performance.
Based on AWST’s experience in the Pilot Project as well as in forecasting for other jurisdictions,
AWST recommends that a dual forecaster centralized wind power production forecast system be
implemented in Alberta. A centralized system is likely to have lower cost per facility and more
consistent performance although it is possible that a decentralized may produce superior
forecasts for some individual facilities due to the more extensive site-specific customization that
may occur in that mode. A centralized system is also more likely to use meteorological data from
individual facilities for the improvement of forecasts for other facilities and aggregates of those
facilities. AWST also recommends that the design of a forecasting program also include a
requirement or incentive for WGFs to provide high quality data in near real-time. Multiple-level
measurements of temperature and wind are especially important in Alberta due to the many
weather regimes (such as SCA events) in which the low-level stability and wind shear play an
important role in determining the changes in power production. Another important consideration
in the implementation of a forecast system is a determination of the specific objectives that are to
be achieved by the implementation of a forecast system. It should be kept in mind that a forecast
system that is designed to meet one objective is likely not to achieve optimal performance for a
different objective. Multiple forecast products will likely be needed to achieve optimal
performance for each objective if there are multiple forecast objectives.
From AWST’s perspective, the most critical next step in Alberta is to establish a prioritized list
of objectives for a wind power production forecast system. Other steps that should be taken
include (1) establishing requirements or incentives to upgrade the quality of meteorological and
operational data from WGFs; (2) developing an off-site near real-time meteorological database;
and (3) establishing an Alberta-oriented forecasting research and development (R&D) program.
There are also a number of steps that the industry as a whole can take to improve future forecast
performance in Alberta and other regions. One of the most important is to establish ongoing
forums for effective communication between forecast users and forecast providers. Other actions
that should be taken by the industry to improve wind power production forecast include (1)
implementing a mechanism for ongoing benchmarks of forecast performance; (2) establishing
measurement and communication standards for WGFs that promote the gathering and
dissemination of high quality meteorological and operational wind plant data; (3) advocating for
the development and deployment of new measurement system technology that will have
significant benefits for wind power production forecasting; and (4) encouraging relevant
research-oriented public and private entities to support wind power production forecasting R&D
programs in North America and identify which forecasting problem areas (e.g. large ramp
forecasting) are most critical to the industry.
4
1 Introduction
AWS Truewind (AWST) participated in an experimental wind power forecasting project with the
Alberta Electric System Operator (AESO). From 01 May 2007 through 30 April 2008, AWST
delivered a 48-hour forecast each hour, for a total of 24 forecasts per day or 8784 forecasts.
Each forecast included the predicted hourly average wind speed and power production for 48
hours into the future. Forecasts were delivered in real time for 7 existing wind generation
facilities (WGFs), 5 future WGFs as well as 4 regional aggregates of 3 WGFs each, and the
aggregate of the existing WGFs, future WGFs and all WGFs. The four aggregate regions were
the southwest region (SW), the south central region (SC), the southeast region (SE) and the
central eastern region (CE).
ORTECH compiled an extensive set of performance data for the forecasts generated by AWST
as well as the other forecast providers. Thus, general forecast performance statistics are not
presented in this report. However, it should be noted that, from AWST’s experience, the overall
magnitude of the forecast error, as measured by parameters such as the RMSE or MAE, for the
Alberta WGFs that participated in this project were, in general, higher than the magnitude of
error achieved for many other sites in North America such as in California or Texas. There are a
number of factors that combine to make wind power production forecasting for Alberta more
difficult than for other locations in North America. Some of the factors may have been specific
to this project and are related to the quality and quantity of meteorological and operational data
available from the wind generation facilities. However, in AWST’s opinion, the most significant
factor is that the forecasting of near-surface winds in the typical weather regimes that affect the
Alberta wind generation regions is more difficult than forecasting the near-surface winds in the
typical weather regimes in other locations. This increased difficulty in Alberta is related to the
fact that the wind generation regions are typically immediately downstream from a significant
mountain range and are subject to large variations in the low-level stability due to the advance
and retreat of shallow cold air masses and the diurnal temperature cycle. Furthermore, in some
cases, the sites are in complex terrain. A further discussion of the factors that affect the relative
performance of forecasts between sites in Alberta and those in other locations in North America
is presented in Section 3.1
This report is divided into 6 sections. The first section provides an overview of the structure of
the project from a forecast provider’s perspective. Section 2 provides a description of the tools
used by AWST to generate the forecasts. Section 3 provides a detailed description of AWST’s
perspective on the issues that impact forecast performance in Alberta as well as a discussion of
some of the problems encountered and lessons learned by AWST during the project. AWST’s
recommendations for an operational Alberta forecasting system are presented in Section 4.
AWST’s vision of the next steps that should be taken to improve wind power production
forecasting in Alberta is provided in Section 5. Finally section 6 provides a more general
recommendation of the direction that the industry should take to improve wind power production
forecasting.
5
2
Background Information
2.1 AWST’s Pilot Project Forecasting System
A customized configuration of AWST’s eWind forecast system was implemented to produce
hourly 1 to 48 hour ahead forecasts of the wind power production for a total of 12 existing and
future wind generation facilities (WGFs) in Alberta. A separate report compiled by AWST
earlier in the project provides a more detailed description of the individual components of the
forecast system. A schematic overview of the eWind configuration used for the AESO
forecasting application is presented in Figure 2-1. The light gray circles depict input data
sources. The dark gray ovals denote the intermediate and final output from the forecast system.
The rectangular boxes depict forecast models. The dark blue boxes represent physics-based
models. The lighter blue boxes represent statistical models. The final output of the system, the
wind power production forecast, is denoted by a black oval.
The top row of circles in Figure 2-1 represents the output data from external (to eWind) NWP
models that are run at government forecast centers. Four different types of external NWP data
are ingested into the eWind forecast process for the AESO application. This data, along with the
raw regional atmospheric data (light gray circle on the left side of Figure 2-1), are used to run
eWind’s own set of NWP models. These models employ higher horizontal and vertical
resolution than the government center models and in some cases also include physics-based
formulations that are more customized for low-level wind forecasting than those in the
government center models. An ensemble of 5 different types of physics-based simulations are
executed in the eWind configuration used in the AESO application. These models produce 3-D
forecasts of meteorological variables on a relatively high-resolution grid. The output from the
physics-based simulations, as it becomes available from each physics-based model cycle, goes
into a “potential predictor” database along with the raw regional atmospheric data and the
meteorological data from the WGFs.
The continuously updated composite NWP and observational database is used to train the
statistical models to produce forecasts of atmospheric variables at the meteorological tower sites.
An ensemble of these forecasts are produced by using two different statistical prediction
procedures (SMLR and ANN) and a number of different training sample sizes, contents and
stratification bins. The result of this process is an ensemble of forecasts for the atmospheric
variables at the meteorological tower sites. This ensemble is converted into a single
deterministic or probabilistic forecast for each variable and forecast hour by the ensemble
composite model. This ANN-based model is trained on historical forecast performance data and
essentially weights each forecast according to its recent performance or its performance in
previous occurrences of the anticipated weather regime.
The hourly forecasts of atmospheric variables at the meteorological tower sites are converted to a
power production forecast by “the plant output models”. These models are typically trained with
measured atmospheric variable and power production data although simulated atmospheric
variable data may be used for those variables that cannot be computed with the available
measured data. The output from the plant output models is a deterministic and probabilistic
power production forecast for each forecast hour.
6
Figure 2-1. A schematic depiction of the data flow and computational process for the
configuration of the eWind forecast system used for the AESO forecasting application.
7
2.2 Overview of the Forecast Evaluation Period
The forecast evaluation portion of the pilot project extended from May 2007 to April 2008.
Forecast providers were provided data for a period of approximately one year prior to the start of
forecast evaluation. In order to place the results into perspective it is useful to know how typical
the weather regimes were in Alberta during the forecast evaluation period and during the period
for which data was provided to train the statistical components of the forecast systems. AWST
has compiled a unique composite dataset that enables a graphical depiction of the regional
departure of the wind from long-term averages. Information for the months during the project
period and the prior year for which data was available was extracted from this dataset and the
regional wind speed anomalies for selected months are graphically displayed in Figures 2-2
through 2-7.
The 50 m wind speed anomaly data indicate that the wind speeds over the Alberta wind
generation area during the first quarter of the project were generally within +/- 5% of the 10-year
average. This can be seen in Figures 2-2 and 2-3. The one minor exception is a negative of
anomaly of about -5% to -10% in central Alberta (in the vicinity of the project’s CE region) in
July. Thus, the weather patterns for the first quarter of the project appear to be fairly consistent
with the typical patterns experienced in southern and central Alberta in May, June and July.
Furthermore the 50 m wind speeds during the same three months of the prior year were also
generally within +/- 5% of the 10-year average.
The conditions during the second quarter of the project exhibited more variability relative to the
long-term averages. The wind speeds were generally 5% to 10% above normal over the SW, SC
and SE regions during August and within +/-5% of normal in the CE region. The wind speeds
during the month of September 2007 were within +/- 5% of normal for all of the Alberta wind
generation regions. The wind patterns for October 2007 were somewhat more anomalous. The
wind speeds were 15% to 20% above the 10-year average across much of southern Alberta
(Figure 2-4) including the SC and SE regions. The positive anomalies were slightly lower on the
eastern slopes of the mountains with positive anomalies of about 10% to 15%,. The wind speeds
in the CE region were near to very slightly above normal with a positive anomaly of about 5% in
some parts of central Alberta.
The wind speed anomalies during the third quarter were highly variable. The October pattern of
above normal wind speeds over many parts of Alberta continued in November. However, during
November the positive anomalies were larger in central and northern Alberta than in southern
Alberta (Figure 2-5). The wind speeds in the CE region were 10% to 15% above the 10-year
average while the mean speed in the SW, SC and SE regions were approximately 5 to 10% above
normal with a tendency for anomalies in the upper end of that range to be present in southeastern
Alberta. The mean wind speeds during December were generally within +/- 5% of the 10-year
average for all of the project’s regions. During January (Figure 2-6) the wind speeds were 5%
to 10% above normal in the SW area and 10% to 15% above normal in the SC and SE regions.
The mean wind speeds in the CE region were about 5% above normal with most of the areas
over central and northern Alberta within +/- 5% of normal. However, two of these three months
had a very different anomaly pattern in the prior year. The mean wind speeds for November
2006 were only 5% to 10% higher than the 10-year average over the project regions. In contrast,
there were very large positive wind speed anomalies in both December 2006 and January 2007 in
all the project’s regions. In December 2006 the wind speeds averaged 15% to 20% above the
10-year average across the SW, SC and SE regions and about 10% to 15% above normal in the
8
CE region. The positive anomalies were even larger in January 2007 (Figure 2-7) with values of
20% to 25% above normal in the SW and SC region and 25% to 30% above the 10-year mean in
the SE region. The positive anomalies were also in the 25% to 30% range in the CE region. It is
clear that the general weather regimes affecting Alberta in the December 2006-January 2007
period were quite anomalous and quite different from that in the two project months of
December 2007 and January 2008. The anomalously high wind speeds suggest in particular a
lack of shallow cold air events, which tend to be associated with light winds. The potential issue
is that statistical relationships developed with data from the prior year may not have been very
representative of the December 2007-January 2008 period. This underscores the point that one
year of training data may not be adequate in some cases.
The wind speed anomaly data for the project’s fourth quarter (February, March and April 2008)
have not yet been compiled by AWST as of the date of this report. However, the mean wind
speeds were generally within +/- 5% of normal for all regions in February and April of 2007 with
a tendency for very weak negative anomalies in February and very weak positive anomalies in
April. However, March 2007, two months prior to the start of the project’s evaluation period,
was a fairly anomalous month with positive wind speed anomalies of 25% to 30% in the SW
region and 20% to 25% in the SC and SE regions. The positive anomalies in the CE region were
in the 10% to 15% range for March 2007.
9
Figure 2-2. Estimated regional-scale 50 m AGL percentage wind speed anomaly (% of 10-year
May average) for the month of May 2007.
July average) for the month of July 2007.
10
October average) for the month of October 2007.
November average) for the month of November 2007.
11
January average) for the month of January 2008.
January average) for the month of January 2007.
12
3 Lessons Learned and Findings
3.1 Comparative Forecast Performance
One of the most basic findings from AWST’s year of forecasting in the Alberta Pilot Project was
that the performance of the wind power production forecasts for sites in Alberta during the Pilot
Project was, in general, not as good as that achieved for many other sites in North America. A
representative example of the performance difference is shown in Figure 3-1. This chart depicts
average annual Mean Absolute Error (MAE) for the 4-hr and 36-hr ahead forecasts for the 7
existing facilities that participated in the Pilot Project (red bars) and for a collection of other sites
in North America for which AWST produces operational forecasts (black bars). The chart
indicates that the average MAE for the WGFs in Alberta was about 20% to 30% higher than the
average of the MAEs for the other sites in North America for both the 4-hr and 36-hr ahead
forecasts. The difference is somewhat larger for the 4-hr ahead forecasts. This comparison
supports the point often made by forecast providers that forecast performance for other regions
are frequently not a good predictor of performance in a new region.
There are several reasons for the less favorable performance for the Alberta WGFs during the
Pilot Project. One factor is the lower level of forecaster experience in Alberta. AWST has been
forecasting for many of the sites included in the North American average for several years and as
discussed later in this report knowledge of the wind regimes and the regime-specific forecast
model error patterns can often result in better forecast performance. An example of the potential
forecast performance impact of the increased knowledge of Alberta wind-regimes gained by
AWST during the project is presented in Section 3.7.1.
A second factor is that the wind regimes in Alberta present, in general, a more challenging
forecast problem than the regimes in many other parts of North America. The higher degree of
difficulty is attributable to the unique juxtaposition of (1) a north-south oriented range of high
elevation mountains in western Alberta, (2) a source region for very cold Arctic and Polar air
masses just to the north of the region and mild maritime air to the west of the mountains and (3)
close proximity to the climatological position of the Polar jet stream during most of the year.
These factors not only lead to a relatively high degree of wind variability on most time scales but
also create several types of difficult-to-forecast wind regimes. A more detailed discussion of
some of the key wind regimes and the challenges they present to wind power production
forecasting is provided in Section 3.3.
A third factor that contributed to the lower performance of the forecasts during the Pilot Project
was the quality of data from the WGFs. One of the most important issues was the lack of
reliable historical and projected turbine availability data. The lack of such data on a historical
basis reduces the quality of the statistical relationships that are used to convert forecasts of the
meteorological variables to predictions of the power production. It also degrades the quality of
the forecast evaluation since the power production used to evaluate the forecast can’t be adjusted
to account for the impact of unavailable turbines. Finally, the lack of projected availability data
adds more uncertainty to the forecast since not only is there uncertainty in the meteorological
forecast but there is additional uncertainty in the number of turbines that will be available to
produce power. In addition to the lack of high quality turbine availability data, there were a
number of issues with the quality and representativeness of the meteorological data from the
WGFs in Alberta. These are discussed in Section 3.4.
13
Figure 3-1. The average annual Mean Absolute Errors (MAEs) of AWST’s 4-hr and 36-hr
ahead forecasts of power production for the 7 existing sites in Alberta and other AWST forecast
sites in north America. It should be emphasized that the MAEs are the average of the MAEs for
individual wind generation facilities and not for an aggregate of those facilities.
3.2 The Impact of Forecast System Components on Forecast Performance
A commonly asked question addressed to forecast providers is what is the relative impact of
different input datasets and forecast system components on forecast performance. This question
is typically motivated by a desire to understand how resources should be invested in order to
obtain the most cost-effective wind power production forecasting solution. In general, this is a
very difficult question to answer because impact of the input datasets and forecast system
components is intimately intertwined and dependent on a variety of factors such as the lookahead time period, the season, and the weather regime.
In order to provide some insight into this issue, AWST analyzed the relative performance of
forecasts generated from different combinations of datasets and forecast system components over
the entire project year for the 7 existing facilities. Some of the results from this analysis are
depicted on Figure 3-2. This chart illustrates the MAE by look-ahead period for four different
forecasts generated from different sets of components from AWST’s forecast system.
The black line with the square markers illustrates the MAE of a forecast produced by obtaining a
predicted wind speed by direct interpolation from the three-dimensional grid of one of AWST’s
NWP models to the turbine hub height of each of the 7 WGFs and then using that wind speed as
input into a manufacturer’s power curve for each WGF. This approach requires absolutely no
power production or meteorological data from the WGFs. Only the location of the WGF and the
turbine characteristics (i.e. hub height, turbine type etc.) are needed in this approach. The NWP
model used in this procedure was the model that exhibited the best performance (i.e. lowest
14
MAE) over the entire project year. This line provides a forecast performance reference point for
what can be achieved with a standard NWP model and no data from the WGF.
The blue line with the triangular markers represents a forecast generated from the same
procedure as the “black line” forecast except that a statistical plant output model is used in place
of the manufacturer’s power curve. This requires historical meteorological and power
production data from the WGF but it is assumed that the data is not available in a real-time mode
and that the historical data is not (although it could be) used to correct the systematic errors in
the NWP forecasts. The difference in MAE between the black and blue lines provides an
indication of the value of employing a statistical plant output using data from the WGF in place
of the manufacturer’s power curve. For the entire 12-month project period there was about a
10% reduction in MAE for most look-ahead times. However, the magnitude of this benefit will
depend upon the quality of the data from the WGF. For example, the lack of reliable availability
data from the existing sites during the Alberta Pilot Project most likely limited the amount of
improvement in forecast performance that could be obtained through the use of a statistical plant
output model since this added additional uncertainty to any relationship of meteorological
variables to power production that would be derived from the data. The meteorological data
representativeness issues discussed in Section 3.4 also play a role in determining the impact of
using a statistical plant output model.
The green line denotes the MAE of a forecast produced by making the meteorological and power
production data from the WGF available to the forecast process in near-real-time and using this
data in the statistical component of the forecast system to correct the systematic errors in the
NWP model forecast data as well as to provide information about the recent trends in power
production and meteorological variables. As in the case of the black and blue lines the NWP
model used in this forecast procedure was the one that had the best overall performance (i.e.
lowest MAE) for the entire project year. A comparison of the green and blue lines indicates that
the near real-time availability of WGF data has a large impact on the forecast performance for
the first few look-ahead hours. There is about a 40% reduction in the MAE for the 1-hour ahead
forecast but it decreases to about a 5% to 10% MAE reduction for a 6-hour ahead forecast.
These reductions are mostly attributable to the use of knowledge of the current production and
meteorological conditions and their recent trends to adjust the NWP-based forecast.
The value of this information rapidly decreases in time and hence the reduction in the forecast
MAE associated with this information also rapidly decreases as the look-ahead period increases.
After approximately 6 hours, most of the benefit of the WGF data is associated with the
“correction” of systematic errors in the NWP forecast. A comparison of the blue and green lines
indicates that the overall impact on the MAE over the entire project year is about 3% to 8%.
However, for specific sites during particular time periods the impact was much larger. The
overall magnitude of this correction also depends upon the skill of the statistical components of
the forecast system in diagnosing complex patterns of systematic errors. This will depend upon
the types of statistical methods that are employed as well as the strategy and parameters used to
identify the error patterns. For example, the patterns of systematic error in NWP forecasts often
have a significant dependence on weather regime or time of day. These patterns may be
obscured if one only considers patterns that appear over all hours of the day or all weather
regimes. Hence, the ability to improve the forecast by adjusting the NWP model output data is
strongly linked to one’s knowledge of the significant weather regimes in the region and how the
various NWP model forecasts behave in each of those regimes. There is reason to believe that
15
the 3% to 8% impact can be significantly increased in Alberta by employing knowledge of how
NWP forecasts behave in each of Alberta’s significant weather regimes. An example of the
potential impact of this approach is illustrated for the shallow cold air regime in Section 3.7.1.
The fourth (red) line on the chart represents a forecast that optimally combines the forecasts from
several NWP model-statistical model combinations by weighting them according to their recent
performance. The difference in the MAE between the green and red lines represents the general
benefit of using a composite of an ensemble of forecasts over using the best single statistically
adjusted NWP forecast as well as the skill in assigning weights to different members of the
ensemble. The reduction in the MAE associated with the use of an optimized ensemble scheme
could potentially be increased if additional forecasts are added to the ensemble. This would be
especially true if the additional forecasts had about the same level of skill (or better) as the
existing ensemble members and the hourly errors of the additional forecasts had a relatively low
correlation with the errors associated with the existing ensemble members. However, a
significant expansion of the number of members in an ensemble can have a large computational
cost, especially if the new members are each produced by a separate high resolution NWP
simulation. In that case one has to consider the cost/benefit ratio of expanding the ensemble.
The impact of the optimized ensemble can potentially also be increased if one can improve the
skill of selecting (or appropriately weighting) the member(s) of the ensemble that are likely to
perform best under a specific set of circumstances (e.g. weather regime, season, time of day etc.).
However, this will typically involve a substantial amount of region-specific research to identify
the characteristic variations in the error patterns for each member of the ensemble.
Figure 3-2. The average of the annual Mean Absolute Errors (MAEs) for four AWST forecasts
generated from different combinations of input data and forecast models for the 7 existing sites
in Alberta. It should be emphasized that these MAEs are the average of the MAEs for individual
WGFs and not an aggregate of those facilities.
16
3.3 Thoughts on Regional Grouping of Sites
The twelve sites in the Pilot Project were grouped into four regions of three sites each. The basis
for the grouping was primarily geographic. However, the question arose several times during the
project as to whether there was a more appropriate grouping of the sites. One approach to
address this question is to determine if there are groups of sites that tend to have similar forecast
errors at the same time (i.e. the forecast errors tend to be highly correlated among sites within the
group but less so with sites outside the group).
AWST used this concept to determine if there was a more appropriate grouping than that used in
this project. The correlation in forecast errors between all pairs of the twelve facilities was
computed. Then the average forecast error correlation for intra-group pairs (within the group)
and inter-group pairs (between groups) was computed. The idea is that the best grouping would
result in the highest difference in correlation between the intra-group and inter-group pairs. That
is, the facilities within a group have much higher forecast error correlations with each other than
they do with facilities outside the group. The results for short-term forecasts are listed in Table
3-1. The data in the table indicates that three of the four groups have a fairly similar level of
correlation difference between the intra-group and inter-group correlations. However, the SE
region is somewhat lower. This suggests that the SE group is less of a cohesive group than the
other three. The fact that this group was composed of both existing and future facilities may
have been a factor. However, AWST believes that the more significant factor is that the sites in
this group are more geographically diverse and do not experience the impact of atmospheric
features at the same time or in the same way. The easternmost site in the SE group behaves quite
differently than the two sites that are further to the west.
Table 3-1.
The difference between the average intra-group and inter-group short-term forecast error
correlations for each of the four geographic groups used in this project.
Group
SW
SC
SE
CE
Average
Correlation Difference
0.2562
0.2379
0.1001
0.2604
0.2137
The analysis was carried further by examining all groups of three with the objective of finding a
grouping that had a higher average forecast error correlation difference than the grouping used in
the project. One grouping was found that had a slightly higher average correlation difference but
the average difference was only slightly higher (0.2188) than the grouping that was used. The
only difference in that grouping was that one site was that one of the sites in the SW group was
moved to the SC region and a site originally in the SC region was placed in the SW region.
Thus, from a short-term forecast error correlation perspective the geographic groupings used in
this project seem quite reasonable and there is not an obvious better choice. However, the
correlation analysis does show that the SE group has a less coherent forecast error structure than
the other three groups.
17
3.4 Impact of Weather Regimes and Events on Forecast Error.
Alberta’s location in a region of climatologically strong westerly flow in the middle atmosphere
combined with the presence of a significant north to south oriented mountain range on it’s
western border results in relatively consistent moderate to strong westerly winds at the surface.
The interaction of the strong mountaintop-level westerly flow with the mountains results in the
enhancement of the flow to the lee of the mountains and the downward mixing of strong winds
to the surface.
The strength of the surface winds are influenced by the strength of the cross-mountain
component of the winds at mountaintop level, the stability of the lower atmosphere which can
enhance or hinder mixing, and local effects due mainly to smaller scale terrain features. For
example, channeling of the flow by Crowsnest pass and the ridge of higher terrain that extends
into the plains south of the southern WGFs results in a local region of stronger winds that
reaches eastward from the mouth of the pass out into the plains.
3.4.1 Change in the Mountaintop Cross-Mountain Wind Component
The interaction of Alberta’s climatologically strong westerly flow with the mountains tends to
enhance the near-surface westerly flow for several reasons. First, the mere presence of the
mountains constricts the eastward moving air and increases it’s velocity, much as a riverbed can
enhance the current wherever a river’s channel is constricted. The river analogy is also helpful
for understanding how the enhanced westerly flow gets mixed to the surface downstream of the
mountains. The mountains act just like a submerged rock in the riverbed. Waves develop
downwind of the mountains just as they do downstream of a rock. The trough of the first wave
tends to constrict the river even more, resulting in an acceleration in the flow. In the atmosphere,
the dip shows up as a trough of lower surface air pressure in the lee (eastern side) of the
mountains. The winds tend to be strongest between the mountains and the axis of the low
pressure trough as air accelerates into the trough
If a rock is large enough and the current strong enough, the waves tend to break. The turbulence
generated by the breaking of the waves mixes higher velocity water from the surface towards the
river bottom where the velocity is lower. This is also true in the atmosphere where a third factor,
the vertical stability plays a role too. An unstable atmosphere (relatively warm at the surface and
cold aloft) with strong cross-mountain flow tends to result in breaking waves and enhanced
vertical mixing of strong westerly flow to the surface.
Finally, the lee pressure trough (i.e. axis of low pressure) can be enhanced when waves of lower
pressure pass over the mountains (see Figure 3-3). These waves are typically associated with
stronger mountaintop-level flow as can be seen in the constriction of the streamlines that peaks at
the trough axis. Because of this, they tend to be focal points for stronger surface winds. Winds
tend to be strongest from about the time the wave axis reaches central or eastern British
Columbia until it crosses Alberta’s eastern border.
Because of this mechanism, variations in the strength of the cross-mountain flow and the passage
of low pressure waves at the mountaintop level are quite important in determining surface wind
speeds to the lee of the mountains. Specific events that are most influenced by changes in the
cross-mountain wind (CMW) component at mountaintop will be referred to as CMW events.
18
The case of November 10-11, 2007 is typical of CMW events. During this time period two
middle-atmospheric (at about 5.5 km or a pressure level of 500 mb) waves passed over Alberta,
as shown in Figure 3-3. Both waves extended downward in the atmosphere to mountaintop level
at about 3 km (not shown). The blue lines in the figure represent lines of constant height of the
500 mb pressure surface in tens of meters. They can be viewed as approximate streamlines of
the wind. We can see that the wave axis, depicted by a thick blue line, tends to coincide with a
packing of the streamlines. This is equivalent to the constricting of a river’s channel and so tends
to increase the flow speed. In this case, the wind remains approximately perpendicular to the
mountains, but in some cases, the approach of a wave can turn the flow from northwesterly
(nearly parallel to the mountains) to southwesterly (nearly perpendicular). Once the wave
passes, the winds can turn back to northwesterly. The net effect is to enhance the effect of the
wave by reducing the cross mountain component of the flow before and after the wave passes.
A comparison of Figures 3-3 and 3-4 provides an indication of the relationship between the wind
variations and the passage of the middle atmospheric wave. Figure 3-3 A-D corresponds with
the observed power trace at the 0, 12, 24 and 36 hour forecast times in Figure 3-4B. In Figure 33A (0 hour forecast in Figure 3-4B), the first wave is just approaching the British Columbia
coast, so winds would be expected to be light. Figure 3-4 shows that power production is indeed
low at this time. Figure 3-3B (12 hour forecast in Figure 3-4B) shows the wave nearly at the
Alberta-British Columbia border. We would expect the winds to have increased several hours
earlier as indeed they did around the 6-hour forecast time. Figure 3-3C (24-hour forecast in
Figure 3-4B) shows the first wave passing into Saskatchewan while the second wave is still in
western British Columbia. We would expect wind speeds to be dropping, but not to remain low
for long as the second wave approaches. Once again, Figure 3-4B shows this indeed happened.
Finally, Figure 3-3D (36 hour forecast in Figure 3-4B) shows the second wave in eastern
Alberta. We would expect winds to still be high, but to be dropping shortly after this time and
they do.
The description above implies that the forecast should be a relatively simple one and that errors
should be low. In fact, the upward ramp was reasonably well forecast, especially 6 hours prior.
The downward ramp was not very well forecast. Why is this the case? First of all, throughout
the event, winds at mountaintop level remained relatively perpendicular to the mountains and
never dropped below about 8 m/s. Observed wind speeds were only available at 2 of the 7
existing facilities. At these two sites, speeds dropped as low as 4 m/s during the low power
times. Other factors such as the presence of cold air at the surface could have prevented vertical
turbulent mixing. This is especially likely during the night hours, which cover much of the 24
hour period at this time of year. Since wind speeds at mountain top level were in the 8-12 m/s
range during the lighter wind times, small errors in wind speed can produce large power errors
due to the steepness of the power curve for these speeds. Another possible source of error is the
tuning of the forecast system to minimize MAE or RMSE. An excellent strategy to minimize
error in this case is to hedge towards a lower ramp amplitude and longer duration. Several raw
NWP model forecasts were considerably better than the delivered forecast. However, the
relatively strong ramps in the raw NWP model forecast were smoothed by the statistical
prediction algorithms designed to minimize RMSE.
19
A
B
C
D
Figure 3-3. A depiction of the middle atmospheric wind pattern at (A) 1200 UTC (0500 MST) 10
November 2007, (B) 0000 UTC 11 November 2007 (1700 MST 10 November), (C) 1200 UTC
(0500 MST) 11 November 2007, and (D) 0000 UTC 12 November 2007 (1700 MST 11
November). The blue lines represent the height (dm) of the 500 mb pressure surface above sea
level and can be viewed as approximate streamlines of the wind at this level. The dashed lines
denote the vorticity (i.e. tendency for the air to rotate). The bold blue line indicates the position
of two troughs (i.e. a perturbations in the flow) of low height, high velocity and high vorticity,
propagate from west to east through the Alberta during the period. The arrows in the vicinity of
the troughs depict the direction of the wind at this level.
20
A
B
Figure 3-4. AWST’s aggregate power forecast (in blue) for the existing facilities delivered at (A)
00 UTC, 10 November 2007 (17 MST 09 November), and (B) 12 UTC (05 MST), 10 November
2007. Observed hourly average power in MW is shown in black. Other lines indicate rawNWP
model output from three different NWP models.
3.4.2 Shallow Cold Air Masses
Shallow cold air masses (SCA) often approach southern Alberta from the northeast or north.
Behind the frontal boundary (i.e. the leading edge of the cold air mass), strong westerly flow
tends to be replaced with light northeast, east or southeast flow. The boundary between
extremely cold air to the north and east and warmer air to the west and south tends to extend
southward along the eastern foothills of the mountains. At some point, the boundary turns
eastward to southeastward into the plains. This frequently occurs between a point just to the
north of the SW power production region and northern to central Montana (see Figure 3-6).
Because of this, the existing facilities are often either in the cold air with the frontal boundary not
far to the south or west, bisected by the boundary or on the warm side of the boundary with the
boundary not far to the north and east. Since the cold air is shallowest just north and east of the
boundary, this often puts the region in very shallow cold air, so shallow that turbines at hilltop
sites can sometimes reach up into or even through the temperature inversion that separates the
warm air and strong westerly flow above from cold air and light northerly to easterly flow in the
lowest levels. Furthermore, when strong west to southwest flow is present at mountaintop level
the cold air can be quickly scoured out by the westerly flow, bringing strong winds to the
surface. For this reason, strong upward or downward ramps in power occur quite frequently
during shallow cold air events
The November 26-29, 2007 period is a good example of a SCA event. During this period, the
boundary between cold and warm air moved back and forth over the region several times,
resulting in several ramps of power both upward and downward. We will focus on the period of
21
the ramp up and then down in power depicted in Figure 3-5B. A large upward ramp in power
production from near zero to 290 MW occurred from 04 to 09 MST on November 28, 2007. A
downward ramp in power nearly as large as the upward ramp immediately followed from 09 to
13 MST. The power then ramped up slightly to about 130 MW before once again ramping down
to zero at midnight.
A
B
Figure 3-5. AWST’s delivered power forecast (in blue) for the existing facilities delivered at (A)
12 UTC (05 MST), 27 November 2007, and (B) 06 UTC, 28 November 2007 (23 MST 27
November). Observed hourly average power (MW) shown in black. Other lines depict raw
physics-based model forecasts from three different models.
Now, let’s look at the meteorological events that were responsible for the power ramps.
Figure 3-6 shows the surface weather patterns (A) before the ramp up, (B) near the peak of the
ramp up, (C) during the low-power period before the second, smaller ramp up, and (D) after the
event when power production returned to near zero. The only surface station depicted in the
plots in southwest Alberta is Lethbridge, which is located about 20 km northeast of the
easternmost wind generation facility in the SC region. At the beginning of the period (Figure 36A), we can see that a cold air mass has propagated southward along the mountains into northern
Montana. Winds in the cold air mass are mostly light and from the west or northwest. The cold
air has made it no farther west than the foothills as shown by the stationary front along the
eastern flanks of the mountains. This is typical of the many shallow cold air masses that intrude
into southern Alberta from the north and east. Nine hours later (Figure 3-6B), the cold air has
retreated from the mountains to just east of Lethbridge. The westerly wind at Lethbridge has
increased to about 5 m/s. Six hours later (Figure 3-6C), the cold air has returned and the wind
has become light easterly. By the end of the event (Figure 3-6D), the cold air has pushed up into
the mountains and the flow is light and from the west. The fact that the cold air has pushed up
into the mountains is evidence that the cold air mass is no longer shallow.
22
A
B
C
D
Figure 3-6. A depiction of the surface wind patterns at (A) 23:00 MST, (B) 08:00 MST, (C)
14:00 MST, and (D) 23:00 MST November 27 and 28, 2008. The red-brown lines represent
constant sea level pressure and approximate streamlines of the wind with actual winds turning
about 30° towards lower pressure. Observing stations are depicted as light blue circles. The
direction the wind is blowing from is represented by light blue lines attached to the station
circles. Temperature (°F) is shown in red, while dew point temperature is in green. Wind speed
is shown by the number of barbs on the direction line. Each long barb represents 10 knots. A
short barb represents 5. Fronts are represented by thick lines, red for warm fronts, blue for cold
fronts and alternating red and blue for stationary fronts. The southwesternmost station shown in
Alberta is Lethbridge while the southeasternmost station is Medicine Hat.
23
A
B
Figure 3-7. AWST’s power production forecast
(in blue) delivered at 06 UTC, 28 November
2007 (23 MST 27 November) for (A) the
Cowley Ridge and Castle River, (B)
Summerview and Soderglen, and (C) McBride
Lake, Magrath and Chin Chute. Observed
hourly average power in MW is shown in
black. Other lines indicate raw NWP model
forecasts from three different NWP (physicsbased) models.
C
24
Figure 3-7 shows the 6-hour ahead forecast separated into smaller generation facility groups
from west to east. The AWST delivered forecast is similar for all groups: a ramp up to near full
power, an extended period of about 12 hours at near full power and a relatively quick ramp down
to near zero. For the westernmost group (Figures 3-7A), the forecast is fairly good. However,
the group of two facilities further to the east (Figure 3-7B) shows a double maximum with a
ramp down to zero between the two peaks. The second peak is at about 50% of capacity. The
delivered forecast maintains the power at a much higher level throughout the period. The
easternmost group of facilities (Figure 3-7C) shows a quick ramp up to near full power at the
beginning of the period followed by an immediate ramp down to zero with no secondary peak.
The raw physics-based model and statistical method forecasts for the westernmost group were
quite good. However, the models maintained the strength and duration of the event for the
eastern two groups. In reality, the second part of the event was cut off by a return to light flow at
these site. For the middle group a brief return to somewhat higher power levels occurred during
this period while the easternmost group experienced no secondary peak.
The evidence suggests that the complex nature of the event is related to a shifting boundary
between a shallow cold air mass with light easterly to northerly flow and a warmer less stable air
mass near the mountains in which momentum effectively mixed to the surface. At first the cold
air retreated well to the east of all seven existing facilities. The cold air then advanced to just
east of the westernmost group, retreated briefly to bring the middle group into the warm air, and
finally deepened and advanced well to the west into the mountains so that all groups were in the
cold air. The delivered forecasts suggest that the models maintained the boundary between
stronger westerly flow and the weaker flow in the cold air too far to the east throughout the
event.
Now let’s look at the event from the perspective of the mountaintop level (about 3 km MSL)
cross-mountain component of the wind. The mountains in southern Alberta are oriented
approximately along a 150°-330° (SE-NW) line, so the cross-mountain flow component is the
component of the 3 km wind from 240° (WSW). The approximate cross-mountain flow
component derived from upper air grid point analysis data and short-term model forecasts spaced
at 3-hour intervals is shown in Figure 3-8 (black line). The observed ~50 m wind speed for the
SW, SC and EF regions are shown as well. The SW region wind speed parallels the 3 km crossmountain flow very closely. In fact, it exceeds cross-mountain flow component by several
meters per second. Since the 3 km flow direction over the period varied from about 280° to
325°, the actual wind speeds aloft were always equal to or slightly greater than the 50 m wind
speeds. Apparently, winds were mixed down to the surface quite effectively in this region close
to the mountains. Local wind speeds may also have been enhanced somewhat by mountain
waves that tend to form to the lee of the mountains when there is a substantial component of the
flow across the mountains
In the SC region, the pattern is somewhat different. Wind speeds there parallel the SW region
wind speeds early in the period, but then decrease by several m/s as the shallow cold air returns
and mountaintop level winds no longer mix to the surface.
25
Figure 3-8. A comparison of the mountain top level cross-mountain flow component and surface
wind speeds for the November 28, 2007 case.
This case illustrates that the interplay between strong westerly flow at mountaintop level and
shallow cold air over the plains are an important factor in determining near-surface wind speeds
in southwest Alberta. It also illustrates the tendency of physics-based models to overforecast
wind speeds and power during SCA events and for the non-regime-based statistical methods to
overforecast wind speed and power even more than the physics-based models. The case of
December 2-5, 2007 described in the appendix is an extreme example of this.
3.4.3 Strong Cold Surges
Another type of event that has a significant impact on forecast performance is a strong cold surge
that propagates southward along the eastern edge of the mountains. These cold surges are
associated with usually brief but occasionally extended periods of moderate to strong northerly
winds followed by a period of much lighter winds until westerly flow once again returns. Often,
the northerly flow is relatively light closer to the mountains, especially in the southwest region
and especially if the cold air is not deep enough to penetrate west of the mountains. Cold surges
can be associated with significant upward or downward ramps in power. During the project’s
third quarter, the only upward ramp associated with a cold surge was with the strong cold surge
of January 27 when westerly flow decreased ahead of the cold front prior to a rapid increase in
winds from the north behind the front. Several significant downward ramps occurred with weak
to moderate cold surges during the third quarter. In each case, relatively strong westerly flow
ahead of the surge was replaced with strong northerly flow behind the surge. Wind speeds then
decreased rapidly between 1 and 9 hours after the initial surge.
26
Cold surges and shallow cold air events exist as a continuum. On one end are SCA events where
the cold air is typically propagating southward very slowly or remaining relatively stationary.
Winds in the cold air are light from the north and east even right near the boundary between cold
and warm air. The slope of the cold air dome is relatively shallow so that the cold air remains
shallow well away from the boundary. On the other end are strong cold surges where the cold
air is rapidly propagating southward. Strong northerly winds extend well back from the frontal
boundary resulting in an extended period of strong winds. The slope of the cold dome is steep so
that the cold air becomes quite deep relatively close to the boundary. At some point from 50 to
several hundred km from the frontal boundary, the winds do become light from the north or east
as they are for SCA events.
Cold surges tend to be less frequent than SCA events. In the third quarter of the project, 7 cold
surges were observed vs. 15 SCA events. Cold surges tended to be of much shorter duration
than SCA events, totaling only 58 hours or 2.6% of all hours as opposed to 496 hours or 22.5%
for SCA events. A cold surge is considered to encompass the period of strong winds after the
initial surge. In general, wind speeds tend to be overforecasted in SCA events but
underforecasted in cold surge events.
The case of January 27-28, 2008 is a somewhat extreme cold surge case. The duration of this
event was 26 hours as opposed to 4 to 9 hours for the other events. It is of particular interest
because of the strong upward ramp that occurred at its onset. Figure 3-9 depicts the surface
weather conditions (A) before the arrival of the cold surge, (B) several hours after it’s arrival
during the period of strong northerly winds, and C) after the strong winds finally weaken over 24
hours later. Through the sequence, winds at Lethbridge, the southwesternmost station in Alberta
went from moderate westerly to strong northerly to light northerly. The strong temperature
contrast across the cold front and it’s relatively rapid southward propagation speed, two common
characteristics of cold surges are also visible on these maps.
27
A
B
C
Figure 3-9. A depiction of the surface wind patterns at (A) 08:00 MST, (B) 14:00 MST, January
27, 2008 and (C) 17:00 MST, January 28. The red-brown lines represent constant sea level
pressure and approximate streamlines of the wind with actual winds turning about 30° towards
lower pressure. Observing stations are depicted as light blue circles. The direction the wind is
blowing from is represented by light blue lines attached to the station circles. Temperature (°F)
is shown in red, while dew point temperature is in green. Wind speed is shown by the number of
barbs on the direction line. Each long barb represents 10 knots. A short barb represents 5.
Fronts are represented by thick lines, red for warm fronts, blue for cold fronts and alternating
red and blue for stationary fronts. The southwesternmost station shown in Alberta is Lethbridge
while the southeasternmost station is Medicine Hat.
28
3.4.4 Nocturnal Mountain Outflow
Nocturnal mountain outflow can be important in times of light flow and clear skies, especially in
the winter when significant snow cover is present in the mountains. Because fresh snow
effectively insulates the ground and prevents heat flux from the ground into the air, nighttime
temperatures drop more quickly where significant snow cover is present. In Alberta, there is
typically significant snow cover present in the mountains while snow cover in the plains is not as
common. Therefore, night-time temperatures are often lower in the mountains than on the
plains. Since cold air is relatively dense, it tends to flow downhill out of the mountains. The
flow of cold air tends to be concentrated in valleys that penetrate a significant distance into the
mountains. One such valley, the valley of the Crowsnest River, opens up in to the plains
immediately west of the SW region. Because of this, strong west winds can develop in the SW
and adjacent SC regions under nocturnal mountain outflow conditions from a few hours after
sunset until a few hours after sunrise. When the mountaintop level flow is stronger, nocturnal
outflow can sometimes influence the diurnal cycle of wind speeds so that the flow is stronger by
day and weaker by night.
The night of February 21-22, 2008 is a good case of nocturnal outflow. Light mountaintop flow
in the region resulted in light winds during the daytime hours of both February 21 and 22.
However, Figure 3-10 shows that the power began ramping up at about 23 MST on February 21
(0600 UTC February 22). The power production peaked at about 210 MW 6 hours later and then
ramped down to near zero by 13 MST (2000 UTC) on February 22. Power began ramping up in
the late evening hours, reached a peak around sunrise and dropped to near zero by midday. The
SW region and the two SC region facilities that are closest to the mountains contributed nearly
100% of the power production during this period. This is typical of a nocturnal outflow case.
Figure 3-10 shows that the 6-hour ahead forecast was quite good. It ramped up the power a few
hours early had the peak about 2 hours late and 30 MW too low, but overall it reproduced the
ramp up and ramp down quite well. The event was well forecast up to 30 hours in advance with
earlier forecasts having slightly smaller phase and amplitude errors. One particular physicsbased model shown in pink consistently forecasted the event very well. This event occurred in
the 4th quarter of the project. A detailed analysis of ramp events was performed for the 3rd
quarter (see section 3.4), but not the 4th quarter. Still, it is interesting to note that the maximum
one-hour upward ramp of about 55 MW associated with this event would not have placed it in
the 20 largest upward ramps by 3rd quarter standards. The maximum one-hour downward ramp
of 60 MW would have just brought it into the top 20 downward ramps of the 3rd quarter. There
were numerous other similar events in late February and early March that were at least partly
influenced by nocturnal outflow. Several of these events had a significant one hour upward or
downward ramp by 3rd quarter standards.
29
Figure 3-10. Predicted (blue) and observed (black) power for the aggregate of existing facilities
on February 21-22, 2007.
Figure 3-11. Observed wind speed and direction at Pincher Creek in the Southwest region from
14:00 MST (2100 UTC) on February 21, 2008 through 13:00 (2000 UTC) on February 22. Peak
wind speed at Pincher Creek corresponds with peak power for the existing facilities region.
3.5 Impact of Wind Farm Data
The wind farm anemometer data (wind speed and direction) provide a key piece of information
to define the relationship between the winds at a particular wind farm and the concurrent power
production. The value of the information provided by the anemometers to the power production
forecast process is dependent on the how well the anemometer locations represent the winds at
the turbine locations. A poorly located anemometer site will not provide very good information
even if a high quality anemometer that is calibrated and well-maintained is employed. We have
done a preliminary analysis of the anemometer siting for power production forecasting purposes
for the 7 existing wind farms in Alberta. This analysis was based on the amount of deviation
from a speed-based power curve evaluated over the 6-month period from March through
30
September of 2007. This was done constructing a speed-based power curve that consisted of a
piecewise polynomial representation of the data gathered during the 6-month period. The
deviations from this speed-based curve were then computed and the standard deviation of those
deviations were calculated. The standard deviation of the residuals is a measure of the dispersion
of the measured pairs of hourly average wind speed and power production around the speedbased power curve.
Table 3-2 lists the standard deviation of the power curve residuals for each of the 7 existing
farms. The farms are listed in order of increasing standard deviation. The standard deviations
range from a low of 8.3% of capacity for farm #1 to 16.9% of capacity for farm #7. A visual
representation of the data corresponding to the farm with the lowest standard deviation (farm #1)
and the farm (farm #7) with the highest standard deviation is shown in Figure 3-12. It is
apparent that there is a large difference in the amount of scatter in the wind speed to power
production relationship among the existing wind farms.
A further analysis indicates that a modest component of these residuals can be explained by
wake effects or terrain blocking associated with certain wind directions due to the layout of
turbines and air density effects. These can be effectively modeled by including additional
parameters in the farm scale power production model. The use of the enhanced model will
reduce the amount of deviation from a statistical power production model for a measured wind
speed. However, a large amount of this scatter is due to differences in the free stream wind
speed between the anemometer site and the turbine locations. This can be addressed by
gathering anemometer data from one or more sites that are more representative of the turbine
locations. One possibility is to obtain the data from nacelle-mounted anemometers in addition to
the wind speed and direction data from anemometers mounted on meteorological towers.
31
Table 3-2. A listing of the standard deviation of the hourly residuals from a speed-based farmscale power curve for each existing wind farm for the 6-month period of March to Sept 2007.
Site
1
2
3
4
5
6
7
Std Dev
8.3%
9.8%
11.4%
14.1%
15.2%
15.4%
16.9%
Figure 3-12. A scatter plot of the hourly average wind speed vs. the concurrent hourly average
power production for existing wind farm #1 (left) and existing farm #7 (right) for the period
May 1, 2007 through October 30, 2007. The farm numbers are as listed in Table 3-2
AWST analyzed the met tower placement, the turbine layout and the terrain structure and
complexity for each of the sites to the extent that was possible with the data available to AWST.
In some cases detailed turbine layout information was not available. The insight from this
analysis is presented in Table 3-3. The comments indicate that the farms with low scatter
generally have better met tower placements than those with high scatter.
32
Table 3-3. Analysis of the Representativeness of Meteorological Towers
Group
Lowest scatter
Wind Farms No.
(from Table 3-2)
Farms # 1 & #2
Comments
The met tower is at same elevation as turbines
and is relatively centered on the prevailing
upstream side. This is close to an ideal
placement for a met tower.
Moderately low
scatter
Farm #3
This is similar to farms #1 and #2 but the
exact turbine layout is unknown. The met
tower may be at a lower elevation than many
turbines by about 20 m and some turbines
may be on 40 m ridges.
Moderately high
scatter
Farms #4,#5 & #6
There is a lot of terrain elevation variation
within two of these farms. At farm #5 the met
tower is at the low point on a ridge about 60
m below the high point. The exact turbine
layout is unknown, however, it is assumed
that the turbines are located in a single line
along the ridgeline. At farm #4, the met
tower is about 60 m below and several km
away from any turbine. Farm #6 is an
exception in this group. There is only about
20-40 m elevation variation and the met tower
is looks to be in the middle. However the met
tower is close to the water, which may reduce
its representativeness. Since the exact turbine
layout is unknown, it is difficult to make an
accurate assessment for Farm #6.
Highest scatter
Farm #7
The prevailing 250° wind direction has quite a
few of the turbines sheltered by a 200 m
ridge, but the met tower is not sheltered
There is a secondary prevailing direction
around 290° for which presumably everything
is sheltered by the ridge. But if any part of
the wind farm is relatively less sheltered for
this direction, it’s the met tower. Turbines
also range in elevation from about that of the
met tower to 60 m higher. The sheltering and
the elevation variation from met tower to
turbine produce a lot of scatter.
33
The preceding analysis provides considerable evidence that the amount scatter in the wind speed
vs power production relationship varies substantially among the existing wind farms even after
factors which can be modeled, such as wind direction effects, turbine availability issues and air
density variations have been considered. The brief analysis of met tower locations suggests that
the tower placement may be a major factor in the amount of scatter. Even if one accepts these
assertions, one would still be justified in asking whether the variations in scatter have much
impact on the error of the power production forecasts. Figure 3-13 provides some indication of
the answer to that question. This figure depicts the relationship between the standard deviation
of the power curve residuals to the mean absolute error (MAE) of the 36-hr AWST power
production forecasts for October 2007. It shows a clear relationship between the standard
deviation of the power curve residuals and the magnitude of the MAE. The farms with the least
scatter tend to have the lower MAE and vice versa. In fact a linear curve fit indicates that 62%
of the variance in MAE between wind farms for this month can be explained by the scatter in the
power curve relationship. This is rather impressive since there are a number of other known
factors that cause variations in forecast performance among the wind farms.
Figure 3-13. The standard deviation of the residuals from a speed-based power curve for each
of the 7 existing wind farms over a 6-month period vs the MAE of the 36-hr AWST power
productions forecasts for the October 1-29, 2007 period for the same wind farms.
A close examination of the wind speed and power production data from the wind farms revealed
some dramatic examples of the variable nature of the relationship between the wind speed
measured at a wind farm’s anemometer and the power production of the wind farm. One
particularly amazing example occurred during a 48-hour period during October 2007 at one of
34
the existing wind farms. A time series plot of the wind speed and wind power for the wind farm
is presented in the top panel in Figure 3-14. A scatter plot of all of the reported power
production and wind speed for each 10-minute interval during this 48-hour period
This cursory analysis includes a fair amount of hand-waving and certainly does not qualify as a
rigorous scientific investigation. Nevertheless, the results appear to be somewhat compelling in
their suggestion that more representative wind farm anemometer data would help forecast
performance. This could be done by sighting met towers at more representative locations.
Several towers may be required for wind farms that are located in complex terrain. Another
alternative is to make the data from nacelle-mounted anemometers (for those wind farms which
have them) available to forecasters. There would be some benefit even if the nacelle-mounted
anemometer could not be available in near real-time. These anemometers do not measure free
stream winds directly since they are behind the turbine rotors but still can be effectively used in
the forecast process.
35
Figure 3-14. An example of the complex and variable nature of the relationship between
measured wind speed and wind farm production as indicated by measured data from one of the
existing Alberta wind farms for a 48-hour period in October 2007.
36
3.6 Forecasting of Ramps
From a grid management perspective, accurate forecasting of ramps, or large and rapid changes
in power output may be more important minimizing the overall MAE or RMSE of the power
production forecasts. From this perspective, we will define a ramp as a change in power output
with a high enough amplitude and over a short enough period of time to cause short-term grid
management issues. Since upward ramps can be more easily managed by curtailment,
downward ramps are more important from a grid management perspective. For downward
ramps, the wind power must be replaced as it is lost to eliminate the need for more drastic
measures such as load shedding. For the purposes of this analysis, we will consider a ramp to be
important if the one hour power change is at least 20% of capacity for upward ramps and 15% of
capacity for downward ramps. With this definition, there were 20 upward and 20 downward
ramps of consequence during the third quarter of the project.
3.6.1 Ramp Forecasting Considerations
AWST’s forecasting system for Alberta was designed to minimize the overall RMSE of the
hourly wind speed and power forecasts. Accurate forecasting of ramp events was not an
objective. Forecasts of large ramps with a short duration are risky from a RMSE minimization
perspective. Inevitable phase errors will result in a large error for several hours. Especially with
RMSE, which considers the square of the error, these phase errors will contribute significantly to
the overall error. For this reason, a forecast system with the RMSE minimization objective tends
to smooth out power ramps over many hours.
Another consideration is that the forecasting system used in the Pilot Project was not configured
to produce probabilistic ramp forecasts. If shallow cold air was just to the north and east of the
existing wind farm area, the two most probable events are (1) that the cold air will remain to the
north and east and the power will remain high, and (2) that the cold air boundary will move
southwestward to the foothills and power production will drop to near zero. There is also a
somewhat smaller probability that the cold air will move westward far enough so that some, but
not all wind farms are in the cold air. In this case, production will drop, but not to zero. A
probabilistic forecasting system can assess the probability of a ramp occurring. In the case of
ensemble forecasting, it can also offer a range of possible outcomes and assign a probability to
each.
Ramp forecasting systems can be designed to forecast the probability of a ramp in any given
hour, it’s probable amplitude (or a probability distribution of amplitudes) as well as uncertainty
in the timing of the ramp and/or it’s duration. Inputs to such a system would include amplitude
and timing of actual ramps forecasted by physics-based (NWP) models, a statistical forecast
method, or the delivered optimized ensemble forecast. Section 3.4.4 explores the performance of
AWST’s physics-based model output and the optimized ensemble forecast as simple ramp
forecast systems.
One final consideration is related to ramp forecasts for aggregates of wind farms. Ramps will
tend to be smaller for aggregates that include a large number of facilities that are distributed over
a wide area and are spread among locations with varied wind regimes. These types of aggregates
tend to include many wind farms or groups or farms that have a power time series that are
relatively uncorrelated with other farms or groups of farms in the aggregate. For this reason,
37
strong upward ramps at some wind farms tend to be offset by downward ramps at others or at
least washed out by weaker ramps or steady production at others. Unfortunately, wind farms are
often built in a few relatively small regions to take advantage of the areas with the highest
climatological wind speeds. This tends to produce aggregates in which the individual farms are
highly correlated. Such aggregates are prone to more frequent large ramps.
3.6.2 Frequency of Ramps During the Project Year
The frequency of upward, downward and all ramps by season for the aggregate production from
the existing facilities is depicted in Figure 3-15. For example, the 15% of capacity power change
point on the horizontal axis shows the percentage of hours in the specified three or twelve month
period which had a power production change of at least 15% in the upward, downward or either
direction. Only hours for which power production data was available for all seven existing
facilities were included in this analysis. This was typically about 2/3 of the 2208 hours in a 3month period. One-hour power production changes larger than approximately 35-45% of
capacity generally do not have reliable frequencies as they represent only about 1 to 4 events.
Several conclusions can be drawn from the ramp frequency analysis:
1. Small to moderate ramps were less common from May to July.
2. Large downward ramps were less common from May to July while large upward ramps
were more common during this period.
3. Moderate to large downward ramps were most common from August through January.
Since this data represents only a one-year period, the values are not necessarily representative,
especially for relatively rare large ramps. It suggests that large downward ramps, which are the
most challenging for system operators, are more common in the fall and early winter. It also
suggests that large upward ramps, which often require curtailment, are more common in the
summer months. Both of these suggestions merit further analysis with a larger dataset.
The relative frequency of upward and downward ramps by ramp amplitude and season is
illustrated in Figure 3-16. It suggests that:
1. Large upward ramps are more frequent than large downward ramps throughout the year,
especially from February through July.
2. Upward ramps are more frequent than downward ramps in all amplitude bins from
February through April.
38
Figure 3-15. The percentage of hours for which a one-hour ramp exceeded a specified amplitude
threshold (% of capacity) by season for the aggregate production of existing facilities region for
(top) upward ramps, (middle) downward ramps, and (bottom) all ramps.
39
Figure 3-16. Frequency of upward and downward 1-hour ramps by season and ramp amplitude
for each quarter of the project year and the entire year.
3.6.3 Analysis of Ramps for Quarter 3 (Nov-Jan) by Weather Regime.
The understanding of the meteorological phenomena that cause large ramp events is an important
precursor to the development of a successful large ramp forecast procedure. The distribution of
ramps by type for the November through January (third) project quarter is depicted in Figure 317. This chart includes 5 types of ramp events.
The most common type of event is related to shallow cold air (SCA). SCA downward ramps
occur when a shallow cold air mass moves into the region from the east and north. Strong west
to southwest flow is replaced by light northerly to easterly flow in the cold air mass. Upward
SCA ramps occur when the cold air retreats back to the north and east and strong west to
southwesterly flow once again takes hold. Shallow cold air events comprise about 35% of the 3rd
quarter sample with a relatively even distribution of upward and downward ramps.
The next three event types are somewhat related. Cross-mountain flow events occur when a
significant change occurs in the cross-mountain component of the mountain top level flow.
Since the mountains in southern Alberta extend approximately along a 150°-330° line, the cross
mountain component of the flow is that component coming from 240° (more or less west
40
southwest). Two factors can increase the cross-mountain flow component: an increase in the
wind speed when the wind direction is close to 240°, or a turning of the wind from directions that
are parallel to the mountains (southerly or northwesterly) towards the 240° direction. Upward
ramps are caused by an increase in the cross-mountain flow component while downward ramps
are caused by decreases. Cross-mountain flow events comprise 32.5% of the sample with
upward ramps outnumbering downward ramps by about 2:1. However, if the two related event
types below are included, the combined events comprise 50% of the sample and the distribution
of upward and downward ramps is relatively even.
High-speed shutdown events occur when the cross-mountain flow component strengthens
enough to bring low-level wind speeds up to the turbine shutdown threshold of about 25 m/s.
For this reason high-speed shutdown events are really a special case of cross-mountain flow
events. In this case, downward ramps occur when the wind increases to above 25 m/s. Upward
ramps occur when the wind speed once again drops below 25 m/s. High speed shutdown events
comprise 7.5% of the sample.
Nocturnal stabilization and daytime mixing events are related to changes in the low-level
temperature profile. As the surface cools at night, air near the surface becomes cooler than air
several hundred meters above the surface. This tends to prevent the downward mixing of higher
wind speeds from aloft, resulting in a sometimes rapid drop in low-level wind speeds and
downward ramps. The resumption of solar heating in the morning warms the surface and causes
mixing to resume, resulting in wind speed increases and upward ramps. Nocturnal stabilization
and daytime mixing events comprise 10% of the sample with downward ramps outnumbering
upward ramps by 3:1. All three observed cases in this category also showed changes in the
cross-mountain flow component that tended to enhance the ramping.
Cold surge events occur when a moderate to strong cold front moves southward into the region.
Strong northerly winds immediately behind the front can cause upward ramps. More often,
though, the strong northerly winds replace strong westerly to southwesterly winds and no upward
ramp occurs. Downward ramps are somewhat more common with cold surges. They tend to
occur when the winds weaken rapidly between 1 and 9 hours after the frontal passage. In one
case, strong winds continued much longer, however they weakened gradually enough that no
strong downward ramp occurred. This case did result in a significant upward ramp as the
westerly flow in the warm air was relatively weak in this case. Cold surge events comprise about
15% of the sample with downward ramps outnumbering upward ramps by 2:1.
41
Figure 3-17. Composition of the ramp event sample by type for the combined sample of the 20
largest single-hour upward ramps and the 20 largest single-hour downward ramps for the 3rd
quarter of the project year (Nov-Jan).
3.6.4 Performance of a Simple Ramp Forecast System
As noted previously, the forecasts produced by AWST for the AESO Pilot Project were not
designed to forecast large ramp events. This section examines the possible performance of a
simple system intended to forecast ramp events in Alberta. A sophisticated ramp forecast system
was not implemented for this project. However, the raw physics-based model output is likely to
produce a better ramp forecast that an optimized combination of raw physics-based model output
and statistical methods that are designed to minimize RMSE. This section explores the relative
performance of the delivered forecast (optimized ensemble) and the raw output of three physicsbased models in forecasting ramps. Since the configuration and application of an objective ramp
forecast analysis tool would take more time and resources than is available for preparing this
report, a manual analysis of a sample composed of the 20 largest single-hour upward ramps and
the 20 largest single-hour downward ramps in the aggregated production of the existing facilities
was performed. The analysis does not include any false alarms (situations where a ramp was
forecasted, but did not occur). A cursory examination of the raw physics-based model power
production forecasts indicates that false alarms are a fairly common but are a bit less common in
the delivered optimized ensemble forecast. This would be addressed by using additional
statistical discrimination parameters in a sophisticated system,
First let’s look at the ramp event hit rate, the fraction of observed ramp events for which the
ramp was actually forecasted. Figure 3-18 shows the hit rate for upward ramps, downward
ramps and all ramps for the delivered (optimized ensemble) forecast and three raw model
forecasts. Two different hit criteria are considered. In the first case, forecast is considered a hit
if the forecast has a one-hour power ramp within 3 hours of the observed ramp that is of the same
sign as the observed ramp with amplitude at least 40% of the observed ramp. The second case is
similar, but with a minimum amplitude that is 20% of the observed rate. For the 40% threshold,
42
the hit rate ranges from 40 to 50% for downward ramps and 60 to 75% for upward ramps. The
overall hit rate ranges from 50 to 62.5%. The lower hit rate for downward ramps is consistent
across all forecast models as well as for the 20% threshold. For the 20% threshold, overall hit
rates are about 20% higher and the spread between downward and upward ramps is about 10%
greater. The overall hit rate is moderate for the 40% threshold and quite high for the 20%
threshold. Furthermore, Figure 3-19 shows that the average phase error is only about 1 to 1.5
hours, although it tends to be larger for the physics-based model forecasts than the optimized
ensemble forecast. It is interesting to note that phase errors for downward ramps tend to be
larger than for upward ramps. The results suggest that it is possible to create a ramp forecast
system that successfully forecasts most ramps within an hour or two of their actual occurrence,
although hit rates for downward ramps are likely to be lower while phases errors are likely to be
larger. Future work will need to examine the false alarm rate.
Figure 3-18. Ramp event hit rate for the 20 largest upward ramps and 20 largest downward
ramps for the 3rd quarter for hit two different hit thresholds. In the first case, a forecast is
considered a hit if the delivered forecast has a one-hour power ramp within 3 hours of the
observed ramp that is of the same sign as the observed ramp with amplitude at least 40% of the
observed ramp. The second case is similar, but with a threshold amplitude that is 20% of the
observed amplitude.
43
Figure 3-19. Ramp event mean absolute phase error for the 20 largest upward ramps and 20
largest downward ramps for the 3rd quarter.
Table 3-4 shows composition of ramp event hits and misses for the 20% and 40% thresholds by
ramp direction and type of event. Three event types are included: shallow cold air, cold surge
and cross-mountain flow. The cross-mountain flow events include three subcategories: nocturnal
cooling/daytime mixing, high-speed shutdown and all other events. Nocturnal cooling tends to
stabilize the atmosphere and cause wind speed reductions while daytime heating which tends to
have the opposite effect. High-speed shutdown events represent an upward or downward ramp
due to the commencement or termination of high wind speed shut down in a significant number
of turbines. The five types of events highlighted in red stand out. They include downward ramps
related to the cross-mountain wind, nocturnal stabilization, daytime mixing, and all high-speed
shutdown events. Together, they represent 65-80% of misses for the 40% threshold and nearly
all of the misses for the 20% threshold. For the 40% threshold, the majority of the remaining
misses are downward ramps related to shallow cold air. The best raw physics-based model
forecast also has some misses for upward ramps related to shallow cold air. High-speed
shutdown events are expected to be difficult to predict. They represent a 100% change in power
output over only a few m/s change in wind speed. Downward ramps related to the crossmountain wind and nocturnal stabilization are somewhat more difficult to explain. One
possibility is that as the mountain top cross mountain wind decreases, it is difficult to predict the
point at which stabilization of the lowest layers of the atmosphere will cut off the downward
mixing of higher velocities from aloft.
44
Table 3-4. Ramp event misses and miss rate by type of event
Total Events,
% of Total
Event Type
Cross-mountain Wind Downramp
High-Speed Shutdown
Nocturnal Stabilization
Cross-mountain Wind Upramp
High-Speed Shutdown
Daytime Mixing
Shallow Cold Air Downramp
Shallow Cold Air Upramp
Cold Surge Downramp
Cold Surge Upramp
Total
4
10.0%
2
5%
3
7.5
10
25%
1
2.5%
1
2.5%
7
17.5%
7
17.5%
4
10%
2
5%
40
40% Threshold
Misses (#, % of
sample)
Delivered Best
Model
6
4
86%
57%
2
2
100%
100%
2
1
67%
33%
0
0
0%
0%
1
1
100%
100%
1
1
100%
100%
3
4
43%
57%
1
3
14%
43%
1
0
25%
0%
1
0
50%
0%
15
14
20% Threshold
Misses (#, % of
sample)
Delivered Best
Model
2
3
28%
43%
2
2
100%
100%
1
1
33%
33%
1
1
10%
10%
0
1
0%
100%
0
0
0%
0%
1
0
14%
0%
0
0
0%
0%
0
0
0%
0%
0
0
0%
0%
6
7
Figure 3-20 shows the mean absolute forecast error of the ramp amplitude for the delivered
(optimized ensemble) forecast and the three different physics-based models. Only ramp events
for which a hit was scored are included in the statistics, hence the difference in the mean error
for the 20% vs 40% hit thresholds. It also shows the change in error when a simple bias
correction is applied to the ramp forecast. This correction is applied by simply subtracting the
bias from each forecast ramp amplitude. A separate bias is calculated for up ramps and down
ramps.
It must be remembered that the bias corrected forecast uses information (bias for the whole
sample) that would not be available at forecast time. The effectiveness of bias correction would
need to be tested by calculating a bias with a historical sample. Since, the absolute ramp
amplitude is too small in 100% of the delivered forecast and 90 to 95% of the raw physics-based
model forecasts, this relationship is likely to persist.
Since a large proportion of MAE for ramp amplitude is due to bias, a simple bias correction
reduces the MAE significantly, especially for the delivered forecast. It is interesting to note that
the uncorrected raw physics-based model forecasts have considerably lower error than the
45
delivered (optimized ensemble) forecast. On the other hand, the bias-corrected delivered
)optimized ensemble) forecast has considerably lower error than the bias-corrected raw physicsbased model forecasts. This preliminary evidence indicates that a bias correction applied to the
delivered (optimized ensemble) forecast would produce the best ramp forecast. The best hit rate
was with a 20% amplitude threshold. However, this analysis does not account for false alarm
rates, which are most likely much higher with a 20% threshold. This suggests that while the
statistical methods in the delivered forecast introduce a bias in the ramp amplitude, they improve
the forecast of the relative ramp amplitude. A simple bias correction takes advantage of the
strengths of the optimized ensemble forecast while eliminating its major weakness.
Figure 3-21 gives some indication of how the delivered (optimized ensemble) forecast adjusts
the ramp forecast. It shows a preliminary sample of 11 shallow cold air ramps. It is based on the
total ramp event from beginning to end rather than large single hour ramps for each event. The
ramp event is comprised of all hours for which the net change in power is at least 1/3 the
maximum hourly change in power for the event. The mean ramp durations of the raw physicsbased model forecasts are only slightly higher than the mean observed duration of 2.6 hours.
However, they have a negative amplitude bias of 35 to 45%. The delivered (optimized
ensemble) forecast mean duration is 3.9 hours, about 50% larger than observed. However, the
delivered (optimized ensemble) forecast bias is only about -20%, about half the value of the raw
model biases. This suggests that the delivered (optimized ensemble) forecast predicts the overall
amplitude of the event quite well, however, it smears the event out over several more hours than
observed. This hedging on amplitude is in response to the tuning of the forecast to minimize the
RMSE hourly power production forecasts.
Figure 3-20. A Comparison of Mean Absolute Error of the ramp amplitude for the 20 largest up
ramps and 20 largest down ramps in Q3 of the project for uncorrected and bias-corrected
forecasts.
46
Figure 3-21. Mean duration and amplitude bias for forecasts of all shallow cold air ramps for
November 2007 through January 2008.
47
3.7 AWST Forecasting Issues
AWST has identified several forecast system issues during the project that negatively impacted
its forecast performance, especially during the third quarter of the project. A couple of the more
significant ones are described below.
3.7.1 Shallow Cold Air Cases
Shallow cold air events on the east side of Alberta’s mountains are often poorly initialized in
NWP models since the nearest existing rawinsonde stations that offer a detailed vertical profile
of winds and temperature are at least several hundred km away in all directions. Surface
observations can pinpoint the location of the boundary between cold and warm air fairly well,
however, since surface observations do not offer a vertical profile, the model analysis tends to
underestimate the depth of the cold air. This makes a physics-based model much more prone to
mixing strong westerly flow to the surface than actually tends to occur. For this reason, the raw
physics-based forecasts tend to have a positive wind speed and power bias and larger overall
forecast error when shallow cold air is present. Figure 3-22 shows the power forecast bias for
shallow cold air (SCA) and non-shallow cold air (NON SCA) events for the third quarter for
both the delivered forecast and two raw physics-based model forecasts. In general the physicsbased models overpredicted power significantly. One model had a smaller bias, probably
because it was able to simulate the shallow cold air events somewhat better than the other.
The delivered (optimized ensemble) forecast had somewhat smaller biases for forecast hours 1-3,
but significantly larger errors for hours 12-48, especially compared to the model with the lower
bias. The use of recent power output data for hours 1-3 probably helped the statistical methods
to outperform the raw models in the early look-ahead hours. The higher bias in the later hours
can be attributed to the fact that the statistical schemes lump all cases into one sample. In
general, the model power bias is negative. Since shallow cold air cases represent less than 25%
of the sample even during the winter period when climatologically they are most common, the
statistical schemes will see an overall negative power bias. They will correct for this bias by
increasing the raw physics-based model forecasted power. This is useful during the NON SCA
hours, but during SCA hours, the forecast quality is actually degraded by this adjustment.
Figure 3-23 shows the power forecast RMSE for all hours, NON SCA hours and SCA hours.
The difference between the overall RMSE and the RMSE for NON SCA hours represents the
approximate expected improvement in RMSE if the RMSE during SCA hours improved to that
of the NON SCA RMSE. It is unreasonable to expect to develop a system that overcomes all of
the SCA forecasting challenges. On the other hand, the statistical adjustment of the NON SCA
forecasts is degraded to some degree by the SCA hours, which represent a sizable minority of the
total sample.
Several approaches could be used to improve the delivered forecast during SCA events:
1. Improve the initialization of shallow cold air in the physics-based model.
2. Find a parameter that can be calculated from observed or currently available physicsbased model forecast data that will accurately predict the classification of each hour in
48
the next 48 hours as an SCA or NON SCA hour. Apply a different statistical correction
to each regime.
3. Find a parameter that can be calculated from observed or available physics-based model
forecast data that is strongly correlated with the presence and/or strength of shallow cold
air. Power forecast error is strongly correlated with the presence or absence of shallow
cold air, as evidenced by the bias differences shown in Figure 3-22. Therefore, this
parameter would likely be strongly correlated with power forecast error. This parameter
can be used in the statistical scheme to adjust the power correction in such a way as to
increase the forecasted power in NON SCA cases and decrease it for SCA cases.
Approach 1 would require the installation and operation of sophisticated observing systems in
the existing facilities region. This is beyond the scope of this project, although it holds promise
for the future. Without an additional focus on approaches 2 or 3, its benefit could be lost through
erroneous statistical correction.
Approach 2 would be hindered by the small SCA sample size until several years of data were
available. Furthermore, it would not be able to model a possible smooth transition between
negative and positive power biases as the relative strength of the SCA increased. Still, it might
be useful to identify the strongest SCA cases in which production is almost always near zero.
Approach 3 has the advantage of being suitable to the first few years of forecasting since it does
not require the reduction of sample size through the creation of two or more regimes. It can also
pick up smooth transitions between strong SCA, weak SCA and NON SCA events. This
approach was implemented in late February, in time for most of the fourth quarter. However, the
majority of SCA events occurred in the third quarter. For this reason, we recomputed forecasts
for the 3rd quarter using the new forecast system that incorporated approach 3. Each recomputed
forecast used only data that would have been available at the time of delivery. The only
significant difference is that a small fraction of the observed power and wind data may have been
delivered too late to be used in the real time system. Since we did not keep arrival time stamps
in our data files, there is no way to know which data may have arrived late. Even so, this would
impact mainly the 1 and 2 hour forecasts with the effect decreasing to zero by about 6 hours. As
can be seen in Figure 3-24, by far the most significant improvements in the forecast were for 3
hours and beyond, so this is unlikely to be a significant issue. The power forecast MAE for the
existing facilities for both the original and modified forecast system is shown in Figure 3-24.
The one-hour forecasts are more or less equivalent. For all other forecast hours, the modified
forecast system has a lower MAE. The MAE reduction is about 2.5% at 2 hours. It increases
rapidly to 16% at 12 hours and then drops slowly to about 12% for the 48-hour forecast. This
represents a significant improvement over the original forecast system. This represents a
significant improvement over the performance of the AWST forecasts that was documented in
ORTECH’s Q3 report and demonstrates AWST’s ability to diagnose and resolve forecast
performance issues.
49
Figure 3-22. A comparison of power bias for Shallow Cold Air (SCA) and non-shallow cold air
(NON SCA) cases by forecast hour for the third quarter for the existing facilities.
Figure 3-23. A comparison of power forecast RMSE for Shallow Cold Air (SCA) and nonshallow cold air (NON SCA) cases by forecast hour for the third quarter for the existing
facilities.
50
Figure 3-24. AWST’s power forecast MAE for the 3rd quarter for the original delivered
optimized ensemble forecast (red) and the improved optimized ensemble forecast rerun in realtime mode (green).
3.7.2 Problems with Rapid Update Cycle (RUC) forecasts
On October 1, 2007, AWST implemented a 4 km resolution simulation using the WRF model
(WRF-RUC) based on the National Centers for Environmental Prediction (NCEP) Rapid Update
Cycle (RUC) model. This simulation was run every 3 hours for 12 hours. It was available about
2 hours after the initialization time, considerably earlier than the other AWST physics-based
model cycles that operate on a 6-hour update cycle.
It was hoped that this model and a statistical method based on its output would significantly
improve the 3-9 hour forecast by making frequent, relatively recent model runs available for
these forecast look-ahead times. This was indeed the case for the first 43 days of forecasting.
However, starting on November 13, the WRF-RUC forecast quality degraded significantly.
After considerable research, it was discovered a bug in the WRF model code caused WRF to
read the real-time snow cover data improperly. As a result, snow cover was greatly
underrepresented in WRF. WRF is a community model. Since AWST did not develop this
complex community NWP model, it would require considerable effort on our part to fix this bug.
For the most part, we must rely on the WRF developer community to address bugs; however, this
happens on their schedule.
Figure 3-25 shows the performance of a statistical method based in the WRF-RUC output data vs
that of statistical methods based on several other models and the delivered optimized ensemble
for (A) the period before significant snow cover was present in the mountains and (B) after
significant snow cover developed in the mountains. During the snow free period, the WRFRUC-based method had the best forecast for the 6-10 hour period. During the snowy period, it
51
was the worst for nearly all hours. Poor performance continued through the winter until about
March 20.
After March 20, the WRF RUC once again performed well. During this period, significant snow
cover was present in the mountains, however, the shorter nights, the presence of older snow
which behaves more like bare ground in it’s effects on the atmosphere, and likely lack of
significant snow on the lower eastern slopes of the mountain made snow cover less of an issue
for wind forecasting in the plains. Other than the periods of March 30-April 2 and April 20-26, it
once again had the best performing forecast, this time from about 3-4 hours through 10 hours.
The two exception periods were both times of snow cover in the existing facilities region due to
spring snow storms. Figure 3-26 shows an example of springtime performance during a snowy
and snow-free period.
In any case, the results show that the RUC approach holds considerable promise for improving
forecasts in the 3-10 hour range, if the snow cover issue is resolved. AWST plans to address this
issue with WRF in the near future. These results also indicate the importance of accurately
modeling the snow cover for wind power production forecasting in Alberta.
A
B
Figure 3-24. A comparison of the mean absolute power forecast error for the existing facilities
for the delivered (optimized ensemble) forecast (blue), the WRF-RUC’s statistical method
(peach) and statistical methods based on other physics-based models (other colors) for the
periods of (A) October 1-November 12, 2007 (a snow-free period), and (B) November 13-30,
2007 (a snowy period). Note that the WRF0RUC forecasts only extend to 12 hours after the
forecast delivery, Plotted values after this time have no meaning since they are based on
persistence of the 12-hr WRF-RUC forecast.
52
A
B
Figure 3-25. A comparison of the mean absolute power forecast error for the existing facilities
for the delivered forecast (blue), the WRF-RUC’s statistical method (peach) and statistical
methods based on other physics-based models (other colors) for the periods of (A) April 3-19,
20082007 (a snow free period), and (B) April 20-26, 20082007 (a snowy period). Note that the
WRF0RUC forecasts only extend to 12 hours after the forecast delivery, Plotted values after this
time have no meaning since they are based on persistence of the 12-hr WRF-RUC forecast.
53
4 Recommendations
Based on its experience in the Alberta Forecasting Pilot Project, AWST has compiled the
following recommendations for the implementation of an operational wind power production
forecasting system in Alberta.
4.1 Centralized vs. Decentralized Forecasting System
One of the most basic issues is whether the forecasts should be provided through a centralized or
decentralized forecasting system. In a purely centralized a system one (or more) forecast
providers are contracted through a single entity (such as the AESO) to provide forecasts for all
wind generation facilities on the Alberta Electric System. The central entity then provides the
forecast information to the individual wind generation resources as well as using the information
for its own purposes. In a purely decentralized system, each wind generation resource would
contract with a forecast provider or potentially could attempt to produce a forecast themselves
without a service provider. Each generation facility would then supply the forecast (schedule) to
the system operator. Each approach has advantages and disadvantages and it is certainly
possible to have a hybrid approach that incorporates some elements of both centralized and
decentralized systems.
A primary factor is cost. A centralized system is likely to have a lower total cost since the
economies of scale would likely enable a forecast provider to deliver forecasts with a lower cost
per generation facility. However, it is possible that the decentralized costs might approach those
of the centralized system if one or two providers were the dominant suppliers for the individual
generation facilities and could thereby achieve a substantial fraction of the economies of scale.
Of course, there would be no assurance that this would happen if a decentralized system was
implemented.
A second factor is forecast quality. In AWST’s opinion it is not clear which approach (if either)
would achieve a better overall forecast performance. In theory, the decentralized approach would
encourage a forecast provider to focus more attention on each individual site and possibly
develop a higher degree of forecasting method customization for the site. If the provider did not
perform well for that site, the owner/operator of the facility could seek another provider that
might achieve better performance. If all owner/operators aggressively sought the best possible
forecasts for their site, it could result in the best system-wide forecast as well. However, in
practice, there would likely be a large degree of variation in the demand for quality performance
for each facility. Some facilities might pay a lot of attention to this and others might decide to
reduce costs by going with the lowest cost provider regardless of quality or trying to concoct
some forecast by themselves without using a forecast provider. The implementation of systemwide forecast performance standard or penalty for poor scheduling performance might suppress
the tendency to cut costs on forecasting. However, if the motivation is solely to avoid penalties
or a meet a minimal performance standard it is not likely that the owner/operator would be
willing to incur an added cost to achieve better performance beyond the minimum standard. The
centralized system will probably insure a more uniform quality but may not result in the best
possible forecast for each facility. However, it may not result in the best system-wide forecast if
some owner/operators are not motivated to pay for and facilitate the highest possible quality
forecast for their facility. It is also possible that the benefits of site-specific customization will
not be very significant and that much of the useful customization will be similar at nearby sites
54
and could cost effectively be accomplished as part of a centralized system. In practice, most of
the customization benefits for individual sites will probably be for the very short-term (0-6 hr)
look-ahead periods. The centralized system also provides more of an opportunity to implement
a multi-forecaster ensemble since two or more forecasters could be hired to provide forecasts for
all generation facilities. This is unlikely to happen in a purely decentralized system.
A third factor is data utilization. In a centralized system, it is likely that data from all of the wind
generation facilities will be available to be used in the production of forecasts for the other
facilities. This can occasionally be a significant asset for short-term (0 to 6 hrs) forecasts since
data from an “upstream” facility might be a useful predictor for future variations at a
“downstream” facility. In a decentralized system it is possible that proprietary issues with the
data will prevent a forecast provider from using data from one facility to benefit the forecasts for
another facility even if the both facilities utilize the same forecast provider. Of course, the
situation would be even more difficult if the facilities used different forecast providers.
In AWST’s opinion the best approach is to implement a two-provider centralized system. This
ensures a higher level of reliability due to the additional redundancy. This would also facilitate
the use of a simple ensemble composite of two forecasts that, in theory, is likely to have better
overall performance than one provider. It is also possible that over a period of time that it will be
established that one provider will perform best under some circumstances and the other provider
under other circumstances. This might also ensure that the system is quickly updated with new
forecasting technology since there will be some element of competition between the providers.
A more than two forecaster system might also be considered but it is AWST’s opinion that the
benefits obtained from additional forecasters is not likely to justify the added cost.
4.2 Technical Requirements
Observed power and meteorological data from each wind farm is essential for two reasons. First,
one or more well sited, well-maintained towers that report wind speed at turbine hub height are
required in order to develop an accurate conversion from forecast wind speed to forecast power.
Second, recent observations of both power and meteorological variables are essential to produce
state-of-the-art 1 to 3 hour forecasts. Figure 4-1 shows various classifications of wind power
plant input data. Class 2 serves as a minimum requirement for state-of-the-art forecasts. Class 3
is preferred and is the standard at most new plants. However, for southern Alberta in particular
there are important reasons to provide Class 4 data. Wind farms in southern Alberta are often
influenced by a shallow layer of cold air that hinders the downward mixing of higher wind
speeds from aloft. Under these conditions, turbines sited on ridges or higher up on slopes may be
producing significant power while those at lower elevations are not. For this reason, two
additional factors become important. First, it is important to measure temperature at two levels
to determine if shallow cold air may be present. Second, the measurement of wind speed at two
levels further aids in determining the inhibition of vertical mixing. Class 4 provides for both of
these measurements. Once known, vertical changes in temperature and wind speed can be used
to fine tune the power conversion to account for the fact that turbines at elevations different from
that of the met tower may be seeing higher or lower wind speeds.
The siting of meteorological towers is also important. Several existing facilities in Alberta suffer
from poorly sited meteorological towers. At one site, the turbines are on a ridge while the met
tower is on the plains below. At another, the meteorological tower is near the turbines that are
55
relatively unsheltered from westerly flow by a nearby ridge while other turbines are more
sheltered and at higher elevation. At a third site, the turbines are suspected to be along a sloping
ridge with the met tower is at the lowest point on the ridge. At a fourth site, the met tower is at
the edge of a bluff while some turbines are suspected to be sited back from the edge of the bluff.
Most of these issues could be addressed with a single, well-sited met tower, for example, on the
ridge with the turbines in the middle of the row. At sites that are spread out over varied terrain,
two or more towers may significantly benefit the forecast.
High data availability is also important both in creating an accurate power conversion method
and in providing a consistently accurate short-term forecast. All meteorological and power data
should ideally have 90% or higher availability.
Finally, historical data is useful both in developing the power conversion equation and in
developing robust statistical forecast methods. Ideally, each site should provide all historical
data that is available since the wind power plant went into operation.
4.3 Methodology and Approach
A key factor in the implementation of an operational wind power production forecast system in
Alberta will be the specification of the forecast requirements. What type of forecast information
is desired? To what forecast performance metrics should the forecast be optimized? How should
the information be displayed? What look-ahead time scales are most important?
It is clear that the forecasting of large ramps in production is a significant need. It should be kept
in mind that a system designed to minimize the overall RMSE or MAE is not likely to do a very
good job at forecasting ramp events. Thus, ramp events forecasts should be considered a
separate forecast product that is distinct from the product that provides hourly values that
minimize the RMSE or MAE over all hours. An attractive paradigm for this would be an event
oriented ramp forecast that would provide the probability of a ramp at several different amplitude
levels (i.e. 20% of capacity per hour, 40% of capacity per hour etc.) during a specified time
window (e.g. 4 hours). The forecast could be disseminated via a customized web display that
could provide different viewing options for different applications.
Finally, there is the possibility that a human could have some input into the forecast process.
This could be implemented on the forecast provider end or the AESO end. There was some
suggestion during the project that an experienced human forecaster might be able to add value to
the forecast performance. One paradigm to implement approach would be to have the human
forecaster select which of multiple forecasts for the same time period is likely to be the best.
Some success with this approach was demonstrated in one case during the project. Another
paradigm would be for the human forecaster to focus on the short-term (0-6 hrs) and look for
atmospheric features in radar, satellite and other data sources that were likely not well
represented in the automated forecast procedures. In general, the current automated systems do
not work well with incomplete information about small-scale features. This was seen in some of
the special event cases that were analyzed in this project (e.g. Sept 6). Human forecasters tend to
do a better job of working with incomplete information because of their superior pattern
recognition skills. This could yield some performance benefits for 0 to 6-hr ahead forecasts.
56
Figure 4-1. Different classifications of wind power plant data used as input into the eWind
forecasting system. Client should strive to provide minimum Class 2 data. Forecasts can be
generated with Class 1 data, but forecast performance may be compromised. Most plants
currently in development utilize Class 3 data.
57
5 Next Steps in Alberta
AWST’s recommendation for the next steps that should be taken in Alberta to obtain the highest
quality wind power production forecast information that optimally meets the needs of both the
AESO and other stakeholders in Alberta are presented in this section. Each recommended step is
described in a separate subsection.
5.1 Establish Forecasting Priorities
It was noted many times in meetings and conference calls during the project that most forecast
systems could be configured and tuned for different objectives. The tuning of the forecast system
to meet one objective will often degrade the performance for another objective. A simple
example of this is the evaluation metric that is designated for minimization by the forecast
system. If a forecast system is configured to minimize the squared error, the root mean square
error (RMSE) is likely to be as low as possible with the available data and the models employed
but the mean absolute error (MAE) will be higher than it could have been if the forecast system
had been configured to minimize the MAE. Thus, one forecast product can’t deliver the best
possible RMSE and MAE. Therefore, it is important to configure the forecast system to obtain
optimal performance for the critical objectives of the forecast user. For example, a minimization
of the RMSE may be most relevant to grid reliability applications because an RMSE
minimization will tend to reduce the magnitude of the largest errors that often are the most
significant in the grid management process. However, a minimization of the RMSE will tend to
increase the average absolute error. The average absolute error is often the key metric for
market-based applications in which the user has an economic consequence that depends upon the
deviation of the actual production from the scheduled amount. In most of these applications, the
economic consequence is directly proportional to the deviation from the scheduled amount and it
does not matter if the accumulated deviation over a period results from a consistent series of
moderate errors or an error series that includes a very small and very large errors. It is only the
total deviation that determines the economic consequence. In this situation, a forecast product
that was designed to minimize the RMSE will result in a higher cost to the forecast user than one
that was designed to minimize the MAE.
Therefore, it is critical that forecast providers have a clear understanding of the specific
forecasting-related objectives of the forecast user and how those objectives are sensitive to
power production forecast error. This will enable the forecast providers to configure their
forecasting systems to achieve optimal performance for the forecast products that the meet user’s
critical objectives. In some cases, it may be necessary to produce multiple forecast products to
meet multiple objectives.
Therefore, one of the first steps that should be taken in Alberta is to establish a prioritized list of
the objectives that the Alberta stakeholders are expecting (hoping) to achieve through the use of
wind power production forecasts. This list should include a specification of the “cost function”
(i.e. how the application is sensitive to forecast error) for each objective. In some cases the cost
function may be easily specified in a quantitative manner. This would be the case, for example,
in a market application where the user’s economic consequence is directly proportional to the
deviation from the forecast. In other cases, the cost function may be more qualitative because
the costs are indirectly sensitive to forecast errors in a complex manner. For example, a grid
operator might note that large wind production forecast errors during the daily periods of rapid
58
changes in load are much more of a problem than at other times.
probabilistic forecasts may be more useful than deterministic forecasts.
In other applications,
5.2 Upgrade Quality of Data from Wind Generation Facilities
The quality and quantity of meteorological, power production and turbine availability data from
the wind generation facilities has a significant impact on forecast performance. There are a
number of issues that have frequently arisen with the data provided by wind generation facilities
in Alberta and other locations. These issues include: (1) absence or inaccurate reporting of
turbine availability information; (2) unrepresentative location for the onsite meteorological tower
or a number of meteorological towers/anemometers that is inadequate to describe the variation in
wind within the areas encompassed by the facility; (3) extended data outages due to
communications or sensor problems; (4) erroneous data due to improper maintenance and
calibration of the sensors; and (5) the lack of secondary measurement data (e.g. two levels of
wind direction and speed and temperatures, turbulence intensity, icing etc.) which at some sites
can be very useful in modeling variations in power production due to changes in boundary layer
stability, wind shear and turbulence intensity or other factors such as icing.
The forecasting process would benefit from establishing an environment in which the owners
and operators of wind generation facilities are motivated and encouraged to provide the highest
quality and most comprehensive dataset for wind forecasting applications that is economically
feasible. This environment may include an establishment of standard protocols for the type and
amount of data that should be gathered at wind generation facilities as well as a mechanism to
educate stakeholders on the value of such data and ultimate benefits that they receive by
investing in the gathering and dissemination of these data.
5.3 Develop an Off-site Real-time Meteorological Database
Many critical grid management decisions require reliable estimates of wind power production in
the 1 to 3 or 4 hour-ahead time frame. The variations of wind on this time scale are driven by
small-scale atmospheric features that are typically not well detected by the standard array of
meteorological sensors that is used as input into NWP models and also used by other human and
automated forecasting procedures. Research has demonstrated that data at locations surrounding
the wind generation facilities can have a significant beneficial impact on the performance of
short-term forecasts. One potential use of such data in the forecasting process is in the
application of time-lagged off-site correlations. These are correlations between the future
changes in wind at the wind generation facility and prior changes (relative to the time the
forecast is produced) in a meteorological variable (wind speed or other parameter) at an off-site
location. In some cases, the prior changes can explain a meaningful portion of the variance of the
future changes at a site. An example of this type of relationship in Alberta is shown in Figure 51. Another way the off-site data might benefit short-term forecasting is through its use as
supplementary initialization data for a rapid update cycle (every hour) of a physics-based (NWP)
model. AWST experimented with the use of such an NWP model configuration during the Pilot
Project but no supplementary off-site data (i.e. in addition to the hourly meteorological data from
standard sources such as airports) was available to use in its initialization.
In many cases the desired off-site data are not available in real-time or not available at all.
Thus, there is an opportunity to improve short-term forecasting by developing a real-time
supplementary meteorological database for the power production forecasting application.
59
AWST’s vision is that there would be three major components to this effort. The first one would
be an investigation of what types and sources of off-site data are available in Alberta from public
and private sources that do not currently make their data available in real-time through standard
meteorological channels such as Environment Canada. The second component would be an
investigation, perhaps in conjunction with Environment Canada or other public entities, to the
feasibility of installing meteorological towers at locations that have been identified as having
significant potential to improve wind power production forecasts in Alberta. The third
component would be an investigation of the possibility of establishing a meteorological remote
sensing network in Alberta that would provide a more comprehensive 3-D dataset of winds in the
atmospheric boundary layer than is possible through the deployment of meteorological towers.
This investigation most likely should also be done in conjunction with Environment Canada and
other public entities. The implementation time horizon for this component would necessarily be
longer than that for the other two, since it would require a larger commitment of resources and
many of the lower cost exciting remote sensing tools (such as low cost boundary layer Doppler
radars) will not be available for a few more years. However, the preliminary indications are that
these tools will offer potentially huge benefits for short-term wind power forecasting since they
will enable the detection of many small-scale atmospheric features that can’t be effectively
detected through the sensor technologies that are currently deployed.
Figure 5-1. The simulated correlation between the one-hour ahead change in wind speed (a
simulated forecast) at the Castle River facility and the 1-hr prior change (a simulated
measurement) in the east-west wind component for each point in the simulation domain for one
of the April wind regimes. The high correlation just to the west of the Castle River site implies
that a wind speed and direction measurement at this location would add to the predictability of
the 1-hr ahead change in wind speed at Castle River.
60
5.4 Establish an Alberta-oriented Forecasting R&D Program
A strong research effort on Alberta-specific meteorological issues is required in order to
optimize the use of current data and models as well as to derive the most benefit from any future
improvements and expansions in onsite and offsite data quality and availability. As noted
previously in this report, and in discussions during the project, there are many Alberta-specific
meteorological phenomena that impact the quality of wind power production forecasts in
Alberta. Many of these phenomena have been noted in Section 3 of this report. It should be kept
in mind that the Pilot Project was not designed to conduct research to improve forecasting of
atmospheric phenomena that are important in Alberta. It was designed to establish a base level of
forecast performance and possibly identify the topics that should be the focus of future research.
The results of this project suggest that there are several areas of research that may yield
considerable benefit to the performance of both short-term and day-ahead forecasting. Some of
the most promising areas of research are summarized in the following subsections.
5.4.1 Customization of the Regime-Based Forecasting Concept
One area of research that is likely to yield significant forecast improvement is the tuning of the
regime-based forecast schemes. Most of the ensemble of forecasts are generated by statistical
techniques that define a relationship between forecast inputs that are known at the forecast
delivery time and the predicted wind speed or power production. The relationship aims to
minimize a given measure of forecast error, typically MAE or RMSE. For forecast look-head
times of three hours or less, inputs typically include current and recent observed meteorological
variables (especially wind speed) and power output. At these times, the base forecast is typically
taken to be persistence. These relationships define corrections to persistence. For look-ahead
times of 6 hours or greater, inputs typically include both observed variables and predicted
variables extracted from an NWP model. The base forecast is taken to be the NWP model
forecast interpolated to the site. These relationships define corrections to the model forecast.
Intermediate term forecasts typically include a mix of both types of variables.
In any case, one hopes that one or two input variables shows at least a moderate to strong
relationship with either the persistence forecast error or the model forecast error. The problem is
that the relationships often vary from one weather regime to the next. If all of the forecasts are
lumped together, one typically gets a relatively weaker relationship that is dominated by some
combination of the most common regime or the strongest relationship for a typical regime. This
“watered down” relationship does not bring as much benefit as it could to the regime for which it
is relevant. It often worsens the forecast for the other regimes.
Regime based forecasting solves this problem by assigning each forecast to a regime. Separate
statistical relationships are developed for each regime. Defining regimes requires extensive local
forecasting knowledge in order to define regimes that are meaningful for a given region. It is
somewhat of an open-ended process since there are essentially unlimited ways to define regimes.
It also requires a relatively long period of record since many regimes are relatively uncommon,
or tend to occur only in a particular season. A record of less than several years or even more
would be insufficient to develop meaningful statistical relationships for the less common
regimes.
61
Creating a regime based forecasting system requires several steps:
1. Study the local meteorology of an extended period to identify specific forecast situations
that tend to recur and fit a common theme. For example, in Alberta, the shallow cold air
regime is relatively common in late fall and winter.
2. Define an objective regime classification technique that is able to reliably separate the
common forecast situations into specific groups. Any situations that do not meet the
criteria for any regime can be classified into their own nonspecific regime.
3. Develop specific statistical relationship for each regime.
The optimized ensemble statistical method assigns weights to each individual forecast based on
recent performance to create the ensemble forecast that is delivered. It too could be customized
to each regime by determining the weights separately for each regime.
5.4.2 Use of Flexible Regimes
Another related approach that has significant potential is the use of “flexible regimes”. These
regimes are defined uniquely for each forecast situation. A prior forecast is considered to be part
of the regime if a specified set of criteria is met. One approach is to calculate the correlation of
the current model forecast with prior forecasts. Prior forecasts that exceed a threshold
correlation are included in the regime. Another is to calculate a specific parameter or set of
parameters from model output. Prior forecasts would be included in the regime if they are
similar enough to the current forecast parameters. In any case, if very few prior forecasts meet
the criteria, this would indicate a relatively rare regime for which the forecast sample size is
insufficient. In this case, the best approach would most likely be to use the raw physics-based
model forecast rather than to use statistical corrections that are not well suited for the rare
regime.
5.4.3 Expanded Use of the RUC Approach
AWST utilized an experimental version of the rapid update cycle (RUC) physics-based model
approach in the Pilot Project. It encountered some problems, as documented in Section 3.5.
However, in situations where the impact of the problems was not significant, it produced a
significant improvement in short-term forecasts over the conventional 6-hr update regional NWP
approach. This improvement was achieved despite the fact that there was essentially no Albertaspecific data added to the initialization process. Most of the improvement was related to the use
of standard meteorological data at a higher frequency than that used in the conventional NWP
cycle approach. There are reasons to believe that further customization of the RUC approach
for the Alberta forecasting application have the potential to produce further significant
improvements in forecast performance.
62
6 Next Steps in Industry
In addition to the steps suggested for Alberta in the previous section, AWST also suggests that
there be several steps taken on an industry-wide basis to optimize the performance and utilization
of wind power production forecasts. The suggested steps are outlined in the following
subsections.
6.1
Establish Forums for Communication
There is a significant need to improve the communication between industry subgroups. A key
component of this is the communication between the forecast providers and the forecast users.
One objective is to educate current or potential forecast users about a number of issues related to
forecast use. These include: (1) the proper interpretation of forecast information; (2) the
strengths and weaknesses of current state-of-the-art forecasts; (3) the need for users to
understand how their application is sensitive to forecast error; and (4) an awareness of the key
role that onsite meteorological and operational data play in the forecast process. A second,
equally important objective, is to educate the forecast provider about the specific needs of
various types of forecast users and the most effective ways to communicate forecast information.
An effective forum would facilitate the interaction of forecast provider and user and result in the
design of forecast systems that optimally meet the needs of different classes of users.
6.2 Establish Forecast Performance Benchmarks
There is also a need to establish benchmarks for forecast performance. Wind power production
forecast performance varies substantially due to a wide range of factors. These factors include:
(1) the look-ahead time period; (2) the type, quality and quantity of meteorological and
operational data from a wind generation facility; (3) the site-specific distribution of wind speeds
along the plant-scale power curve (the sensitivity of wind power production forecasts to wind
speed errors is proportional to the slope of the power curve); (4) the size of the wind generation
facility and the diversity of the wind resource within the domain of the facility; (5) the types and
scales of meteorological features responsible for the variations in wind speed and direction at a
site (some are more predictable than others); (6) the objective of the forecast (e.g. a forecast
designed to minimize the RMSE generally won’t do as well as it could when measured by the
MAE metric); and (7) the input data and forecasting techniques used by the forecast provider.
The fact that there are a large number of factors, many beyond the control of the forecast
provider, that impact forecast performance means that there is no simple answer to the question
of “how good are power production forecasts?” This creates a number of issues. Users don’t
have a clear indication of what level of forecast performance should be expected from a forecast
provider.
It is also difficult to determine the trend in forecast performance over time.
Independent researchers (such as those at universities) find it difficult to identify the critical
problem areas in forecast performance and thus don’t effectively focus their research on the
critical problems. Policymakers and planners do not have a resource to obtain accurate and caserelevant forecast performance information for decision-making purposes – e.g. what level of
uncertainty do we need to anticipate when designing our grid management protocols? Many of
these issues can be addressed by creating a mechanism to establish a forecast performance
database. This can be done in many ways. There should be a dialogue to establish the most
economical, widely-acceptable and effective way to do this.
63
6.3 Improve Data Quality from Wind Farms
There is an enormous variation in type, quantity and quality of meteorological and operational
data supplied to forecast providers by wind generation facilities. There is a diverse set of reasons
for this. However, the quality and cost of forecasts would be improved by establishing standards
for the type and quality of meteorological and operational data that is gathered by a wind
generation facility as well as the mechanism (web services, ftp etc.) and format (file types,
contents etc.) through which data is communicated from the wind generation facilities to the data
users (the system operators, forecast providers etc.).
6.4 Advocate/Lobby for Relevant New Measurement System Technology
There is little doubt that the most significant factor that limits the performance of wind power
production forecasts on all short-term (0 to 48 hours) time scales is the availability of quality
data. While modest improvements can, no doubt, be squeezed out of existing data sources
through the use of more sophisticated or innovative physics-based or statistical modeling
techniques, evidence suggests that major improvements are likely to be linked to new or better
data sources that will enable the detection of atmospheric features that can’t be effectively
detected with current observing system. This is especially true for short look-ahead times.
The deployment of many new sensing systems by public entities will occur as part of the
continuing evolution of atmospheric sensing programs for general weather forecasting.
However, a specific sensing systems will not provide equal (or anything close to equal) benefit
for all forecasting applications. Hence, there are some types of new atmospheric sensing
technology that will have much greater benefits to the wind power production-forecasting
problem than others. The longer-term evolution of wind power production forecasting
performance will benefit from an industry effort to advocate for the deployment of sensing
technologies that will have a substantial benefit to wind power production forecasting. This
should include both space-based and ground-based sensing systems. There are several currently
proposed or developing systems in each category that have great potential to improve wind
power production forecasting. However, in some cases, their deployment priority is lower than
other systems that would have less of an impact on wind power production forecasting. In view
of the rapidly rising support for renewable energy, a strong message of support from the industry
may help to raise the priority for the systems that are most beneficial to wind power production
forecasting.
In the shorter term, the industry could also enlarge the pool of data available to forecast
providers by facilitating the use of data from existing sensor networks. There are a number of
currently deployed public and private sensor networks which could provide valuable data but at
present this data is can’t be used because: (1) it is difficult to access and use; (2) the existing
communications infrastructure can’t make the data available in near real-time; (3) the cost of
acquiring the data is too high for use by an individual forecast provider; (4) the data is
proprietary and the owner does not make it available to others; and (5) none of the relevant
people in the wind industry has knowledge of the sensor network. It is likely that some of these
obstacles can be overcome by a coordinated action within the wind industry. For example, it
would be useful to have a centralized database of non-standard real-time meteorological data
sources that were relevant to wind power production forecasting.
64
6.5
Support Wind Power Production Forecasting R&D
With or without the available of new data sources, it will be important to conduct research of
specific wind power production forecasting problems and to continue the development of aspects
of both physics-based and statistical models that are particularly relevant to wind power
production forecasting. The industry should advocate the support of such research. The first
step is to identify the critical problem areas in wind power production forecasting and make that
information available to the research community. A second step would be to encourage
provincial, state and federal governments to support research and development in the critical
topic areas.
65

Forecaster Report

Transcription

Similar documents

HPC and numerical modeling tools at the Chilean Weather

LIPA Front-End Transition Cost Estimate

Insurance version 3 - WeatherBELL Analytics

Vitalii Shpyg

Jonathan cainer www.CAiNER.COM

Bonneville Power Administration - Northwest Hydroelectric Association

pid2015 - november 12th - van nelle fabriek rotterdam

Long range forecast services in KMA and Introduction of Joint

Shpyg Vitalii - c