Validation of H-SAF Precipitation Products

Transcription

Validation of H‐SAF Precipitation Products
Bożena Łapeta, Silvia Puca and Precipitation Validation Team
Satellite Remote Sensing Centre, Institute of Meteorology and Water Management, Kraków, Poland
(In the presentation materials from H‐SAF PVR documents were used)
Presentation Overview
• H‐SAF prcipitation products validated
• Validation with ground data:
– methodology
– results
• Conclusions
NOAA Training on “New and Emerging Technologies, Sensors, and Datasets for Precipitation” 15‐17 October 2012, Sao Jose dos Campos, Brazil
Precipitation Products Validation Group (PPVG)
Coordination : DPC (Italy)
Italy
The PRECIPITATION
PRODUCT VALIDATION
GROUP
is composed by 24
experts in hydrology, rain
gauge data, radar data,
and meteorology coming
from 8 countries.
DPC
Silvia Puca
Coordination
[email protected]
Belgium
IRM
Emmanuel Roulin
Pierre Baguis
[email protected]
[email protected]
Bulgaria
NIMH
Gergana Kozinarova
[email protected]
Georgy Koshinchanov
[email protected]
Germany
BfG
Peter Krahe
[email protected]
Hungary
OMSZ
Kereney Judit
[email protected]
Italy
DPC
Silvia Puca
[email protected]
Gianfranco Vulpiani
[email protected]
Emanuela Campione
[email protected]
Alexander Toniazzo
[email protected]
Uni.
Ferrara
Federico Porcù
[email protected]
Lisa Milani
[email protected]
CIMA
Simone Gabellani
[email protected]
Nicola Rebora
[email protected]
Bozena Łapeta
[email protected]
Rafal Iwanski
[email protected]
Jan Kanak
[email protected]
Marian Jurasek
[email protected]
Luboslav Okon
[email protected]
ahmet oztopal
Ibrahim Sonmez
[email protected]
[email protected]
Poland
Slovakia
Turkey
IMWM
SHMÚ
ITU
TSMS
Precipitation products
Product
Resolution (Europe)
Cycle (Europe)
Timeliness
Precipitation rate at ground by MW conical scanners (H‐01)
10 km (with CMIS)
6 h (with CMIS only)
15 min
15 km (with other GPM)
3 h (with full GPM)
Precipitation rate at ground by MW cross‐track scanners (H‐
02)
10 km
6h
5 min
Precipitation rate at ground by GEO/IR supported by LEO/MW
(H‐03)
8Km
15min
5min
3 h
15 min
Cumulated rain 3 and 24 h (H‐
05)
10 km
(from merged MW + IR)
H‐SAF validation methodology
1.
The Common Validation is the result of the validation activities done by all the Institutes involved in the HPPVG:
– both rain gauges (4100 posts) and radar data (40 C band radars) are used;
– it is based on statistical scores evaluated on multi‐categorical and continuous statistics;
– the statistical scores are monthly averages;
– the same up‐scaling techniques by all the institutes (if proposed by developers).
2.
Specific validation
Each Institute in addition to the common validation methodology has developed a
specific validation methodology based on its own knowledge and experience.
– lightning data, numerical weather prediction and nowcasting product
– case studies: convective/stratiform precipitation, day/night, land/ocean
How to compare satellite products with ground data?
Comparison between the satellite data and ground data were done on the satellite product native grid.
There are several approach to bring the observations comparable:
• to compare untransformed data, e.g. comparing areal data to observations at a nearest gauge station.
• to upscale the reference observations to areal averages corresponding to the resolution of the precipitation products but in an equal‐area map projection (interpolation of RG data; averaging of radar data within the product pixel)
How to combine satellite products with ground data?
Radar and rain gauge instruments provide many measurements within a single satellite IFOV, those measurements were averaged following the satellite antenna pattern of AMSU‐B, SSMI and SEVIRI.
Gaussian filter used to average ground data within satellite H‐02 pixels
Rain Gauges network (4100)
Data Sources
Instrument
characteristics
Time domain (near
real time/ case
studies)
I will put the most actual
Raingauges
Telemetric and mechanic
Near real time, case
studies
10 – 30 min
Time resolution (15
(telemetric),
min, 30 min)
map (from Jakub)
3 – 24 h (mechanic)
Spatial distribution
(whole national
Whole national territory
territory/ limited
area)
Number of station
(please attach a
map)
~390 mechanic (RMI) +
12 telemetric (RMI) +
4160 telemetric (SETHY)
Operational/ for
research only
Operational (RMI) +
research (other
networks)
Data quality check
Telemetric:
automatically checked /
mechanic: autom. +
manually checked
Radar network (40 C‐band)
Data Sources
Instrument
characteristics
Time domain (near
real time/ case
studies)
Raingauges
Telemetric and mechanic
Near real time, case
studies
10 – 30 min
Time resolution (15
(telemetric),
min, 30 min)
3 – 24 h (mechanic)
Spatial distribution
(whole national
Whole national territory
territory/ limited
area)
Number of station
(please attach a
map)
~390 mechanic (RMI) +
12 telemetric (RMI) +
4160 telemetric (SETHY)
Operational/ for
research only
Operational (RMI) +
research (other
networks)
Data quality check
Telemetric:
automatically checked /
mechanic: autom. +
manually checked
H-SAF validation methodology
Main steps of the common validation:
•
•
•
•
Selection of satellite pixels falling into the region of interest;
Taking into account quality index information (to be implemented);
Radar: selection of the radar data synchronous with the satellite ones;
Rain gauge: selection of the radar data synchronous with the satellite ones and spatial interpolation of rain gauge data;
• Up‐scaling of ground data at the resolution of the native satellite grid, or nearest‐neighbour matching;
• Statistical score calculation.
The common validation is now performed by means of a unified software developed at DPC, Italy (contact person: A. Rinollo, [email protected])
H-SAF Precipitation Products Validation and Application, Training Course
6th IPWG Workshop Sao Jose dos Campos, Brazil 15-19 October,
Common methodology
MC statistic:
– ACCURACY
– POD – FAR – BIAS
– ETS
– OR
– HSS
Plots:
‐ Scatter plot
‐ Probability density function
CS statistic:
‐ Number of points
‐ observed Mean rain (rate or cumulated)
‐ Satellite Mean rain (rate or cumulated)
‐ Observed Maximum rain (rate or cumulated)
‐ Satellite Maximum rain (rate or cumulated)
‐ Mean error
‐ Multiplicative bias
‐ Mean absolute error
‐ Root mean square error
‐ correlation coefficient
‐ Standard deviation
Period: January 2009‐ March 2010
The PP Validation System
BELGIUM
‐RMI
BULGARY
‐NIMH
GERMANY
‐BFG
HUNGARY
‐HMS
ITALY
‐Uni. Fe
POLAND
‐IMWM
SLOVAKIA
SHMU
TURKEY
‐ITU, TSMS
comparison national radar and rain gauge data with precipitation products on satellite native grid using common software
• evaluation of the monthly continuous scores and contingency tables for the precipitation classes
• evaluation of PDF producing numerical files called ‘DIST’ files and plots
• numerical files called ‘CS’ and ‘MC’ files
• numerical files called ‘DIST’ files and plots
ITALY
‐DPC
The PP validation leader collect all the validation files (MC, CS and DIST files), verify the consistency of the results and evaluate the monthly common statistical results
Some references on validation
Further information about validation technique, ground data quality calculation, ground data interpolation and upscaling, and unified software can be found in:
S. Puca et al., The validation service of the Hydrological SAF geostationary and polar
satellite precipitation products, NHESS, 2012 (submitted paper).
A. Rinollo et al., A common protocol for the validation of satellite rainfall estimations using radar data over the European territory, NHESS, 2012 (submitted paper). S. Puca et al., The Hydrological SAF validation service of geostationary and polar products,
Proc. EUMETSAT conference, 2012 (in progress).
A. Rinollo et al., A quality index for radar‐based rainfall estimation and the impact of its introduction on the validation of H‐SAF satellite precipitation products, Proc. EUMETSAT conference, 2012 (in progress).
Example of H‐01 product
Results for H‐01
H‐01 RMSE
2
0
‐2
‐4
‐6
‐8
‐10
‐12
‐14
‐16
‐18
350
300
250
200
> 10 mm/h
1‐10 mm/h
< 1 mm/h
> 10 mm/h
%
H01‐reference [mm/h]
H‐01 ME
150
1‐10 mm/h
100
< 1 mm/h
50
0
Spring Summer Autumn Winter 2009
2009
2009 2009/10
2009
2009 2009/10
1
0.8
0.6
0.4
POD
0.2
FAR
0
Spring
2009
Summer Autumn
2009
2009
Winter
2009/10
Results for H‐01
2
0
‐2
‐4
‐6
‐8
‐10
‐12
‐14
‐16
> 10 mm/h
1‐10 mm/h
< 1 mm/h
ME Rain Gauges
Spring Summer Autumn Winter
H01‐ refernce mm/h
ME Radars
2
0
‐2
‐4
‐6
‐8
‐10
‐12
‐14
‐16
‐18
‐20
> 10 mm/h
1‐10 mm/h
< 1 mm/h
Summary H‐01
Variability with season
• For heavy and medium precipitation the performance are rather similar across seasons. • For light precipitation summer is substantially worse, and autumn better.
• The FAR is rather high in all seasons, whereas the POD is better in summer and worse in winter.
Variability with precipitation type (or intensity)
• Heavy and medium precipitation have very similar performances through all seasons and geographical areas.
• For light precipitation there is a substantial degradation in spring and winter, and very substantial in summer.
Results for H‐02
H‐02 ME
H‐02 RMSE
0
350
300
‐4
250
‐6
200
‐8
> 10 mm/h
‐10
150
1‐10 mm/h
‐12
1‐10 mm/h
100
< 1 mm/h
‐14
> 10 mm/h
%
‐2
< 1 mm/h
50
‐16
0
‐18
2009
2009 2009/10
2009
2009 2009/10
H‐02
0.6
0.4
0.2
POD
0
FAR
Spring
2009
Summer
2009
Autumn
2009
Winter
2009/10
Results for H‐02
ME Rain Gauges
ME Radars
H02‐ refernce mm/h
0
‐2
0
‐2
‐4
‐6
‐8
‐10
‐12
‐14
‐16
‐18
> 10 mm/h
1‐10 mm/h
< 1 mm/h
‐4
‐6
> 10 mm/h
‐8
1‐10 mm/h
< 1 mm/h
‐10
‐12
‐14
Summary H‐02
• For heavy and medium precipitation the performance are rather similar across seasons.
• For light precipitation autumn is substantially worse, and winter better. • The FAR is rather high in all seasons, whereas the POD is better in summer and worse in winter. In summer POD is higher than FAR.
• For light precipitation there is a substantial degradation in spring and summer, and very substantial in autumn.
Results for H‐03
H‐03 ME
H‐03 RMSE
0
300
‐4
250
‐6
200
‐8
> 10 mm/h
‐10
%
‐2
> 10 mm/h
150
1‐10 mm/h
‐12
‐14
1‐10 mm/h
100
< 1 mm/h
< 1 mm/h
50
‐16
‐18
0
2009
2009 2009/10
2009
2009 2009/10
H‐03
0,8
0,6
0,4
POD
0,2
FAR
0
Spring 2009
Summer Autumn Winter 2009
2009
2009/10
Results for H‐03
ME Rain Gauges
H03‐ refernce [mm/h]
ME Radars
0
‐2
‐4
‐6
‐8
‐10
‐12
‐14
‐16
‐18
0
‐2
‐4
‐6
‐8
‐10
‐12
‐14
‐16
‐18
> 10 mm/h
1‐10 mm/h
< 1 mm/h
> 10 mm/h
1‐10 mm/h
< 1 mm/h
Summary H‐03
• For heavy and medium precipitation the performance are rather similar across seasons
• For light precipitation summer is substantially worse, and winter better. • The FAR is rather high in all seasons, whereas the POD is better in summer and worse in winter.
• For light precipitation there is a substantial degradation in spring and autumn, and very substantial in summer.
Example of H‐05 3h product
Example of H‐05 24h product
Results for H‐05
Results for H‐05
Summary H‐05
• The performance are rather similar across seasons, however summer is better and winter worse. • The FAR is rather high in all seasons, whereas the POD is better in summer.
• POD values are higher for 24 h accumulated product than for 3h one
Conclusions
Variability with geographical area
It may be observed that the performances are rather consistent across the various geographical areas, especially for heavy (> 10 mm/h) and medium (1‐10 mm/h) precipitation. This has been favoured by the adoption of a common validation methodology across the various participating Institutes. Even for inner lands and coastal areas the performance are rather similar.
Variability with validation tool
The performances resulting from validation by radar and those by rain gauges are rather similar. This is very important because User should not mind about which tool has been used for the validation: the information on the performance is regarded as a property of the product, not of the ground truth.
Cases Study
 High RG measurements connected with heavy precipitation.
 Days when H01 and H02 products were available more or less at the same time (max time span ‐ 30 min);
11 May 2009 ‐ convective precipitation on the frontline moving across Poland.
The meteorological situation resulted in heavy convective precipitation that occurred in the afternoon, at the South of Poland. The 6 hour cumulated precipitation measured at the SYNOP stations exceeded 60 mm.
11 May 2009
11 May 2009
11 May 2009
11 May 2009
11 May 2009
11 May 2009
70
60
H‐03 rain rate [mm/h]
50
40
30
20
10
0
0
10
20
30
40
50
60
70
RG rain rate [mm/h]
11 May 2009
H‐01 [mm/h]
H‐02 [mm/h]
H‐03 [mm/h]
Max RG
45.6
20.7
79.2
Max SAT
51.5
15.0
28.9
Mean RG
3.6
2.5
3.3
Mean SAT
4.7
1.8
2.4
ME
1.1
‐0.7
‐1.0
St.Dev
5.9
3.2
5.9
RMSE
6.0
3.3
6.0
RMSE %
4.6
2.3
3.3
Parameter
11 May 2009
25
H03
(49.7 N; 19.41 E)
RG
Rain rate [mm/h]
20
15
10
5
00:12
00:57
01:42
02:27
03:12
03:57
04:42
05:27
06:12
06:57
07:42
08:27
09:12
09:57
10:42
11:27
12:12
12:57
13:42
14:27
15:12
15:57
16:42
17:27
18:12
18:57
19:42
20:27
21:12
21:57
22:42
23:27
0
Time UTC
11 May 2009
Temporal variability of RG and H‐
03 rain rates (selected posts)
Case study ‐ conclusions
 H‐02 and H‐03 tends to underestimate high precipitation values while for H‐01 over‐
estimation has been found.  The quality of H‐SAF rain rate products in convective precipitation intensity estimation found for those cases is very good: the Mean Error varies from ‐1.0 mm/h to 1.1 mm/h and RMSE is equal 4.6%, 2.3% and 3.3% respectively for H‐01, H‐02 and H‐03. The best results were obtained for H‐02.
Conclusions
 The ability of H‐SAF rain rate products in convective precipitation recognition was found to be very good for H‐01 and quite good for H‐02. For H‐03, values of POD and FAR are the same. It should be also mentioned that ACC values are very high for all products.
 The spatial distribution of convective precipitation was well described by H‐01 and H‐
02 products, however, the size of precipitation area was slightly overestimated. On the other hand, the maximum values of rainfall were properly localised by H‐01.
 The H‐03 product seems to be too rainy ‐ the precipitation area was significantly overestimated. Moreover, distribution of rainfall intensity was found to be too homogeneous and the spots with heavy rainfall were missed. These features can be also seen in the results obtained for the analysis of temporal variability of rain rate performed for selected H‐03 pixels
Thank you for your attention!

Validation of H-SAF Precipitation Products

Transcription

Similar documents

- Aspire4Sport

ZAGROŻENIA OKRESOWE WYSTĘPUJĄCE W POLSCE

REGIONAL VARIATION IN EXTREME RAINFALL VALUES