Climate simulations - T

Transcription

DLR.de • Chart 1 > Challenges of high-resolution climate simulations > Bastian Kern • HPCN Workshop, Göttingen > 10.05.2016
A diagnostic interface for ICON –
Coping with the challenges of high-resolution climate
simulations
Bastian Kern
Deutsches Zentrum für Luft- und Raumfahrt
Institut für Physik der Atmosphäre
Oberpfaffenhofen
Weather – Climate?
Screenshot, 26.04.2016:
http://www.wetter.de/klima/europa/deutschland/
muenchen-s99000036/april.html
Overview
Numerical models
numerical weather prediction
climate simulation
Challenges (and possible answers)
Interface for on-line diagnostics
implementation, performance
Outlook
The first weather forecast
1904: “Das Problem der Wettervorhersage,
betrachtet vom Standpunkte der Mechanik und
der Physik”, Met. Z.
Necessary and sufficient conditions for solving
the forecast problem:
• Know with sufficient accuracy the (initial)
state of the atmosphere
• Know with sufficient accuracy the (physical)
laws determining the evolution of the
atmosphere
 One diagnostic and one prognostic step
Suggests a graphical calculation method.
Gaining higher accuracy using smaller fixed
time intervals, while higher calculation costs.
http://www.nb.no/ (blds_01075)
1922: “Weather Prediction by Numerical
Process”
Calculation of initial pressure tendency from
full partial differential equations.
Took two years by hand to calculate for one
point (in the centre of the calculation region).
“Forecast”: p = 145 hPa in 6h
Lewis Fry Richardson by Walter
Stoneman, NPG x164373
© National Portrait Gallery, London,
CC BY-NC-ND 3.0
Process”
Level
LFR
Model
filtered
1
48.3
48.5
-0.2
2
77.0
76.7
-2.6
3
103.2
102.1
-3.0
4
126.5
124.5
-3.1
Surface
145.1
145.4
-0.9
P. Lynch, Journal of
Computational Physics, 2008,
Table 1
Process”
“Forecast Factory”: 64000 human computers
calculating weather forecasts in a big “theatre”
 first massive parallel computer system
Lewis Fry Richardson by Walter
Stoneman, NPG x164373
© National Portrait Gallery, London,
CC BY-NC-ND 3.0
1946 proposal: “Weather prediction was, par
excellence, a scientific problem suitable for
solution using a large computer.”
John von Neumann
LANL (http://www.lanl.gov/)
Courant, Friedrichs, Lewy, Math. Annalen, 1928
John von Neumann
1946 proposal: “Weather prediction was, par
excellence, a scientific problem suitable for
solution using a large computer.”
Charney leader of meteorological group
• unfiltered primitive equations
 Richardson
• baroclinic quasi-geostrophic system
 computationally too demanding
• barotropic vorticity equation
 satisfactory initial results
1950: first weather forecast on ENIAC.
5 scientist plus 3 operators
5 weeks
2 satisfactory “forecasts”
one 24h “forecast” took 24h
John von Neumann
ENIAC (U.S. Army Photo)
John von Neumann
MIT Museum
MIT Museum
http://www.nco.ncep.noaa.gov/sib/verification
Evolution in weather prediction
http://www.ecmwf.int/en/forecasts/charts/medium/
anomaly-correlation-ecmwf-500hpa-height-forecasts
Evolution in computing power
“PHONIAC”
ENIAC integrations as
Java application on a
Nokia 6300
ENIAC: 24h
PONIAC: <1s
Lynch & Lynch, Weather, 2008
CRAY-1:
~250 MFLOPS (vector)
Nokia 6300:
~237 MFLOPS
ENIAC:
140 kW
CRAY-1:
115 kW
Nokia 6300: 1.5 W
Challenges in weather prediction
Atmosphere as chaotic system
Slight deviations in initial
conditions can cause large
differences in forecast
https://commons.wikimedia.org/wiki/
User:Dschwen, CC BY 2.5
Challenges in weather prediction
Atmosphere as chaotic system
Slight deviations in initial
conditions can cause large
differences in forecast
Sophisticated methods for initial
conditions
Ensemble forecasts
http://www.daserste.de/wetter (26.04.2016)
Weather forecast  Climate projection?
Weather forecast is an initial value problem
Deterministic chaotic system
 boundary for meaningful forecast about 14 days
Climate models are essentially numerical weather prediction models
(some changes)
(some changes)
How can we “forecast” climate in 2100?
(some changes)
How can we “forecast” climate in 2100?
We can do climate projections
 statistical analyses
Climate projection is boundary value problem
Model produces realistic weather systems from internal variability
Numerical Weather Prediction
Climate projection
http://www.daserste.de/wetter (26.04.2016)
http://www.ipcc.ch/report/ar5/syr/
Climate simulations
https://www.dkrz.de/about/media/galerie/Media-DKRZ/hlre-3
Climate simulations
„Mistral - Phase 2“ (spring 2016)
~ 68.000 cores
3 PetaFLOPS
memory: ~200 Terabyte
disks:
50 Petabyte (Lustre)
Top 500, November 2015
http://www.top500.org/list/2015/11/
Climate simulations – Challenges
“With the HLRE-II system "Blizzard", about
8 Petabytes of new simulation results had
to be archived per year. Starting with the
availability of the new HLRE-3 "Mistral" in
summer 2015, we expect the yearly data
growth to increase to up to 75 Petabytes
per year.” (www.dkrz.de)
1) simulation
2) data output
memory
3) postprocessing
(subset)
3b) postprocessing
(reduced)
3a) postprocessing
disk
(scratch/work)
archive (tape)
4) archiving
1) simulation
2) data output
memory
5) postprocessing
(subset)
5b) postprocessing
(reduced)
5a) postprocessing
3) archiving
disk
(scratch/work)
4) get data
(subset)
archive (tape)
6) archiving
(reduced)
1) simulation
2) data output
memory
5) postprocessing
(subset)
3) archiving
disk
(scratch/work)
5b) postprocessing
(reduced)
5a) postprocessing Bottleneck 1: ~400 GB/s
4) get data
(subset)
archive (tape)
6) archiving
Bottleneck 2: ~20 GB/s
1) simulation
2) data output
(reduced)
memory
1a) onlinediagnostics
3) archiving
(reduced)
disk
(scratch/work)
archive (tape)
ICON – ICOsahedral Non-hydrostatic modelling framework
Joint development of Deutscher Wetterdienst (DWD) and Max Planck
Institut für Meteorologie (MPI-M) for numerical weather prediction and
climate applications
Important development goals
• Improved conservation properties compared to GME and ECHAM
• Non-hydrostatic dynamical core for application on all scales
• Variable grid resolution (two-way-nesting)
• Application as global and regional model
• Improved performance on parallel computers (compared to GME /
ECHAM)
ICON – Grid structure
G. Zängl, DWD
ICON – Grid structure and refinement
G. Zängl, DWD
HD(CP)² - High Definition Clouds and Precipitation for
advancing Climate Prediction
HD(CP)² project:
• High requirements to hardware
 High spatial resolution (Limited Area at 100m ~ 56 M
grid cells + outer nesting domains)
 Small integration time-step
• Scalability on O(105) cores
 Communication and memory handling
ICON:
• Unstructured grid and parallel decomposition
• Nesting of multiple domains
Figure: https://code.zmaw.de/projects/icon
HD(CP)² - Model Diagnostics (subproject)
• HD(CP)² Phase I milestones:
• Diagnostic Interface in ICON
• Prototype implementation of advanced diagnostic tool
• Implementation in ICON:
• Modular Earth Submodel System (MESSy)
• #IFDEF MESSY …
MESSy – Modular Earth Submodel System
P. Jöckel
• Prototype advanced diagnostic tool GRAGG (Grid AGGregation)
• “Aggregation on user defined regular grids”
 examples: spatial mean, joint-PDFs
GRid AGGregation (GRAGG)
• Init-Phase: Grid cell indices
CPU1: 10,11,12,17,19
CPU2: CPU3: 17, 23
CPU4: 1,2,3,4,12,13,14
CPU5: • Integration time-step: Partial results per CPU
• Output time-step: (MPI-) All-Reduce
• Problems:
• Min/max in integration time-step
Inter-CPU communication
• “Hybrid” parallelisation (MPI + OpenMP)
17
23
11 12
19
1
4
3
2
12 13 14
10
17
20
5
15
6
16
GRAGG
ICON 3D
Spatial mean on user-defined grid
Diff.
GRAGG
ICON 3D
jointPDF in highlighted grid cell
Diff.
• Prototype advanced diagnostic tool GRAGG (Grid AGGregation)
• “Aggregation on user defined regular grids”
 examples: spatial mean, joint-PDFs
• VIsual Satellite OPerator (VISOP), developed at LMU
 example
VIsual Satellite OPerator (VISOP)
• Leonhard Scheck et al., Jour. Quant. Spect. and Rad. Transfer, 2016
800nm MODIS simulation, “inner nest” @ 312m, 26.04.2013, 12 UTC
approx. 3-D
calculated from VISOP data (2D fields)
3-D
calculated from ICON output (3D fields)
Performance test on “Mistral”
ICON, 3-domains, std. ICON output,
15 min. diagnostic submodel output
std.
optimised
• Diagnostic Interface: no overhead
• VISOP (optimised): max. +2.72%
(column based)
• GRAGG (optimised): max. +60.79%
(high MPI communication)
Kern & Jöckel, GMDD, in prep.
Outlook
• Further optimisation of prototype diagnostic tool
• Memory footprint
• MPI communication
• Parallel output of diagnostic interface
Outlook
• Further optimisation of prototype diagnostic tool
• Memory footprint
• MPI communication
• Parallel output of diagnostic interface
• HD(CP)² Phase II (1 April 2016 – 30 March 2019)
• Advanced on-line diagnostics
• On-line feature identification, feature tracking,
and combination with trajectories
Summary
• First weather forecast
• Numerical weather prediction and climate simulations
• Challenges of high resolution simulations
 bottlenecks
• ICON as new tool for NWP and climate applications
• Diagnostic interface based on MESSy
• Examples of diagnostic tools
• Performance
• Outlook

Climate simulations - T

Transcription

Similar documents

Minutes of the Meeting of Creditors