MUSCATE: multi-satellites, multi-sensors and multi

2016 CONFERENCE ON BIG DATA FROM SPACE
MUSCATE:
MULTI-SATELLITES, MULTI-SENSORS AND MULTITEMPORAL THEIA INFRASTRUCTURE
Simon BAILLARIN
CNES – Toulouse, France
On behalf of main authors: J. DONADIEU, C L’HELGUEN
1
WHAT IS THEIA ?
• THEIA: French national multi-agency organisation for
the studies of continental surfaces
• THEIA promotes the use of satellite data by scientific
community and institutional actors.
2
MUSCATE OVERVIEW
MUSCATE = THEIA processing facility of HR optical satellite products
3 Families of Satellites:
•
•
•
SPOT 1 to 5: Spot World Heritage project
LANDSAT 5-7-8
SENTINEL-2 (A & B)
Already more than
30 years of data !
MUSCATE OVERVIEW
MUSCATE = THEIA processing facility of HR optical satellite products
3 Families of Satellites:
•
•
•
SPOT 1 to 5: Spot World Heritage project
LANDSAT 5-7-8
SENTINEL-2 (A & B)
3 Types of Products (CEOS standard):
•
•
•
Level 1C: ortho-rectified product in TOA reflectance
Level 2A: level 1C product in surface reflectance
Level 3A : temporal synthesis of level 2A products
Level
1A
1G
1T
1C
2A
SPOT
LANDSAT L1G
LANDSAT L1T
SENTINEL 2
4
MUSCATE
products
Level distributed
1C
Level 2A
Level 3A
3A
MUSCATE ARCHITECTURE
SPOT,
LANDSAT,
SENTINEL2 DATA
Administrators
Production facility
Acquisition
facility
Exchange
space
Data
Catalog
Processing
Orchestrator
(PHOEBUS)
Distribution facility
5
THEIA shared services - Website
Archiving
MUSCATE ARCHITECTURE
SENTINEL2
Production facility
Acquisition
facility
Exchange
space
Catalog HMI
PHOEBUS HMI
Administrator /
Data
Catalog
Catalog
PHOEBUS
Processing
orchestrator
Orchestrator
(PHOEBUS)
Catalog
data base
Processing on
Cluster
Distribution facility
6
THEIA shared services - Website
Archiving
MUSCATE: PRODUCTION FACILITY
EXAMPLE OF SENTINEL 2 PROCESSING
MACCS algorithm (multi-temporal) :
• Clouds (and shadows) detection,
• Aerosol Optical Thickness (AOT) & Water Vapour estimation,
• Atmospheric effects Correction
MUSCATE: PRODUCTION FACILITY
EXAMPLE OF SENTINEL 2 PROCESSING
MACCS algorithm (multi-temporal) :
• Clouds (and shadows) detection,
• Aerosol Optical Thickness (AOT) & Water Vapour estimation,
• Atmospheric effects Correction
Initialisation phase (backward)
t0
t0
t0 + 5
t0 + 10
t0 + 15
t0 + 20
t0 + 25
Level 1C
t0 + 5
t0 + 10
t0 + 15
t0 + 20
t0 + 25
t0
Level 2A
MUSCATE: PRODUCTION FACILITY
EXAMPLE OF SENTINEL 2 PROCESSING
MACCS algorithm (multi-temporal) :
• Clouds (and shadows) detection,
• Aerosol Optical Thickness (AOT) & Water Vapour estimation,
• Atmospheric effects Correction
Initialisation phase
t0
Nominal phase (recursive)
Level 2A
t0 + 5
t0
Level 1C
9
t0 + 5
Level 2A
Level 2A
MUSCATE: PRODUCTION FACILITY
EXAMPLE OF SENTINEL 2 PROCESSING
Retrieve of Level-1C
products on PEPS platform
and ingestion in MUSCATE
catalog
MACCS L2 processing
(atmo. correction)
Level 1C
Level 2A
Dispatch to
MUSCATE
distribution
facility
Level-2A
already avail.
Level-1C
T1
L2 processing T3
T2
Level-2A
MUSCATE Catalog
LEGENDE
REAL
MUSCATE catalog database
PHOEBUS
End of
of
Processing
PHOEBUS
End
Processing
pooling
processing
pooling
start
processing
start
PHOEBUS
(tasks list with all available
inputs)
PREDICTED – available inputs
PREDICTED – unavailable inputs
10
Archiving
HPC Cluster
MUSCATE PROCESSING CAPACITY
MUSCATE : an Ambitious Project …
•
•
•
Automatically process up to 2000 products a day (SPOT / LANDSAT /
SENTINEL-2)
Offer reprocessing capabilities
Cumulative Volume of MUSCATE data ~1 PB
1000
500
SENTINEL-2A
SENTINEL-2B
Volume in To
750
~1 PetaByte in 2024
Input
Output
Total
250
0
2014
2016
2018
2020
Years
11
2022
2024
2026
MUSCATE PROCESSING CAPACITY
MUSCATE : an Ambitious Project …
•
Automatically process up to 2000 products a day (SPOT / LANDSAT /
SENTINEL-2)
Offer reprocessing capabilities
Cumulative Volume of MUSCATE data ~1 PB
•
•
… with “limited budget”!
•
Use of CNES HPC centre (resources shared among projects)
•
•
•
Extensive re-use of CNES software components:
•
•
12
150 computing nodes
Common storage means
PHOEBUS (processing orchestration framework),
SIGMA (ortho-rectification), and MACCS (conversion in surface reflectance)
processors
OUTLOOK AND CHALLENGES
Host of new scientific processing chains
 Snow detection from Level-2A
 Temporal
13
synthesis & landcover maps
OUTLOOK AND CHALLENGES
Internet website: http://www.cesbio.ups-tlse.fr/multitemp/wp-content/uploads/2016/02/SudOuestMosaic_France2014_V1_ColorIndexedT.html
14
OUTLOOK AND CHALLENGES
Host of new scientific processing chains
 Snow detection from Level-2A
 Temporal
 Extend
15
synthesis & landcover maps
capabilities with new satellites, sensors and processing
CONCLUSION
MUSCATE Objectives:
• Deliver high quality products
• Offer a full coverage of France over 40 years by SPOT,
LANDSAT and SENTINEL-2 (including overseas).
• Distribute 10 times the surface of France with level 2A
SENTINEL-2 products
• Host and promote new algorithms developed by scientists
Already Available Datasets:
• Preliminary version of MUSCATE currently functional
(LANDSAT and SPOT products) :
http://www.theia-land.fr/en
Schedule:
• Full version of MUSCATE currently in qualification phase
• Beginning of operational production by summer 2016
16
Thank you
Main authors :
Simon BAILLARIN
[email protected]
Joëlle DONADIEU
[email protected]
Céline L’HELGUEN
[email protected]
Co-authors : M. LEROY, P. PACHOLCZYK, O. HAGOLLE, M. HUC, J. GASPERI, B. PETRUCCI, O. MELET,
M. PAVY, E. SARRAZIN, R. NGO, L. LEBEGUE, V. LONJOU, C. DESJARDINS, A.-C. GERARD, A. VIDOT,
D. CLESSE
17