AGILE Data Center

AGILE Data Center: Scientific Analysis
and First Year Overview
Carlotta Pittori
5th Science AGILE Workshop, ASDC 12-13 June 2008
 The
AGILE Data Center (ADC) is the scientific part of
the AGILE Ground Segment based a ASDC/ESRIN
 ADC is composed by:
• the AGILE Team Processing Group
(AGILE Team personnel)
• the AGILE Science Support Center
(ASDC personnel)
 ADC is in charge of all scientific activities related to
the analysis, archiving and distribution of AGILE data
AGILE Data Center
Science Support Center group at ASDC:
Paolo Giommi
ASDC Director
F. Tamburelli
(AGILE in calibrazione @ LNF)
A. Antonelli , G. Fanari, B. Preger, C. Pittori, F. Verrecchia, D. Gasparrini, F. Acerra, S. Stellato
M. E. Pennisi, S. Cutini, P. Santolamazza, R. Primavera (not in phot.)
(ASDC-INAF + DATASPAZIO)
AGILE orbital parameters
Baseline equatorial orbit: 550 Km, 3º inclination
Semi-major axis: 6922.5 km (± 0.1 km)
Requirement:
6928.0 ± 10 km
Inclination angle: 2.48° (±0.04°)
Requirement:
< 3°
Eccentricity:
Requirement:
0.002 (±0.0015)
< 0.1°
ASI Malindi Ground Station:
~ 10 minutes ground contact every ~ 90 min
ASINet: Malindi ⇒ TPZ Fucino ⇒
AGILE GS Architecture
S-Band
Fucino, Italy
Fucino, Italy
ASINet
Command, HK, Sci raw data
Fucino Gateway
Fucino, Italy
Satellite
Control
Center
Mission
Control
Center
Flight
Dynamics
Center
AGILE Operation Control Center
Command, HK, Sci raw data
TM L0 &
Aux Data
Long Term Plan
Obs Planning File
Kenya
ASI Malindi
Ground Station
ASINet
Fucino Gateway
Calibration,
SW for data analysis, ...
Frascati, Italy
AGILE
DATA
CENTER
@ASDC
AGILE Team
Processing Group
AGILE Science
Support Center at
ASDC
ASINet
ASINet
Data
Agile Pointing Program
APPC
Products
and SW
internet
internet
AGILE Data Center
Agile
Team
internet
internet
AO/GOP
Scientific
Community
AGILE data flow and Ground Segment organization (from Agile Science Management Plan)
AGILE Mission Board
ASI
Mission Director
ASI Science
Data Center
AGILE
Principal Investigator
Management
structure
Data flow
AGILE
Operation
& Control
Center
AGILE Team at
Scientific Institutes
AGILE Data Center (ADC)
at ASDC - ESRIN
AGILE Team
Processing Group
(ATPG)
(AOCC)
Malindi
Ground
Station
Scientific
Community
AGILE Science
Support Center
(ASSC)
Public
outreach
ADC scientific analysis and archiving
of AGILE data
From scientific telemetry (TM)
Level–0:
 Preprocessing → Level-1 data
 Quick-Look Analysis
 Standard analysis → Level-2 data
(photon list, logfiles)
 Scientific analysis (source
detection, diffuse gamma-ray
background)
 Archiving and distributing all
scientific AGILE data
INPUT: Row data
(TM Level-0)
Preprocessing:
Level-1 data
Primary data generation: Quicklook &
Standard analysis Level-2 data
(photon list and logfile)
Scientific analysis:
Level-3 data
OUTPUT: High level data products
(count maps, spectra, light curves…)
Summary of ASDC activities for AGILE:
(from Agile Science Management Plan)
•
•
•
•
•
•
•
•
•
Running the Quick Look Analysis
Running the standard data reduction Analysis
Performing, when necessary, the Interactive data Analysis
Managing Announcement of Opportunities
Contributing to the management of the AGILE Pointing Program
Archiving all the data (raw, cleaned and calibrated, scientific)
Distributing the data to the scientific community
Providing scientific support to the users community
Officially interface the project for both data and proposals via
dedicated web pages
•
Providing the standard software support for the data analysis
AGILE Data Center
Long Term Plan/Obs Planning File
Data Analysis
System
TM, L0,
Aux Data
Mission Planning
System
L1, L2, L3
Products,
SW Development
and Integration
System
AOCC
TM, L0,
Aux Data
ASINet
ASINet
Agile Pointing Program
APPC
Calibration,
SW for data analysis
AGILE
Team
Data
analysis
SW
Data (Aux, L0,L1, L1cor)
Data Archive
System
Data and SW Distribution
System
internet
internet
Products, Catalogues,
SW for Data Analysis
AO/Guest Observer Program
Agile Data Center
Scientific
Community
GOP
AO
ADC operation scheme:
2.
Data Processing
&
Archiving
1.
GS Activity
&
MCC Interface
1.1
Mission
Planning
1.2
Data Flow
Monitoring
APP Manag
(& OB status)
LTP/OPF
Nominal
1.3
Contingency
Management
2.1
Automatic Pipeline
Monitoring
TM LV0
TM PPS Branch
(LV0 → LV1)
OSM Manag
Quick Scientific
Monitoring
SuperAGILE
pipeline
5.
Payload
Configuration
Control
2.2
Archive
Management and
Data Backup
AUX Branch
Auxiliary
TOO/PRR
3.
AO/GOP
GRB Alert
Manag
Correction
(LV0 → LV1corr)
GRID QL:
• QLSTD (ΔT bricks
LV1corr⇒ LV2)
• QLSCI_V & QLSCI_I
GRID Standard Analysis
on OB
(LV2 → LV2_cl & LV3)
AO Data Request
Manag
2.3
Interactive Data
Analysis
GO Data & SW
Distribution
GRID QLUSR
Calibration
HM Alert
Manag.
CALDB
Updates
OB Bricks
Validation
GO Help Desk
4.
Public WEB Pages
&
User Support
Scientific Analysis
(LV3 refinement)
AGILE GRID & SA
Catalogs
On-line archive
in MMIA and VO
FAQ & Scientific
Help Desk
Documentation
AGILE Mission Planning:
• 30 Long Term Plans - 8 TOO/PRR
• 5860 orbits, June 11 2008 (~ 94% Fine Pointings)
• 17 Cycle-1 Observation Blocks completed from Dec 1 to May 10
AGILE data analysis SW Structure & Data Flow:
AOCC
PREPROCESSING
PROPOSAL
MANAGEMENT
P/L CONFIGUR.
CONTROL
Level-0 TM data
(binary)
USERS
PLANNING
Level 1
(TM in FITS)
CORRECTION
Corrected
Data
AUX. DATA
RECEIPT &
PROCESSING
Data pre-processing
DATA DISTRIBUTION
HEALTH
MONITORING
INTERACTIVE
ANALYSIS
Primary
Data
STANDARD
ANALYSIS
QUICK LOOK
ANALYSIS
CALIBRATION &
RESPONSE ANALYSIS
& Correction
Calibration
Results
Ground
Calibrations
Archive
Level-1 cor TM
data (FITS)
High level
data products
SCIENTIFIC
ANALYSIS
GRID QLA
SuperAGILE QLA
(Δt ~ few orbits)
(orbit)
QLA results available
to the community
Level-2 QL data
GRID Standard
Analysis
(Observation Block)
SuperAGILE Standard
Analysis
Standard Level-2 data
(GRID photon list + logfile)
and Level-3 data
(imaging/spectral/timing)
GOP
SuperAGILE standard
monitoring results available to
the community
AGILE SW structure & Data Flow
Raw TM from MCC
Auxiliary Data from MCC
LV0 Data
Auxiliary Data
Telemetry Preprocessing
Auxiliary Data
Processing
LV0 + LV1 Data
LV1 Data
LV1corr Data
Data Correction
LV1corr Data
Standard Analysis
LV2 Data
Auxiliary Data
AGILE Mission
Archive
LV2 Data
Scientific Analysis
LV1,2 Data
LV2,3 Data
User Data
Processing
LV2_CL & LV3 Data
AgileDataCenter@ASDC
AGILE Scientific Data Archive
AGILE Archive at ASDC includes:
• data from LV0 up to LV3
• storage area for auxiliary data
Archive root
LV0
One directory
for each contact
LV1
One directory
for each contact
LV1corr
One directory
for each contact
qlstd
One directory
for each run
qlsci
One directory
for each run
user
One directory
for each run
auxiliary
One directory
for each file type
AGILE Data Archive Size
First 13 months:
LV0 LV1 and LV1corr archives are updated on contact basis.
For each contact, ~ 14MB of LV0 Data is delivered to ASDC
 62 GB LV0 data archived up to now
 165 GB LV1 data archived up to now
 175 GB LV1corr data archived up to now
QL Standard Analysis is executed on a basis of 3 hours time slots
 24 GB of QL LV2 data archived up to now
Scientific Analysis is executed on regular time basis (i.e. once, twice a day, and on
Observation Blocks). User analysis is asynchronous
 25 GB of LV2 and LV3 data archived up to now
Auxiliary data are received on different time slot basis (contact, once a day, twice a
week…)
 6 GB of Auxiliary data archived up to now
 Estimate for next years: ~ 1 TB/year
← AGILE homepage at ASDC
AGILE Services:
ADC internal web I/F to AGILE archive and DataBase:
2.
Data Processing
&
Archiving
1.
GS Activity
&
MCC Interface
Processing and Data Management @ ASDC:
Automatic data
processing
pipelines
Interactive data
analysis sessions
Shell script
export tool
Link to data
and proc. logs
Automatic archiving, I/F data access and visualization (Java Applet)
Commissioning
Phase:
First GRID light
AGILE Vela PSR Count
Map by ADC, 24/5/2007
(~ 20000 s)
The EGRET gamma-ray sky (E > 100 MeV)
9 years: 1991 - 2000
AGILE 9 months COUNT MAP
(July 2007- April 2008)
AGILE 9 months EXPOSURE MAP
(July 2007- April 2008)
ASDC multimission tools
⇒ 15 Astronomer’s
Telegrams (ATel)
⇒ 8 GCN Circulars
(GRB Coordinates Network)
SuperAGILE non-GRB sources detected between
Nov 2007 and Jan 2008
AGILE GRID Quick Look Test Catalogue:
under construction
First AGILE GRID data reprocessing with in-flight calibrations
and available software updates: completed on April 9th.
Second reprocessing with updated software and calibrations to be started next month.
AGILE First Consolidated Source Catalogue:
to be published within October 2008
A. Chen
Scientific program open to the community: Cycle-1 Guest
Observer Program. GO data taking started on Dec. 1, 2007
AGILE AO1:
Submitted proposals: 29
GOP Schedule
• SW build GO 1.0 + test dataset:
released on May 22, 2008
Approved/P. Approved: 24 • GO data validation: on-going
Requested Targets: 122
• GO data distribution: started on June 5
Approved Targets: 100
Pulsars: 39
AGN: 31
3EG sources: 30