EGI - ITI

Transcription

EGI - ITI

Towards a Sustainable
Federated Grid Infrastructure
for Science
Dieter Kranzlmüller
Ludwig-Maximilians-Universität (LMU) München &
Leibniz Supercomputing Centre (LRZ)
[email protected]
267 Sites
54 Countries
114,000 CPUs
20 PetaBytes
>16000 Users
>200 VOs
>140,000 Jobs/Day
Archeology
Astronomy
Astrophysics
Civil Protection
Comp. Chemistry
Earth Sciences
Finance
Fusion
Geophysics
High Energy Physics
Life Sciences
Multimedia
Material Sciences
…
ITI 2009, Cavtat, Croatia
Status January 2009:
http://project.eu-egee.org/index.php?id=417
2
Pilotproject EGEE:
High Energy Physics
• Large Hadron Collider (LHC)
– Amount of data: ~4 PBytes/year
• EGEE Tasks:
– Data management
– Distributed analysis
(globally)
– Batch processing
3
Biomedicine:
WISDOM
• World-wide In Silico Docking On Malaria
– Neglected diseases
– Biomedical data challenges
– Docking experiments:
46 million ligands
– Runtime: 6 weeks
– Data: 1 TByte
– 1000 CPUs in 15 countries
(~80 CPU years)
4
267 Sites
54 Countries
114,000 CPUs
20 PetaBytes
>16000 Users
>200 VOs
>140,000 Jobs/Day
Archeology
Astronomy
Astrophysics
Civil Protection
Comp. Chemistry
Earth Sciences
Finance
Fusion
Geophysics
High Energy Physics
Life Sciences
Multimedia
Material Sciences
…
Production Grids are reality today
but …
Status January 2009:
http://project.eu-egee.org/index.php?id=417
5
EGEE Projects
•
EGEE
– 01 April 2004 – 31 March 2006
– 71 Partners, 27 Countries
– EC Contribution: 32 Mio €
•
EGEE-II
– 01 April 2006 – 30 April 2008
– 91 Partners (11 JRU – 48 partners), 32 Countries
The world’s largest production
•
EGEE-III
– 01 May 2008 – 30 April 2010
– 42 Partners (mostly JRUs)
grid infrastructure alone requires
> 16 Mio € EU funding
+ add. national funding per year
This does not include the
hardware resources
6
Today’s Challenges
• Project-based funding: Typical funding cycles
of today’s grid infrastructures => 2-4 years
• Protection of Investment: Investments in grids,
both from funding organizations and from users,
need to be protected
• Dependency: Some application domains
depend on production grids already today
• Long-term perspective: Grid users ask for a
longer term perspective
7
Grid Stakeholders Today
on
on
Grids
Grids
International
InternationalScientific
Scientificand
andResearch
ResearchCollaboration
Collaboration
European
European
Commission
Commission
Funding
Funding
Agency
Agency11
Funding
Funding
Agency
Agency22
…
Funding
Funding
Agency
AgencyNN
8
International
Scientificand
andResearch
Collaboration
European
European
Projects
Projects
European
European
Commission
Commission
National
National
Grid
Grid
Projects
Projects
Regional
Regional
Grid
Grid
Institution
Institution
…
Local
Local
Grid
Grid
Community
Community
Funding
Funding
Agency
Agency11
Funding
Funding
Agency
Agency22
…
Funding
Funding
Agency
AgencyNN
9
Grid Users Today
• Dependency
• Protection of Investment
• Long-term Perspective
European
European
Projects
Projects
European
European
Commission
Commission
National
National
Grid
Grid
Projects
Projects
Regional
Regional
Grid
Grid
Institution
Institution
…
Local
Local
Grid
Grid
Community
Community
Funding
Funding
Agency
Agency11
Funding
Funding
Agency
Agency22
…
Funding
Funding
Agency
AgencyNN
10
e-Infrastructure Reflection Group
(e-IRG)
International
Scientificand
andResearch
Collaboration
e-IRG
e-IRGRecommendation,
Recommendation,12/2005:
12/2005:
European
European
Projects
Projects
European
European
Commission
Commission
National
National
Grid
Grid
Projects
Projects
Funding
Funding
Agency
Agency11
“The
“Thee-IRG
e-IRGrecognizes
recognizesthat
thatthe
thecurrent
current
Regional
Local
project-based
financing
model
of
grids
Regional
Local
project-based
financing
model
of
grids
…
Grid
Grid
(e.g.,
EGEE,
Grid
Grid continuity
(e.g.,
EGEE,DEISA)
DEISA)presents
presents
continuity
Institution
Community
Institution
Community
and
interoperability
and
interoperabilityproblems,
problems,
and
andthat
thatnew
newfinancing
financingand
andgovernance
governance
models
modelsneed
needto
tobe
beexplored
explored––taking
takinginto
into
account
the
grid
account
therole
roleof
ofnational
national
gridinitiatives
initiatives
Funding
Funding
Funding
Funding
… ininthe
as
recommended
Luxembourg
e-IRG
asAgency
recommended
the
Luxembourg
e-IRG
2
Agency
N
Agency
2
Agency
N
meeting.”
meeting.”
11
Grids in Europe
www.eu-egi.eu
Each NGI
… should be a recognized national body
with a single point-of-contact
… should mobilize national funding and resources
… should ensure the operation of a national e-Infrastructure
… should support user communities
… should contribute and adhere to intl. standards and policies
12
National
Global
Testbeds
Evolution
EGI is not a simple continuation
of EGEE, but should offer a
possible solution for all
European Grid infrastructure
Routine Usage
Utility Service
13
projects
to achieve sustainability
European Grid Initiative
Goal:
• Ensure the long-term sustainability of grid infrastructures
in Europe by establishing a new federated model
bringing together NGIs to build the EGI Organization
Objectives:
• Ensure the long-term sustainability of the European einfrastructure
• Coordinate the integration and interaction between
National Grid Infrastructures
• Operate the European level of the production Grid
infrastructure for a wide range of scientific disciplines to
link National Grid Infrastructures
14
•
•
Ensure the long-term sustainability of the European e-infrastructure
Coordinate the integration and interaction between National Grid
Infrastructures
• Operate the European level of the production Grid infrastructure for a wide
range of scientific disciplines to link National Grid Infrastructures
• Provide global services and support that complement and/or coordinate
national services (Authentication, VO-support, security, etc);
• Coordinate middleware development and standardization to enhance the
infrastructure by soliciting targeted developments from leading EU and
National Grid middleware development projects;
• Advise National and European Funding Agencies in establishing their
programmes for future software developments based on agreed user needs
EGI Vision Paper
and development standards;
• Integrate, test, validate and package software from leading Grid middleware
developmenthttp://www.eu-egi.eu/vision.pdf
projects and make it widely available;
• Provide documentation and training material for the middleware and
operations. (NGIs may wish to make the material available in turn in their
local language);
• Take into account developments made by national e-science projects which
were aimed at supporting diverse communities.
• Link the European infrastructure with similar infrastructures elsewhere;
• Promote Grid interface standards based on practical experience gained
from Grid operations and middleware integration activities, in consultation
with relevant standards organizations;
•ITI 2009,
Collaborate
closely with industry
technology and service providers, as 15
Cavtat, Croatia
Dieteras
Kranzlmüller
well as Grid users, to promote the rapid and successful uptake of Grid
technology by European industry.
42 European NGIs
+ Asia, US,
Latin America
+ OGF-Europe
+ PRACE
+…
16
EGI Policy Board
17
International
Scientificand
andResearch
Collaboration
European
European
Projects
Projects
European
European
Commission
Commission
National
National
Grid
Grid
Projects
Projects
Regional
Regional
Grid
Grid
Institution
Institution
…
Local
Local
Grid
Grid
Community
Community
Funding
Funding
Agency
Agency11
Funding
Funding
Agency
Agency22
…
Funding
Funding
Agency
AgencyNN
18
Future Grid Stakeholders
EGI
Design
Study
International
Scientificand
andResearch
Collaboration
European
European
Grid
Grid
Projects
Projects
European
European
Commission
Commission
National
National
Grid
Grid
Initiative
Initiative11
National
National
Grid
Grid
Initiative
Initiative22
…
National
National
Grid
Grid
Initiative
InitiativeNN
Goal of EGI Design Study:
Funding
Funding
Funding
Conceptual
setup …and operation
of a
Funding
Funding
Funding
Agency
Agency
Agency
Agency11
Agency22
AgencyNN
new organizational model of a
sustainable pan-European
19
grid infrastructure
EGI Webpage
www.eu-egi.eu
20
EGI Knowledge Base - Main
NGI
Representatives
to provide their
input
and update their
local information
http://knowledge.eu-egi.eu
21
EGI Knowledge Base Croatia
22
EGI Knowledge Base
23
EGI_DS Deliverables
• D4.3 Guidelines for NGIs ut
p
n
– How to set up an NGI
i
r
u
u
e
– Minimum requirements
i.NGIs
yo for
g
e -e .eu
d
i eu gi
v
.
e
o
r
w
• D4.4 EGI Blueprint
Proposal
p w eu
e //w @
s
t financing of EGI
– Structuraleaoutline
: and
c
p
Pl htt nta
co
• D3.1 EGI Functions Definition
– Detailled functional description of EGI
24
The EGI Model
EGI Model
• NGIs
– ensure the operations of grids in each
country
26
27
EGI Model
• NGIs
country
– constitute the EGI Council, the
governing body of EGI
28
EGI Model
• NGIs
country
– constitute the EGI Council, the
governing body of EGI
• EGI.eu (was: EGI.org)
– central coordination part
– the “glue” between the NGIs to link
existing NGIs and to support evolution
of NGIs
29
Location of EGI.eu
• Activities of EGI.eu
– Operation of the European level of the
production grid infrastructure (incl. security)
– Provisioning of grid services and support at
EGI Call for Proposals
pan-European level, complementing and
• 8services
Bids from NGIs received
coordinating national
• Ranking
provided by
– Coordination of middleware
development
and
Evaluation
Board
standardization, integration
and testing
• Decision
by EGI PB on
– Training dissemination
and outreach
2 March 2009: Amsterdam,
30
The Netherlands
EGI Structure
EGI.eu
31
User Communities
EGI User Involvement:
• Coordination with Specialized Support Centers (SSC)
• Services for new
and small communities
32
• Coordination of training and event organization
The EGI Infrastructure
EGI Grid Infrastructure
… should be
• a large-scale, production Grid
infrastructure
• built on national grids that interoperate
seamlessly at many levels,
• offering reliable and predictable
services to a wide range of applications
34
EGI Infrastructure
International
Scientificand
andResearch
Collaboration
European-level Grid Services
EGI.eu
EGI.eu
Middleware X
Middleware B
Middleware B
Middleware A
ARC
UNICORE
gLite
Requirements:
•National
Scalability andNational
interoperability
National
National
National
National
… andGrid
Grid
• Grid
Availability
reliability
Grid
Grid
Grid
Initiative
Initiative
Initiative N
Initiative11
Initiative22
• ManageabilityInitiative N
• Sustainability
35
• Autonomy of NGIs
EGI Operations Tasks
Operation of tools and services
• Grid configuration repositories
• Grid accounting repositories
• Grid repositories for SLA
compliance and performance
monitoring
• Grid operations portal
• NGI Grid oversight
User support
• Central ticket handling system
• Gathering requirements for user
support tools
Other international tasks
• MW deployment/roll-out and
support
• Resource allocation & brokering
support
Security
• Interoperations between NGI’s
• Security policy development and
and with other grids
maintenance
Proposal (V3.0)
• Network support
• Coordination of security and EGI Blueprint
incident response
• Definition of best practises,
http://www.eu-egi.eu/blueprint.pdf
procedures, requirements
• Expert team for security
vulnerabilities
• •Catch-all
production
grid
core
Functions
of
EGI
services
• Financing of EGI
• Transition to EGI
36
EGI Operations =
EGI.eu critical services + NGI international tasks
EGI.eu
EGI.eu global tasks
NGI international tasks
NGI local tasks
37
EGI Infrastructure
International
Scientificand
andResearch
Collaboration
European-level Grid Services
EGI.eu
EGI.eu
Middleware X
Middleware B
Middleware B
Middleware A
ARC
UNICORE
gLite
Middleware
… is essential
for the grid
National
National
National
National
National
National
…
Grid
Grid
Grid
to evolve with
Grid
Grid… continues
Grid
Initiative
Initiative
Initiative
Initiative11
Initiative22
InitiativeNN
new functionalities
… development is a
38
long-term process
EGI Middleware
EGI.eu
• should ensure continuity
• should steer the process of providing the
necessary requirements and feedback to
the middleware developers
• should certify selected components
• should promote interaction with
standardization bodies
39
Challenges in an
EGI environment
Challenges for IT
• Scalability:
Federated Hierarchical Approaches
• Examples:
– Grid configuration repositories
– Grid accounting repositories
– Grid repositories for SLA compliance and
performance monitoring
41
D-Grid Project D-MON
ComputingService (DB)
VO-specific View (OGSA-DAI)
42
Challenges for IT
• Scalability
• Manageability:
Tools for managing resources, services,
and VOs
• Example:
– New community joining the grid infrastructure
requires resources
43
Challenges for IT
• Scalability
• Manageability
• Dependability, Availability, Survivability,
Reliability
• Examples:
– Fault tolerance and rollback mechanisms
– Safety and privacy
44
Challenges for IT
• Scalability
• Manageability
Reliability
• Integration:
with supercomputers, clouds, data
repositories, …
45
Leibniz Supercomputing Centre
of the Bavarian Academy of Sciences
Supercomputing@LRZ
Grid@LRZ
46
Challenges for IT
• Scalability
• Manageability
Reliability
• Integration
• Higher-level services
47
Example: Biomedical
Investigation
• Parallel simulation
of blood flow
on the Grid
• Online visualization
of simulation
results on the
desktop
• Interactive steering
of simulation
• Grid is „invisible“
48
Cooperation with University Amsterdam
Flood Management
49
Challenges for IT
• Scalability
• Manageability
Reliability
• Integration
• Higher-level services
• Accounting & Billing
50
Financing EGI
EGI Financial Model
• The majority of EGI costs are generated by the
grid resources and their provisioning in each
country
• The costs related to the coordination tasks of
EGI.eu should be only a tiny fraction
• The sustainability of EGI depends on the
financial sustainability of the NGIs
• Some initial cohesion funding is expected to be
provided by the EC
52
EGI.eu Funding Model
Membership fees
General
Central
Management
INCOME
Contributed
Contributedby
byNGIs
NGIsthrough
through
EGI
EGIdistribution
distributionkey
key
COST CENTERS
Decided
Decidedby
bythe
theEGI
EGICouncil
Council
Covers
Coversmainly
mainlyEGI.eu
EGI.eu
coordination
coordinationcosts
costs
COSTS COVERED
Coordination
53
Membership fees
Service Charges
General
Central
Management
Service
Provisioning
Coordination
Operations
INCOME
Paid
Paidby
byEGI
EGICustomers
Customersfor
for
specific
specificservices
services
COST CENTERS
For
ForEGI.eu,
EGI.eu,NGI
NGIor
or
commercial
commercialservices
services
COSTS COVERED
54
Membership fees
General
Central
Management
Coordination
Service Charges
Project Funds
Service
Provisioning
Operations
Projects
Projects
INCOME
For
Forspecific
specific
shortor
mid-term
shortor
mid-term
COST CENTERS
goals
goals
Through
Through
applications
applications
to
tofunding
funding
agencies
agencies
COSTS COVERED
55
Membership fees
Service Charges
Project Funds
General
Central
Management
Service
Provisioning
Projects
COST CENTERS
Coordination
Operations
Projects
COSTS COVERED
Not initially applied
INCOME
EGI.eu does not own any
56
resources itself
EGI Timeline
30 NGIs + CERN
http://www.eu-egi.eu/partners/loi/
• 25 May:
– NGIs provide LoI to indicate willingness to join EGI
• 29 May: 7th EGI PB Meeting Amsterdam
– (Provisional) EGI Council established through LoI
– EGI Project Teams appointed
• 1 July:
– MoU signed, EGI Council setup completed
– EGI.eu director appointed/identified
• 1 October:
– Financial contributions from NGIs received
• December: EGI Project(s) submission
57
EGI – European Grid Initiative
• EGI consists of NGIs + EGI.eu
• Your input is required to define
mechanisms and functionalities of EGI
• The EGI environment offers many
interesting challenges for IT researchers
Consensus is only possible
if compromises are made
58
http://www.eu-egi.eu
[email protected]
59

EGI - ITI

Transcription

Similar documents

Heritage Category: Listing List Entry No : 1227165 Grade: II County

HISTORICAL MAP PACK LEGEND

hotel marjan - INVESTMENT MANAGEMENT CROATIA

OŠ Bilje, Croatia - The Science Lab in the Kitchen

4* Meteor Hotel - Concorde Travel

BerLin UrBAn Living- De ZwArTe HonD ArcHiTecTs

iTi Tropicals, Inc. - Dairy Foods Magazine

hotel galeb - INVESTMENT MANAGEMENT CROATIA

Modernizing Our Grid - Asia Pacific Resilience Innovation Summit