Scientific Research and the Grid - High Energy Physics Research

Scientific Research
and
the Grid
Randall Sobie
Institute for Particle Physics
University of Victoria
The Need for the Grid
Expect massive increases in amount of data being
collected in several diverse fields over the next few years
l
l
l
l
l
Astronomy - Massive sky surveys
Biology - Genome databases etc.
Earth Observing
Digital Libraries, Museums . . .
Particle Physics
1PByte ~1000 TBytes ~ 1M GBytes ~ 1.4M CDs
Petabyte
Terabyte
Gigabyte
Randall Sobie
Scientific Research and the Grid
What is the Grid to a scientist?
Isolated, local computational resources
???
Users must explicitly specify the
computing resource and understand the
local infrastructure
Unlimited ubiquitous computing
!!!
Easy to use
Grid
Complexity of infrastructure hidden
from user
Randall Sobie
Scientific Research and the Grid
Particle Physics
University of Victoria and other Canadian Universities are
involved in large international collaborations
Stanford Linear Accelerator
Understand why the universe is
made of matter not antimatter
CERN Laboratory in Geneva
Understand the origin of mass
Randall Sobie
Scientific Research and the Grid
CERN Laboratory
27 km accelerator
Large detectors
Thousands of scientists
Operational in 2007
Vast amounts of data
(multi-PB per year)
Canada has focused its effort on the ATLAS experiment
Total investment in accelerator and detector > $60 M
Randall Sobie
Scientific Research and the Grid
Why do we need a Grid?
ATLAS expects to collect 1 PB of
data per year for 10-15 years
Physicists will want fast access
to a large faction of the data
We cannot afford to build a
single computational centre to
process and store the data.
(i.e. our needs exceed Moore’s law)
Randall Sobie
Scientific Research and the Grid
CERN – Birthplace of the WWW
¡
Science is a community effort
l
l
¡
In 1990 T.Berners-Lee invented the
World Wide Web
l
¡
Conceived to meet the demand for
information sharing between scientists
working all over the world
The Grid
l
Randall Sobie
CERN is a community of 6500 scientists.
Good communication is essential
The particle physics community is the driving
force behind the development of the Grid
Scientific Research and the Grid
LCG
desktops
portables
small
centres
Tier-2
Weizmann
Santiago
RAL
Tier-1
MSU
IC
IN2P3
IFCA
UB
FNAL
Cambridge
CNAF
Budapest
Prague
FZK
Taipei
PIC
TRIUMF
BNL
Legnaro
ICEPP
CSCS
Rome
CIEMAT
Krakow NIKHEF
LHC Grid Project at CERN (LCG)
Tier 0 :
Tier 1 :
Tier 2 :
Tier 3 :
Randall Sobie
CERN
Store and Process Raw Data
Physics Analysis and Generate Simulated Data
Physics Analysis
Scientific Research and the Grid
USC
ATLAS Canada Computing Model
TRIUMF Laboratory Vancouver
Link between the international
LCG Grid and the Canadian Grid
of University sites
Isolate the (shared) University
sites from the large international
user community
Developed with Grid Canada
Randall Sobie
Scientific Research and the Grid
Grid Canada
¡
Formed by NRC, CANARIE and C3.ca to foster Grid
development in Canada
l
l
¡
Supports the Canadian Certificate Authority
l
¡
Passport and visa to the international Grid
Computational and Storage Grid Projects
l
l
Randall Sobie
Collaboration of interested researchers (Victoria,
Alberta, NRC, CANARIE)
www.gridcanada.ca
Grid X1 – experimental computational Grid being developed
for particle physics and other applications
Data Grid for Forestry
Scientific Research and the Grid
Grid Canada Grid X1
Victoria 168 CPUs
Alberta 120 CPUs
NRC
50 CPUs
Victoria
Intel CPUs with Linux OS
Globus Toolkit V2.4
OpenPBS/Maui scheduler
Alberta
NRC
No root access at UVic and NRC
No user access.
grid.phys.uvic.ca
Randall Sobie
Scientific Research and the Grid
Link to LHC Grid at CERN
ATLAS-Canada has developed an
interface between the LHC Grid –
TRIUMF – GC Grid X1 and WestGrid
Each resource publishes
information on its status or the
status of the resources below it
The Resource Brokers use this
information to allocate jobs
Randall Sobie
Scientific Research and the Grid
High speed network
A Grid cannot work without a high speed network
We are working to develop the network infrastructure for
the Canadian Grid and the links to international sites
CANARIE
BCNET
ATLAS-Canada
TRIUMF
TRIUMF
STAR-LIGHT
STAR-LIGHT
10 Gbps
2.5 Gbps
Longest known single hop network.
TRIUMF to CERN via optical internet exchanges in Chicago
(Starlight) and Amsterdam (Netherlight)
Rates in excess of 1 Gbps
Randall Sobie
Scientific Research and the Grid
Plans and Activities
¡
Continued development of the GC Grid X1
l
l
¡
Participation in the ATLAS “data challenges”
l
¡
Series of tests of increasing size and scope
High speed network tests
l
l
Randall Sobie
Data grid capability, monitoring, network
Other applications (BaBar experiment at Stanford)
Canada and CERN
Between Canadian sites
Scientific Research and the Grid
Role of University IT Departments
¡
¡
UVictoria has one of the largest research computing
facilities in Canada
Three main components:
l
l
l
¡
Randall Sobie
High Performance Computer (1999)
Intel-based Computer Cluster (2003)
200 TB High Performance Storage Facility (2003)
Managed by the University IT Group
Scientific Research and the Grid
Challenges in Research/Grid Computing
¡
Demanding applications / demanding researchers
¡
Not 24x7 service but cutting edge technology
l
l
l
l
¡
Grid is completely new to IT group
l
l
¡
Randall Sobie
UVic was an IBM Beta site for LTO2 tape technology
Millions of large files stressing the backup and HSM
Hardware with high power requirements (and heat)
Gigabit networks stressing computers and software
New software
Trust and security issues
Exciting environment for IT personnel
Scientific Research and the Grid
Summary
¡
¡
¡
The Grid offers a solution to the computational and
storage challenges for science
Particle physics needs the Grid for future
experiments (ATLAS)
Grid Canada is a group working on Grid challenges
for Canadian researchers
l
l
¡
Randall Sobie
Established a computational grid : GC Grid X1
Certificate Authority
University IT Groups have made significant
contributions to research computing in Canada
Scientific Research and the Grid
Randall Sobie
Scientific Research and the Grid