Il Sistema informativo del middleware gLite

Il Sistema informativo del
middleware gLite
Dr. Marco Fargetta
[email protected]
INFN, Italy
Catania, 13 Dec 2010
IV Corso di formazione INFN per
amministratori di siti GRID
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
1 / 33
Outline
Introduction
Background
GLUE Schema
LDAP
BDII
R-GMA
Monitoring
Conclusions
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
2 / 33
Introduction
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
3 / 33
gLite Components
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
4 / 33
The Information System
I
What?
I
I
Why?
I
I
I
I
System to collect information on the state of resources
To discover resources of the grid and their nature
To check for health status of resources
To provide data in order to manage the workload more
efficiently
How?
I
I
Monitoring and publishing fresh data on the state of
resources
Adopting a well known data model
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
5 / 33
Who and how use the IS
I
Users
I
Retrieve information about resources:
I
I
I
I
Site managers
I
I
Where can I run my job?
Where can I copy my files?
Which software packages are available on a given CE?
Publish information about the resources and services
they provide
Middleware
I
I
WMS: matching job requirements and allocating the
resources
Monitoring Services: retrieving information about the
status and availability of resources
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
6 / 33
Information Services
I
I
The gLite Data Model uses GLUE Schema
Two IS systems available in gLite: BDII and R-GMA
I
The Berkeley DB Information Index (BDII)
I
I
I
I
has been the LCG middleware Information System provider
is an evolution of Globus Meta Directory System (MDS)
is based on Lightweight Directory Access Protocol (LDAP)
servers
The Relational Grid Monitoring Architecture (R-GMA)
I
I
I
Dr. Marco Fargetta
is an implementation of the Grid Monitoring Architecture
(GMA) standardised by the Open Grid Forum (OGF)
is a relational implementation of the GMA
is strongly Web Services Oriented
Amm. siti Grid
Catania, 13 Dec 2010
7 / 33
Background
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
8 / 33
Why is an Information schema required?
I
I
Resources are geographically dispersed, span
multiple trust domains and are heterogeneous
Resources can be dynamically contributed by
different owner institutions
I
I
a precise and shared description of resources among
information consumers and resource providers is
necessary
The same resources description has to be readable
from different Grid infrastructures to allow
interoperability among them
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
9 / 33
GLUE Schema Overview
I
I
I
I
GLUE stands for Grid Laboratory Uniform
Environment
Describe the Grid resources information stored in
the IS
Independent from the underlying technology
Actual release is mapped on
I
I
I
LDAP
XML
ClassAd (Condor Matchmaking language)
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
10 / 33
GLUE Schema Overview
I
I
The Grid components are represented as objects
which have attributes and relations to other objects
The entities of the GLUE Schema are organised
hierarchically
I
Include the concept of Site, Cluster, Computing Element,
Storage Element, and an abstraction of service
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
11 / 33
GLUE Schema Structure
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
12 / 33
LDAP Overview
I
I
I
It is a protocol that defines the method by which
directory data is accessed
Optimised for reading, browsing and searching
information (write-once-read-many-times service)
Data is represented as a hierarchy of objects
(entities) forming a tree structure
I
Data Information Tree (DIT)
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
13 / 33
LDAP Data Information Tree
I
Distinguished Name (DN)
I
I
I
Unique name that
unambiguously identifies an
entry
Es.
dn:o=INFN,c=Italy,dc=grid
Attributes Types
Meaning
Domain Component
Country Name
Organisation Name
Organisation Unit Name
Common Name
Dr. Marco Fargetta
Attribute
DC
C
O
OU
CN
Amm. siti Grid
Catania, 13 Dec 2010
14 / 33
LDAP Referrals
I
I
I
An LDAP server can store a partial DIT
To obtain the DIT all servers need to be linked
together
All client requests have to start at the global
directory LDAP 1
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
15 / 33
BDII
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
16 / 33
GRISs, local BDII and BDII
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
17 / 33
gLite IS Architecture
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
18 / 33
The BDII
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
19 / 33
LDAP clients
I
I
I
BDII servers use standard LDAP protocols
All LDAP client are able to access BDII information
Command line or Graphical application for every OS
are available to access an LDAP server
Es. ldapsearch
ldapsearch -xxx -h [hostname] -p 2170 -b
mds-vo-name=local,o=grid
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
20 / 33
Es. JXplorer
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
21 / 33
gLite User Commands
I
I
gLite provides several tools to manage the IS
Two gLite commands in the UI for the IS queries
I
I
lcg-infosites: shows some information related to a
Grid component
lcg-info: supports complex queries to retrieve more
specific information
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
22 / 33
lcg-infosites
Usage
lcg-infosites –vo voname -[v] -f [site name] [option(s)] [-h
| –help] [–is BDII]
I
The options can be one among the following:
I
se, ce, closeSE, rb, lrc, lfc, vobox, fts, sitenames, tag, all
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
23 / 33
lcg-info
Usage
lcg-info –list-[CEO|se|service|site|attrs] [–bdii bdii] [–vo
vo] [–sed] [–debug] [–query query] [–attrs list]
I
One or more attributes can be retrieved for specific
sites:
I
The query element allows to identify the site
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
24 / 33
R-GMA
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
25 / 33
R-GMA Overview
I
I
I
R-GMA is based on Grid Monitoring Architecture
(GMA) standard defined by the GGF
Implements a Consumer-Producer model
Information are stored in a SQL relational database
using a virtual DB per VO
I
I
I
I
Information are always updated because the consumer
get information directly from the provider
Users can query information using a subset of the SQL
Web service oriented
CLI and Web user interface
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
26 / 33
R-GMA Producer-Consumer Model
I
The Producer stores its
location (URL) in the Registry
I
The Consumer looks up
producer URLs in the Registry
I
The Consumer contacts the
Producer to get all the data or
to listen for new data
I
Data flows directly from the
Producer to the Consumer
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
27 / 33
Monitoring
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
28 / 33
Monitoring
I
I
Several centralised monitoring systems available
Official monitoring are:
I
I
I
GridMap managed by CERN at http://gridmap.cern.ch
GStat managed by Academia Sinica at
http://goc.grid.sinica.edu.tw/gstat
Grid Real Time Monitor by Imperial College London at
http://rtm.hep.ph.ic.ac.uk
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
29 / 33
Conclusions
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
30 / 33
Conclusions
I
I
I
I
Two Information System are provided by gLite:
BDII and R-GMA
Information are organised following the GLUE
Schema
Users can contact the top BDII in the hierarchy to
get the information of all the resources
Current implementation use only BDII to check the
state of the resources
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
31 / 33
Some References
I
gLite 3.1 User Guide
https://edms.cern.ch/file/722398/1.2/gLite-3-UserGuide.html
I
Wiki CERN
https://twiki.cern.ch/twiki/bin/view/EGEE/InformationSystem
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
32 / 33
Questions!!!
Dr. Marco Fargetta
Amm. siti Grid
Catania, 13 Dec 2010
33 / 33