report of the broadcast of the total lunar eclipse 2014 - Sky

REPORT OF THE BROADCAST OF
THE TOTAL LUNAR ECLIPSE 2014
CREDITS
The Seventh Framework Programme of the European Union (EU, FP7/2007-2013,
INFRASTRUCTURES-2011-2, INFRA-2011-1.2.1: e-Science environments) has promoted the
broadcast though the GLORIA project – GLObal Robotic telescopes Intelligent Array for
e-science” (Grant Agreement –Number 283783).
INFRAESTRUCTURE
1.- Portal Web Broadcast.
The infrastructure deployed by CanarCloud (www.canarcloud.es) was based on a VMware
vSphere Cluster 5 (www.canarcloud.es/vmware/) based also on an Intel Nehalem architecture
interconnected to a centralized RAID storage system capable of supporting the load of the
several thousands of requests on the website of the broadcast along with a DNS balancer. This
interconnected all web instances, both physical and virtual of all collaborators, ensuring an
equitable load sharing of all website visits among all instances.
REPORT OF TECHNICAL BROADCAST OF THE TOTAL LUNAR ECLIPSE 2014
Report by Alejandro Hernández Galtier on behalf of CanarCloud (www.canarcloud.es) for GLORIA
1
Given the high expectations and in order to reduce the time and rate of transfer of resources
included in the website (logos and static images, javascript and style files, etc ...) was
implemented a Proxy with cache of static content in a CDN service by gloria.cdn.canarcloud.net.
The operation of this architecture was as follows:
●
Clients made the request to the web portal of the broadcast that should be designed
so that the size (in bytes) were minimized.
● A Load Balancing system decided which of all servers that form the cluster served
every request that comes in terms of the policy implemented.
● The client was served by the server selected by the load balancing system.
● When the client loads the page directly in his browser, this, asks
gloria.cdn.canarcloud.net to load static content through the proxy, which provides a
minimum of 3h cache.
Web Collaborators
Was requested assistance from public or private institutions, associations, foundations, etc. to
aid in the dissemination via Internet of gateway of the broadcast (live.gloria-project.eu) in an
attempt to prevent the collapse of massive confluence requests.
Web Basics Collaborators
In this type of collaboration, institution, organization, association, etc.. temporarily attached a
machine or set of machines to the cluster property proposed. This machine (or machines) that
are grouped together, served web portal with all the contents of the broadcast.
●
●
The Consorci de Serveis Universitaris de Catalunya [1 web server] → www.csuc.cat
TOTAL [1 web server]
Advanced Web Collaborators
Collaborating with one or more machines compatible with VMware vSphere 5 with more than 4
GB of RAM, more than 4 cores, 3 or more IPs and a bandwidth of 100 Mbps or more have
collaborated integrating these machines as remote nodes of the cluster.
●
●
●
●
●
●
●
●
CanarCloud [1 cluster vSphere 5 + 2 nodes + 2 VPS] → www.canarcloud.es
Instituto de Astrofísica de Canarias [1 vSphere 5 node + 1 VPS] → www.iac.es
University of Warsaw (Poland) [4 VPS] → www.uw.edu.pl
INAF IASF of Bologna [1 vSphere 5 node + 2 VPS] → www.inaf.it
University College Dublin, Ireland [1 VPS] → www.ucd.ie
Extremadura Center for Advanced Technologies [11 VPS] → www.ceta-ciemat.es
Czech Technical University (República Checa) [1 VPS] → www.cvut.cz
Astronomical Institute of the Academy of Sciences (República Checa) [4 VPS] →
www.asu.cas.cz
● Faculty of Sciences and Mathematics (Chile) [1 VPS] → ingenieria.uchile.cl
REPORT OF TECHNICAL BROADCAST OF THE TOTAL LUNAR ECLIPSE 2014
Report by Alejandro Hernández Galtier on behalf of CanarCloud (www.canarcloud.es) for GLORIA
2
●
TOTAL [26 VPS]
Websites with embedded player
Besides the previous contributors, we also find websites that requested the addition of a
player on their websites consuming our video signal but independently supporting web visits
itself.
●
SINC Agency →
www.agenciasinc.es/Retransmisiones/Eclipse-total-de-luna-desde-el-valle-sagrado-d
e-los-incas
● Yahoo.es www.yahoo.es
● Canarias 7 → www.canarias7.es/eclipseluna.cfm
● El Día → www.eldia.es
2.- Video streaming.
The infrastructure deployed by CanarCloud (www.canarcloud.es) was based on 2 instances
Wowza responsible for obtaining the streams generated in the 2 connections provided and its
distribution in the Youtube’s Gloria channel as in the partners with his own streaming
infrastructure.
The operation of this architecture was as follows:
Streams Generation
● Connection 1: with a total duration of 40 minutes from 6:30 – 7:10 UT (8:30 a 9:10
CEST). We saw the sunrise from Tenerife (the Teide’s shadow) and the beginning of the
Total Lunar eclipse from Peru.
○ The final signal was sent to the instance lunar2014.canarcloud.tv for
distribution through the CDN.
● Connection 2: with a total duration of 15 minutes from 8:20-8:35 UT (10:20 a 10:35
CEST) to observe the final of the Total eclipse.
○ The final signal was sent to the instance lunar2014.canarcloud.tv for
distribution through the CDN.
● Additionally: From April 15 at 5:50 UT (7:50 CEST) at 9:40 UT (11:40 CEST), every
minute a live image of the Moon was shown on the website (from the partiality to the
totally).
Live through the portal of the broadcast
● Customers who accessed the broadcast portal (live.gloria-project.eu) found a player
that obtained the signals from each channel making a request to
www.youtube.com/watch?v=aGT75EaLT0E.
Live through the collaborators
● Collaborators with his own infrastructure of video broadcast, obtained the signal
REPORT OF TECHNICAL BROADCAST OF THE TOTAL LUNAR ECLIPSE 2014
Report by Alejandro Hernández Galtier on behalf of CanarCloud (www.canarcloud.es) for GLORIA
3
directly to the reserved instance in the CDN and redistributed it using their own
infrastructure so that customers, who wish, could direct access to the broadcast by
web portals of this collaborators.
● Those collaborators who did not have their own infrastructure of video broadcast or,
for technical reasons, could not connect such infrastructure to ours, could get the signal
directly from Youtube channel www.youtube.com/watch?v=aGT75EaLT0E.
Video Collaborators
Wowza collaborators
Contributors with one or more machines compatible with VMware vSphere 5 with more than 4
GB of RAM, more than 4 cores, 3 or more IPs and a bandwidth of 1 Gbps have collaborated
integrating these machines as remote nodes of the cluster and enabling deployment of Wowza
instances joining to the instances deployed by CanarCloud.
●
CanarCloud [Composed CDN by instances m1.large Wowza Amazon EC2]
www.canarcloud.es
Televisions and Repeaters
Contributors with Live-Stream infrastructure have been redeployed our own signal through
their web portals or television.
●
●
●
●
GLORIA Youtube Canal → www.youtube.com/watch?v=aGT75EaLT0E
CanarCloud Blog → www.canarcloud.tv/blog/2014/04/12/eclipse-lunar-2014
Sky-Live.TV → www.sky-live.tv
La Vanguardia→ www.lavanguardia.com
STATISTICS
1.- Portal Web broadcast
To obtain statistics of visits of the web portal of the broadcast (live.gloria-project.eu) was
enabled a statistics collection service of Google Analytics for global data of the portal and
locally the Awstat collector to analyze the individual contribution of each collaborator.
The period analyzed includes visits from 14 (0h UT) and 16 (0h UT) of April of 2014.
Google Analytics
Under this service we can see that in the analyzed period we obtained 146,840 visits and
215,033 pageviews, with a maximum of 25.690 visits and 46.180 pageviews in the moment of
maximum occultation, as we can see in the chart below.
REPORT OF TECHNICAL BROADCAST OF THE TOTAL LUNAR ECLIPSE 2014
Report by Alejandro Hernández Galtier on behalf of CanarCloud (www.canarcloud.es) for GLORIA
4
Top 10
Country of Origin
Browser
Operating System
Mexico
21,27%
Chrome
53,68%
Windows
54,81%
Peru
13,26%
Safari
11,02%
Android
24,67%
Spain
12,09%
Android
10,90%
iOS
13,11%
Colombia
10,07%
Firefox
10,85%
Macintosh
3,29%
Chile
8,71%
InternetExplorer
6,38%
Linux
1,22%
Argentina
5,76%
Safari (in-app)
2,92%
BlackBerry
1,02%
United States
5,53%
Opera Mini
1,43%
Windows Phone
0,91%
Venezuela
4,16%
Opera
1,09%
(not set)
0,40%
Poland
2,82%
BlackBerry
0,88%
Series40
0,28%
Ecuador
2,33%
S40 Ovi Browser
0,28%
SymbiamOS
0,09%
Awstat
According to the statistics collection tool, Awstat, locally installed in most instances web
partners, we can see the contribution of each one of the contributors to the total percentage
of hits, pages and broadcast traffic.
REPORT OF TECHNICAL BROADCAST OF THE TOTAL LUNAR ECLIPSE 2014
Report by Alejandro Hernández Galtier on behalf of CanarCloud (www.canarcloud.es) for GLORIA
5
Totaling the data obtained from those collaborators with Awstat installed, we see that globally
the portal live.gloria-project.eu has received 171,850 visits and 456,157 pagesviews,
providing 25.82 GBytes of data.
Static Proxy gloria.cdn.canarcloud.net
Analyzing the data supplier by the static proxy deployed, note that there has been 3,105,866
requests which is equivalent to a traffic of 16.20 GBytes of data with a rate of % of success of
91.59%, which means a saves on data transfer provided by the web collaborators close to
38.46% and a corresponding increase of performance.
Websites embedded player (own or loan)
If we analyze the data provided by those collaborators who have publicized the event in their
own web pages using, both our own infrastructure player as their own infrastructure, we find
the following information:
CanarCloud Blog (www.canarcloud.tv/blog/2014/04/12/eclipse-lunar-2014/)
Through the Blog de CanarCloud were obtained 599 visits and 1,650 pageviews.
Sky-Live.TV (www.sky-live.tv)
Through the web portal Sky-Live.TV were obtained 30,597 visits and 52.286 pages views.
REPORT OF TECHNICAL BROADCAST OF THE TOTAL LUNAR ECLIPSE 2014
Report by Alejandro Hernández Galtier on behalf of CanarCloud (www.canarcloud.es) for GLORIA
6
Agencia SINC
(www.agenciasinc.es/Retransmisiones/Eclipse-total-de-Luna-desde-el-valle-sagrado-de-los-in
cas)
Through the web portal of the SINC Agency were obtained 3,253 visits and 3,922 pageviews.
Canarias 7 (www.canarias7.es/eclipseluna.cfm)
Through the web portal of 7 were obtained 5,304 visits and 6,222 pageviews.
Yahoo(es.noticias.yahoo.com/blogs/astronomia-terricolas/eclipses-sangre-y-d%C3%ADas-sin
-gravedad-los-hoax-121740703.html)
Through the web portal of Yahoo were obtained 1,800 visits and 2,200 pageviews.
2.- Video streaming.
We can observe during the 2 connections that a total of 18,849 visitors logged on to direct,
which 5,626 visitors of them observed simultaneously the broadcast.
Televisions and Repeators
CanarCloud Blog (www.canarcloud.tv/blog/2014/04/12/eclipse-lunar-2014/ )
We can observe during the 2 connections that a total of 65 visitors logged on to direct, which
19 visitors of them observed simultaneously the broadcast.
La Vanguardia
(www.lavanguardia.com/ciencia/20140411/54405715284/primera-luna-sangre-semana-sant
a.html)
We can observe during the 2 connections that a total of 2,200 visitors logged on to direct
Yahoo
(es.noticias.yahoo.com/blogs/astronomia-terricolas/eclipses-sangre-y-d%C3%ADas-sin-grave
dad-los-hoax-121740703.html)
We can observe during the 2 connections that a total of 1,810 visitors logged on to direct
TOTAL and COMPARATIVE.
REPORT OF TECHNICAL BROADCAST OF THE TOTAL LUNAR ECLIPSE 2014
Report by Alejandro Hernández Galtier on behalf of CanarCloud (www.canarcloud.es) for GLORIA
7
Figure: Comparison of visits and viewers to the website and video of the broadcast in the various events, TSE­2010 (Total Solar Eclipse, July 11th, 2010, Easter Island), TLE­2010 (Total Lunar Eclipse, December 20th, 2010, Teide), TLE­2011 (Total Lunar Eclipse, June 15th, 2011, Teide), NL­2011 (Northern Lights, in August 2011, Greenland), VT­2012 (Transit of Venus, June 6th, 2012, Australia / Japan / Norway ), NL­2012 (Northern Lights, in August 2012, Greenland), TSE­2012 (Total Solar Eclipse, November 13th, 2012, Australia), NL­2013 (Northern Lights, August 2013 Greenland), TSE­2013 (Total Solar Eclipse, in November 2013, Kenya), TLE­2014 (Total Lunar Eclipse, April 2014, Perú).
INCIDENTS.
1.- Web Portal gloria-project.eu.
As we can observe in the following chart compiled by the web portal application Awstat,
gloria-project.eu suffered an increase in visitors higher than expected so it collapsed upon
reaching 100.00% capacity RAM + SWAP contracted.
REPORT OF TECHNICAL BROADCAST OF THE TOTAL LUNAR ECLIPSE 2014
Report by Alejandro Hernández Galtier on behalf of CanarCloud (www.canarcloud.es) for GLORIA
8
In percentage terms compared to 1000 daily visitors on average, April 15 there was an
increase higher than 4000.00% visitors, resulting in an increase of the traffic higher than
2500.00% .
Currently the web portal gloria-project.eu portal is sharing resources in a VPS server with 5GB
of RAM and 6GB of SWAP with 2 instances of Tomcat designed to support the liveray and the
new portal of users.gloria-project.eu. Because Tomcat, by design, requires a Heap of RAM
reserved for management, is necessary to reserve 3GB of RAM for both instances, which
leaves less than 2GB of RAM available (discounting system consumption) to meet the demands
associated to the access of the Wordpress associated to gloria-project.eu, which, as
recommended by Wordpress and our experience gives us a limit of close to 10,000 visitors.
RECOMENDATIONS.
In order to avoid future collapses, we recommend the following changes:
●
Hire more resources and, if possible, isolate on different VPS the gloria-project.eu
portal and Tomcat instances to apply individulized optimization’s criterias to each VPS.
REPORT OF TECHNICAL BROADCAST OF THE TOTAL LUNAR ECLIPSE 2014
Report by Alejandro Hernández Galtier on behalf of CanarCloud (www.canarcloud.es) for GLORIA
9
●
Optimize the Wordpress configuration through tools of cache and Lazy Load to
minimize the load generated by each visit to the site.
● Optimize the size of all CSS + JS from third discarding unnecessary files, as was done
with the development of live.gloria-project.eu portal.
REPORT OF TECHNICAL BROADCAST OF THE TOTAL LUNAR ECLIPSE 2014
Report by Alejandro Hernández Galtier on behalf of CanarCloud (www.canarcloud.es) for GLORIA
10