Document 6513017

Overview
Performance of Distributed Systems
(COMP40550/COMP47070)
ASE module, 1-5 December 2008
!
!
Practicals: start each afternoon at 2pm, B1.06 (unless
otherwise notified…)
Dr. John Murphy
Room B2.15,
B2.15, ee-mail: [email protected],
[email protected], tel: 716 2928
2928
Lectures: start each morning at 9:30am, Computer Science
Lecture Theatre (should be finished by 12:30…)
!
Recommended (not required) Textbook:
!
Prof. Liam Murphy
2002, ISBN 0-201-72229-1
Room B2.22,
B2.22, ee-mail: [email protected]
[email protected],, tel: 716 2914
Head Demonstrator: John Bergin
e-mail: john.bergin
@ucd.ie
ie
john.bergin@ucd.
!
!
Contacting us: email preferred…
Website:
http://www.csi.ucd.ie/staff/lmurphy/home/ase_perf-eng.htm
How to Pass this Module
!
!
!
!
Attend Lectures
Read the Lecture Notes, & understand them – ask
questions if you don’t understand
! 1 short In-class test, worth 25%, on Friday 9:30am,
to make sure you do this!
Attend Practicals and ask the demonstrators questions
Assignment Report (due after this module is over…)
! Worth 75% of the marks in this module
!
!
We reserve the right to interview any student about
their report – so don’t plagiarise code/text/…
No Final Exam
Performance Solutions: A Practical Guide to Creating Responsive, Scalable
Software, Connie U. Smith and Lloyd G. Williams, pub. Addison Wesley,
Textbook
!
!
Connie Smith’s company’s website:
http://www.perfeng.com/
However, our lecture notes (on the web) will be
your primary source material.
What does performance mean?
Performance of Distributed Systems
(COMP40550/COMP47070)
!
!
A Brief Introduction to
Performance Engineering
!
Typical performance engineering tasks
!
Specifying system performance requirements
!
Evaluating design alternatives
!
Comparing the performance of 2 or more systems
!
Determining optimal parameter value(s)
!
Finding performance bottleneck(s)
!
Characterising the load on the system
!
Capacity planning – system sizing for the future
!
Forecasting performance at future loads/workmixes/etc
A measure of how a system operates
! Examples: response time; throughput; efficiency; users’ QoE; …
Possible goals of performance engineering:
! Achieve the best performance for a given cost
! Achieve acceptable performance at minimum cost
! Determine performance/cost of each of a number of alternatives
! Find performance problem(s) and determine how to fix them
! Determine when and how the system should be modified/upgraded
so that its performance remains acceptable
! and many others…
Performance Engineering is an “art”
! requires detailed knowledge of system and its environment, and
choice of appropriate evaluation techniques & tools
! BUT: there are some general principles…
Why is performance evaluation difficult?
!
Lots of mistakes can be made!
! No goals for the performance study; or goals are unclear, or biased
! Unsystematic approach e.g. arbitrary parameter and workload
values
! Inappropriate performance metrics e.g. easily computed but
irrelevant
! Unrealistic workload
! Inappropriate level of detail in the model
! No analysis; or incorrect analysis e.g. simulation run too short
! Ignoring variability, or future possibilities
! Inappropriate presentation of results
! …and many many others…
Misleading performance results
!
Could be deliberate e.g. vendor benchmark results, or incorrect analysis
!
Example: ratio game
Average Throughputs:
System
Workload 1
A
20
B
10
Throughput measurements:
System
Workload 1
Workload 2
A
20
10
B
10
20
Systematic approach to performance evaluation
!
State goals of the study and define system scope
!
Define system services and possible outcomes of user requests
!
Select performance metric(s)
!
Identify system parameters that affect performance
!
Workload 2 Average
10
15
20
15
Average Throughput wrt B:
System
Workload 1 Workload 2 Average
A
2
0.5
1.25
B
1
1
1
Average Throughput wrt A:
System
Workload 1 Workload 2 Average
A
1
1
1
B
0.5
2
1.25
A and B have the
same performance
“A is better than B”
“B is better than A”
!
Select which parameters are variable
Select evaluation technique(s)
!
Analytical modelling
!
Simulation/emulation
!
Experimental (real system/prototype)
!
Select workload(s) to be studied
!
Design experiments
!
Analyse and interpret the data (statistical techniques)
!
Present the results in an understandable manner