Sample Report Standard VoIP Assessment (your logo)

(your logo)
Sample Report
Standard VoIP Assessment
Call Summary
Call Quality Summary
The chart below shows the percentage of all calls completed during the assessment that
fell into each category of call quality. "Unavailable" call quality indicates that a call
could not be connected or failed.
Call Quality Summary
Unavailable
5%
Poor
23%
Good
41%
Good
Acceptable
Poor
Unavailable
Acceptable
31%
Good - At least 4.03 MOS
Acceptable - At least 3.60 MOS
Poor - Any lower value
Factors Affecting Call Quality
The chart below shows each call quality impairment factor's contribution to the overall
call quality. Each factor's contribution is shown as a percentage of all call-quality
impairments.
Factors Affecting Call Quality
Delay
16%
Jitter
4%
Lost Data
12%
Delay
Jitter
Lost Data
Codec
Codec
68%
Calls by Group
Call Quality Summary by Call Group
The chart below shows the call quality values for up to 10 call groups, ranked from
highest MOS to lowest.
Call Quality Summary by Call Group
4.50
4.00
3.50
MOS
3.00
2.50
2.00
1.50
1.00
MOS
R aleigh_B ldg1-­‐
S anJ o s e-­‐
R aleigh_B ldg1-­‐
R aleigh_B ldg1-­‐
S anJ o s e-­‐
R aleigh_B ldg1-­‐
R aleigh_B ldg2 : B ellev ue : S anJ o s e : R aleigh_B ldg2 : B ellev ue : G .711u S anJ o s e : G .711u G .723.1-­‐A C E L P G .723.1-­‐A C E L P G .723.1-­‐A C E L P G .711u (64 k bps )
(64 k bps )
(64 k bps )
(5.3 k bps )
(5.3 k bps )
(5.3 k bps )
4.38
4.21
3.85
3.64
3.51
3.28
Call Quality Evaluation by Call Group
The chart below shows the average MOS values for up to 10 call groups, indicating the
percentage of completed calls that fell into the MOS result ranges defined for the
assessment.
Call Quality Evaluation by Call Group
100%
80%
60%
40%
20%
0%
Good
R aleigh_B ldg1-­‐
S anJ o s e-­‐
R aleigh_B ldg1-­‐
R aleigh_B ldg1-­‐
S anJ o s e-­‐
R aleigh_B ldg1-­‐
R aleigh_B ldg2 : B ellev ue : S anJ o s e : R aleigh_B ldg2 : B ellev ue : G .711u S anJ o s e : G .711u G .723.1-­‐A C E L P G .723.1-­‐A C E L P G .723.1-­‐A C E L P G .711u (64 k bps )
(64 k bps )
(64 k bps )
(5.3 k bps )
(5.3 k bps )
(5.3 k bps )
100.00%
73.96%
38.39%
0.00%
0.00%
0.00%
A c c eptable
0.00%
10.27%
26.79%
100.00%
14.73%
0.30%
P oor
0.00%
7.29%
27.08%
0.00%
77.08%
86.01%
Unav ailable
0.00%
8.48%
7.74%
0.00%
8.18%
13.69%
Factors Affecting Call Quality by Call Group
The chart below breaks out call quality impairment factors to show the impact of each (in
points subtracted from the MOS) on the overall call quality for up to 10 call groups with
the lowest MOS values.
Factors Affecting Call Quality by Call Group
4.50
4.00
MOS
3.50
3.00
2.50
2.00
1.50
1.00
D elay
R aleigh_B ldg1-­‐
S anJ o s e-­‐
R aleigh_B ldg1-­‐
R aleigh_B ldg1-­‐
S anJ o s e-­‐
R aleigh_B ldg1-­‐
S anJ o s e : B ellev ue : R aleigh_B ldg2 : S anJ o s e : B ellev ue : R aleigh_B ldg2 : G .723.1-­‐A C E L P G .723.1-­‐A C E L P G .723.1-­‐A C E L P G .711u (64 k bps ) G .711u (64 k bps ) G .711u (64 k bps )
(5.3 k bps )
(5.3 k bps )
(5.3 k bps )
0.29
0.16
0.06
0.06
0.05
0.03
J itter
0.00
0.01
0.00
0.05
0.09
0.00
L o s t D ata
0.14
0.02
0.00
0.45
0.06
0.00
C o dec
0.71
0.71
0.71
0.00
0.00
0.00
MOS
3.28
3.51
3.64
3.85
4.21
4.38
Calls by Day
Call Quality Evaluation by Day
The chart below shows each day's average MOS values with a line graph. The bar graph
evaluates each day's MOS values according to the MOS result ranges defined for the
assessment and shows the percentage of Good, Acceptable, Poor, and Unavailable calls.
100%
4.50
80%
3.80
60%
3.10
40%
2.40
20%
1.70
0%
2/28/2003
3/1/2003
Good
45.51%
A c c eptable
30.45%
P oor
Unav ailable
MOS
3/2/2003
3/3/2003
3/4/2003
3/5/2003
3/6/2003
3/7/2003
46.53%
46.18%
40.63%
40.97%
35.42%
33.85%
33.71%
34.38%
32.47%
29.51%
29.69%
32.47%
30.21%
32.20%
22.44%
16.49%
17.71%
25.17%
25.52%
26.74%
29.51%
24.24%
1.60%
2.60%
3.65%
4.69%
3.82%
5.38%
6.42%
9.85%
3.94
3.97
3.96
3.90
3.89
3.84
3.79
3.84
MOS
Call Quality Evaluation by Day
1.00
Factors Affecting Call Quality by Day
The chart below breaks out call quality impairment factors to show the impact of each
impairment (in points subtracted from the MOS) on the overall call quality per day.
Factors Affecting Call Quality by Day
4.50
4.00
MOS
3.50
3.00
2.50
2.00
1.50
1.00
2/28/2003
3/1/2003
3/2/2003
3/3/2003
3/4/2003
3/5/2003
3/6/2003
3/7/2003
D elay
0.08
0.07
0.07
0.08
0.09
0.09
0.10
0.09
J itter
0.00
0.00
0.00
0.01
0.02
0.03
0.05
0.04
L o s t D ata
0.04
0.02
0.03
0.06
0.06
0.09
0.10
0.10
C o dec
0.36
0.35
0.35
0.35
0.35
0.35
0.36
0.34
MOS
3.94
3.97
3.96
3.90
3.89
3.84
3.79
3.84
Calls by Hour
Call Quality Evaluation by Hour
The chart below shows each hour's average MOS values with a line graph. The bar graph
evaluates each hour's MOS values according to the MOS result ranges defined for the
assessment and shows the percentage of Good, Acceptable, Poor, and Unavailable calls.
100%
4.50
80%
3.80
60%
3.10
40%
2.40
20%
1.70
0%
12 1 2 3 4 5 6 7 8 9 10 11 12 1 2 3 4 5 6 7 8 9 10 11 A A A A A A A A A A A A P P P P P P P P P P P P
M M M M M M M M M M M M M M M M M M M M M M M M
MOS
Call Quality Evaluation by Hour
1.00
44 43 44 42 43 43 44 43 39 38 32 38 38 35 37 37 38 36 41% 44 42 43 45 44
Good
A c c eptable 34 32 33 33 33 33 30 32 30 30 33 29 29 30 31% 32 29 30 30 29 33 34 32 33
15% 19% 18% 18% 17% 21% 21% 21% 23 27 31% 27 32 29 26 29 30 29 25 24 23 17% 19% 21%
P oor
Unav ailable 7% 6% 5% 7% 7% 2% 4% 4% 8% 5% 5% 7% 1% 6% 6% 2% 4% 4% 4% 2% 2% 6% 4% 2%
3.9 3.9 3.9 3.9 3.9 3.9 3.9 3.9 3.8 3.8 3.7 3.8 3.7 3.8 3.8 3.8 3.8 3.813.9 3.9 3.9 3.9 3.9 3.9
MOS
Factors Affecting Call Quality by Hour
The chart below breaks out call quality impairment factors to show the impact of each
impairment (in points subtracted from the MOS) on the overall call quality per hour.
Factors Affecting Call Quality by Hour
4.50
4.00
MOS
3.50
3.00
2.50
2.00
1.50
1.00
12 1 2 3 4 5 6 7 8 9 10 11 12 1 A A A A A A A A A A A A P P
M M M M M M M M M M M M M M
2 3 4 5 6 7 8 9 10 11 P P P P P P P P P P
M M M M M M M M M M
D elay
0.07 0.070.07 0.08 0.070.08 0.070.08 0.09 0.09 0.12 0.10 0.11 0.11 0.090.09 0.09 0.10 0.08 0.08 0.080.07 0.070.07
J itter
0.00 0.000.00 0.00 0.00 0.01 0.010.00 0.03 0.020.07 0.05 0.040.04 0.050.04 0.03 0.04 0.01 0.01 0.010.00 0.010.00
L o s t D ata 0.04 0.050.03 0.03 0.030.05 0.030.04 0.05 0.08 0.11 0.07 0.11 0.08 0.070.09 0.11 0.10 0.06 0.04 0.050.02 0.040.04
C o dec
0.33 0.340.35 0.36 0.350.35 0.350.35 0.35 0.360.35 0.35 0.350.36 0.350.35 0.34 0.350.35 0.35 0.350.37 0.340.35
MOS
3.97 3.943.95 3.95 3.953.93 3.943.93 3.89 3.873.76 3.83 3.793.82 3.853.84 3.84 3.813.92 3.93 3.923.95 3.963.95
Sample Data : Summary Page
Once all testing and verification of the network is complete, a brief summary document should be
developed to provide a quick reference to the current state of the network. Additionally, for easy
reference, a zipped file should be created to include all documentation from the entire process.
What follows is an example of a summary document.
The final results were very positive as every network segment tested score near toll-quality or better. The
following chart shows each segment, the compression algorithm, the number of simultaneous calls and the
MOS score. A 4.0 is considered toll-quality.
Wireless Link
G.729a
9 Calls
4.00
Wireless + Wan
G.729a
5 Calls
3.96
*Corporate WAN
G.729a
15 Calls
4.01
*Remote1 WAN
G.729a
3 Calls
4.01
Even though the network scores are fantastic as indicated by above table, a few issues were discovered that
need to be mentioned. These issues can easily affect voice/video quality in the future.
The first item is DSCP packet. The network was verified to pass and trust incoming DSCP values on all
subnets except for 172.16.5.x and 172.16.15.x. This means that IP voice traffic from these subnets can not
adhere to layer 3 QoS until this issue is fixed.
The second item is COS / DSCP packets. Sniffer tests were run to verify that layer 2 COS values were
properly passed and were not overwriting DSCP values with mapping of some sort. Many of the subnets
passed these headers correctly, but the following ones could not be verified during onsite visit:(172.6.x):
4,5,6,7,10,15,18,26,28.
Router utilization was extremely high. This is due to the massive overhead required to encrypt VoIP. At 5
calls to Mx, 15 to corporate and 3 to Remote1, all routers utilization exceeded 45%. This is very dangerous,
as it takes very little additional traffic to cripple a router once this level is reached. In fact, during VoIP testing
such an occurrence happen with both the Mx and Remote1 routers on a few occasions, most notable when
Video conferencing was in session or if we tested more then the number of calls listed in the table above. A
new router and circuit should eliminate this problem from Remote1. An introduction of a hardware
accelerator at Mexico should help the problem, but it will still need to be monitored very closely.
Very strange results were obtained the first evening as 30% of calls were unavailable. However, calls there
were available all scored toll-quality. The exact cause of this was never determined. It could have been a
network problem as Clients’ management software indicated a few outages during the evening. It could have
simply been the console computer, as it locked up the next morning when assessment was stopped. It could
have been an overload in calls, as we reduced all calls by 20% for the final test. The general feel, by all
involved was that this was a strange occurrence, but that the final “near-perfect” test was more reality and
thus discounts this first test to almost meaningless. However, this situation should be monitored closely to
see if something similar happens at some time in the future.
WAN utilization reached 50% on both Mx and corporate sites at time of heavy traffic, VoIP traffic, and video
conferencing. This should be monitored closely in the near future and adjusted accordingly.