Using TrueSpeed™ VNF to Test TCP Throughput in

Using TrueSpeed™
VNF to Test TCP
Throughput in a Call
Center Environment
TrueSpeed VNF provides network operators and enterprise users with repeatable,
standards-based testing to resolve complaints about poor network performance. Based on
IETF RFC 6349 TCP throughput testing methodology, TrueSpeed VNF performance tests serve
as a neutral, third-party evaluation of network quality.
One of the most common challenges to offering broadband
connectivity services is the measurement of actual achieved
throughput. Customers have throughput expectations that are set
by the contractual bandwidth they signed up for with their service
provider. Designed as a virtualized test suite, TrueSpeed VNF lets
RFC 6349 and TrueSpeed VNF
RFC 6349 specifies a practical methodology for measuring end-to-end
TCP throughput in a managed IP network with a goal of providing a
better indication of the user experience. In the RFC 6349 framework,
carriers and their call center agents identify, understand, and solve
TCP and IP parameters are also specified to optimize TCP throughput.
throughput problems very easily and efficiently.
RFC 6349 specifies conducting these three test steps:
This application note briefly describes how TrueSpeed VNF leverages
yy Path MTU detection (per RFC 4821) to verify the network maximum
RFC 6349 and explains the steps required to use the solution in a
transmission unit (MTU) with active TCP segment size testing to ensure
carrier’s call center environment.
that the TCP payload remains unfragmented
yy Baseline round-trip delay and bandwidth to predict the optimal TCP
window size for automatically calculating the TCP bandwidth delay
product (BDP)
yy Single and multiple TCP connection throughput tests to verify TCP
window size predictions that enable automated “full pipe” TCP testing
Application Note
RFC 6349 TCP specifies these results and metrics:
yy TCP transfer time measures the time it takes to transfer a block of
data across simultaneous TCP connections and compares that result
with the ideal TCP transfer time, which is derived from the network
path bottleneck bandwidth and the various Layer 1/2/3 overheadsTCP
efficiency measures the percentage of bytes that do not need to be
retransmitted in the transfer and provides insight in to packet
loss in the network
The advent of virtualized network functions gives network operators
increased flexibility when deploying test solutions. TrueSpeed VNF
virtualizes TrueSpeed test capability so operators can leverage their
existing field test instruments while putting a central TrueSpeed
server on commercial off the shelf (COTS) hardware. This enables
quick evaluations of customer experience and provides actionable
information to resolve any problems discovered. Figure 3 illustrates a
typical use case scenario for TrueSpeed VNF.
yy BDP measures the increase in RTT during a TCP throughput test from
the baseline RTT, which provides an indication of network congestion
during test
For more information on the RFC 6349 test methodology and an
explanation of the test steps and test metrics, please refer to:
http://www.viavisolutions.com/sites/default/files/technical-libraryitems/rfc6349-wp-tfs-tm-ae.pdf.
Figure 2. End-user testing with a downloadable client
TrueSpeed is the industry’s first completely automated
implementation of RFC 6349. Figure 1 illustrates a traditional scenario
using the Viavi Solutions TrueSpeed test.
Figure 3. Technician testing with a T-BERD/MTS portable instrument
Figure 1. Test scenario for TrueSpeed throughput testing
2 Using TrueSpeed VNF to Test TCP Throughput in a Call Center Environment
Client/Server Testing in a Call Center
If a customer calls a service provider customer care call center
complaining of low network throughput, the customer care technician
can initiate a TrueSpeed VNF test to determine if the network is
operating as expected or not. The test process consists of five steps:
1.A business services customer calls the service provider customer care
with a complaint.
2.A customer care technician creates a customized TrueSpeed test with an
Step 2. A Customer Care Technician Creates a Customized
TrueSpeed VNF Test
Network operators typically have customized service offerings to
best meet customer needs. This means that every customer may
have a different information rate (throughput or CIR) and potentially
different quality-of-service (QoS/ DSCP/TOS) parameters associated
with their service. TrueSpeed VNF lets the customer care technician
create a test with customized test parameters that match the
characteristics of the customer’s specific service plan.
encoded URL.
3.The business customer enters the URL in their web browser and runs
the test.
4.The business customer and customer care technician see a common
view of test results.
5.Finger pointing ends!
Step 1. A Business Customer Complains
The customer (often an IT professional) may have perceived a problem
with their network service due to slow downloads, slow file transfers
between locations, sluggish business application performance, or by
using a free bandwidth testing tool. Often, the next step that the
customer takes is to call their service provider’s customer care for help
resolving the problem
The call center technician or engineer enters test parameters for
the test, such as customer ID and CIR, and clicks the Generate URL
button to create an encoded URL that uniquely identifies the test
parameters and the time duration for which the URL is valid. The
customer care technician then sends this URL to the customer so that
the customer can initiate the test when ready.
3 Using TrueSpeed VNF to Test TCP Throughput in a Call Center Environment
Figure 4 shows a TrueSpeed configuration page where a customer care
technician creates a customized test. Table 1 shows configurable data
for the test.
Table 1. Configurable data for a TrueSpeed VNF configuration page
Customer
information
Name, e-mail address, phone number, and
company; this data will be used in the test
report for tracking purposes.
Test request
information
Technician name, technician ID, and test
name; this data will be used in the test
report for tracking purposes
Test creation date and expiration date to determine the time when the test will be valid;
the TrueSpeed VNF server will deny test
requests initiated after the expiration date;
admin users can configure the default test
life and whether or not standard users (such
as customer care technicians) can modify
test expiration dates
Test parameters to
customize the test to
match the customer’s
service parameters
Upstream and downstream committed information rate (CIR) are configured to match
the throughput parameter for the customer’s
service; upstream and downstream CIR can
be configured independently
TCP port sets the destination TCP port number for the test traffic sent from the client
on the customer’s computer to TrueSpeed
VNF; this parameter can be customized to
ensure proper traversal of firewalls and NATs
Figure 4. TrueSpeed VNF configuration page with test URL
4 Using TrueSpeed VNF to Test TCP Throughput in a Call Center Environment
Number of window walks configures how
many different target throughput rates will
be used in the test; setting this parameter
to 1 will create a test with a single target
throughput that is equal to the CIR; setting
this parameter to 2 or higher will add additional tests with lower target throughputs
Connection count runs the test with either a
specified number of TCP connections (such
as 1 for a data backup
application) or lets the TrueSpeed VNF server
pick the optimal number of TCP connections
to simulate multiple users
on the service
Step 3: The Business Customer Runs the Test
The business customer does not need to configure or change any
Together with the actual URL to run the test, the call center technician
parameters on the test. In addition, once the business customer has
provides the business customer with hardware performance
the URL, the customer care technician does not need to intervene to
requirements of the laptop/desktop computer depending on the CIR
change any test parameters.
to be tested. Supported OSs include Windows XP, Windows 7/8, and
The TrueSpeed VNF client user interface does not require configuration
LINUX. The business customer then enters the URL into a standard
web browser (Internet Explorer, FireFox, Chrome).
of any test parameters—these are already encoded in the URL.
Upon initiation by the business customer, the TrueSpeed VNF client
conducts standard RFC 6349 tests. First, the client and server will
measure RTT and MTU size on the link between them and uses them
to calculate the associated BDP. It then runs one or more “walk the
window” tests, with each test filling the transport pipe to a certain,
configurable degree. For each window test, retransmission and RTT
statistics are collected and displayed in the detailed test report on the
server. The information provided indicates the window size at which
any potential problems arise. A window test might pass according to
throughput achieved, but if retransmissions start to occur, a shaper/
buffer may need tuning.
The customer then:
1.Downloads and installs the client software to their computer.
2.Downloads the configuration file that contains customized configuration
parameters for the test.
3.Clicks Start to initiate the test between the customer computer and the
TrueSpeed VNF server.
Figure 5. Opening the test URL and downloading the TrueSpeed VNF client
Figure 6. Status screen on client computer
5 Using TrueSpeed VNF to Test TCP Throughput in a Call Center Environment
Step 4. Business Customer and Customer Care Technician See a
Much more detailed result data appear at the TrueSpeed VNF server
Common View of Test Results
at the end of a test. These results are stored in a database for future
The results displayed on the client side show a basic view of the RFC
use or can be accessed as PDF reports. Also, pertinent client computer
6349 results, providing a very easy-to-read dashboard. Results include:
yy Measured maximum segment size (MSS)
yy Measured round trip time (RTT)
yy Measured throughput vs. target throughput for each window walk
configuration OS settings such as TCP window settings and a
snapshot of simultaneously-running processes are sent back to the
server. This client-side configuration data is very valuable since it can
provide further insight into possible client-side configuration issues
that affect performance.
Test information
Common view of
client results
Figure 8. Portion of final test report shared with the customer
Figure 7. Final test status screen on client computer
6 Using TrueSpeed VNF to Test TCP Throughput in a Call Center Environment
Table 2. Information reported to customer care technician
Common view of client
results
Information shown to business customer
Results diagnosis
The analytics engine interprets results to
determine if the network is performing as
expected and indicates the likely source of
a problem
Detailed results
Laptop information such as the CPU type,
operating system, maximum processor
utilization, and a list of processes used to
rule in or out poor laptop performance as
the cause of poor test results
Server information such as the CPU type,
operating system and maximum processor
utilization used to rule in or out poor
performance on the server as a cause of
poor test results
Information on each window walk
including actual vs. target throughput,
buffer delay, TCP efficiency, and graphs
of throughput vs. RTT and throughput vs.
TCP efficiency over the course of the test
Step 5. Finger Pointing Ends
The common view of test results lets both the business services
customer and the service provider’s customer care technician identify
if the network is performing as expected or if there is indeed a
problem. If a problem exists, both parties can work together on
identifying a solution.
Figure 8. Final test report shown to customer care technician
7 Using TrueSpeed VNF to Test TCP Throughput in a Call Center Environment
Problem Scenario Examples
The VNF test is conducted in the manner described in the previous
This section shows how TrueSpeed VNF analyzes various problem
report in Figure 10.
scenarios, diagnoses causes, and recommends possible solutions
sections and the results are very poor, as seen in the summary
covering:
yy Policed Traffic Issues — problems caused by a policer when end-user
traffic is unshaped
yy Network Over-Buffering — due to over-subscription or lower-speed
hops
yy Inadequate End-Host CPU Resources — where the client laptop is
resource-constrained
Policed Traffic Issues
When TCP traffic is sent into a network policer without first being
traffic shaped (or conditioned), the resulting throughput can be much
lower than the committed information rate (CIR). A traffic policer allows
bursts to occur up to the committed burst size (CBS) and drops packets
which exceed the contracted CIR and CBS limits. Since unshaped
TCP traffic, such as in a Gigabit Ethernet (GE) enterprise network, is
inherently bursty, the TCP traffic may experience serious packet drops.
This causes TCP to repeatedly enter slow start congestion avoidance.
Figure 9 shows a client laptop and TrueSpeed VNF server communicating
through a 100 Mbps CIR and CBS policed network. In this example, the
client and server each have GE connections to the switches and the switch
WAN is GE as well (but policed to 100 Mbps CIR).
Policer:
100 Mbps CIR
64 KB CBS
Emulated network
(40 ms RT delay)
Client laptop
Ethernet
switch
Ethernet
switch
TrueSpeed VNF
Figure 9. Policed network without traffic shaping
8 Using TrueSpeed VNF to Test TCP Throughput in a Call Center Environment
Figure 10. Summary results from a TrueSpeed VNF test
Figure 11 shows the summary results. Throughput was much lower
Network Over-Buffering
than the target throughput or the Layer 4 CIR (94.93 Mbps). The
Another example problem scenario is the case of network over-
diagnosis in the network provider report clearly shows that the RFC
subscription. This can result from a “downshift” in network capacity
6349 TCP efficiency was poor and a possible diagnosis is policer drops
and induced network queuing. Figure 13 shows a WAN network at
(amongst several others).
100 Mbps link speed with no traffic policing. Client traffic is sent at
GE rate and the network is over-subscribed such that egress queuing
occurs as the traffic enters the 100 Mbps ports of the Ethernet
switches.
GE
Client laptop
GE
100 Mbps links
Ethernet
switch
Emulated network
(20 ms RT delay)
Ethernet
switch
TrueSpeed VNF
Figure 13. Network with lower speed (100 Mbps) WAN
Figure 11. TCP efficiency points to packet loss
Further examination confirms the extent of the loss as shown in Figure
12, the throughput versus retransmission loss graph.
Figure 12. Severe and constant retransmission behavior
9 Using TrueSpeed VNF to Test TCP Throughput in a Call Center Environment
Summary results again show that throughput was much lower than
the target throughput or the Layer 4 CIR (94.93 Mbps). The diagnosis
in the network provider report shows that the RFC 6349 buffer delay
was high. A possible diagnosis, shown in Figure 14, is a slower hop.
Figure 14. Buffer delay points to a slow hop
Further examination confirms the extent of the RTT increase as
The summary results again show that throughput was much lower
shown in the throughput versus RTT graph in Figure 15. Notice the
than the target throughput or the Layer 4 CIR (94.93 Mbps). The
relationship of increased RTT to a decrease in TCP throughput and
diagnosis in the network provider report shows that the RFC 6349
that the buffer delay is nearly 50% over the course of the entire test.
buffer delay was high. In this case, there are no slower-speed hops
so network queuing is not an issue. The diagnosis also shows that
the RFC 6349 buffer delay was high, so other possible diagnoses are
investigated (Figure 17).
Tip:
Throughput ~=
Window * 8
RTT
Figure 15. RTT increases under load causing high buffer delay
In situations of high buffer delay but good TCP efficiency (100%
TCP efficiency means there was no loss), smaller queue settings on
interfaces should be considered. Also, link speeds should be verified;
sometimes weak links exist due to auto negotiation issues or an old
switch that hasn’t been upgraded.
Figure 17. Buffer delay points to possible client CPU resource issues
Since TrueSpeed VNF is an agent on the client PC, the detailed report
shows the model of the client laptop and most importantly the CPU
utilization during the test (Figures 18 and 19).
Inadequate End-Host CPU Resources
Another problem scenario is the case of end-host CPU resource issues
which can either be the result of under-powered hardware or other
applications consuming too much CPU. The network configuration
shown in Figure 16 is used and in this example, notice that the
network is pure GE with no policer and no lower-speed hops. The
client traffic test is configured for a target of 100 Mbps CIR.
GE
Client laptop
Ethernet
switch
GE
GE Links
Emulated network
(20 ms RT delay)
Ethernet
switch
TrueSpeed VNF
Figure 16. GE network with no policing, customer CIR = 100 Mbps
10 Using TrueSpeed VNF to Test TCP Throughput in a Call Center Environment
Figure 18. Client configuration
Not only does TrueSpeed VNF indicate that CPU utilization was
excessive during the test, it also provides a list of processes running
during the test. Virus scanners, replication services, and such can be
culprits. In this example, “CPUSTRESS.EXE” was the issue.
Conclusion
TrueSpeed VNF provides network operators and enterprise users with
a repeatable, standards-based test methodology to resolve complaints
about poor network performance.
With TrueSpeed VNF, operators can leverage their installed base of
commercial off the shelf (COTS) server resources to quickly evaluate
customer experience of their network and provide actionable
information to resolve any problems discovered. Based on IETF
RFC 6349 TCP throughput testing methodology, TrueSpeed VNF
performance tests serve as a neutral third-party evaluation of network
quality. Operating as a virtual network function in conjunction with
software hypervisors, Red Hat Linux, and x86 compute resources,
Figure 19. Client CPU consumption and process snapshot
In each of these scenarios, the key point is that with TrueSpeed VNF
the network provider has unprecedented visibility into network
performance characteristics and client-side configuration/resource
consumption. No additional interaction with the end customer
TrueSpeed VNF deploys quickly and tests reliably in all parts of an
operator or enterprise network.
It enables carriers and their call center agents to identify, understand,
and solve Layer 4 throughput problems very easily and efficiently,
reducing costs and increasing customer satisfaction.
is required, eliminating time-consuming communications (and
Moreover, this new virtual approach is leveraging the large installed
miscommunications).
base of T-BERD/MTS instruments supporting TrueSpeed VNF as
well. Together with the well understood RFC 6349 standard, they are
building a foundation for useful TCP throughput measurements.
11 Using TrueSpeed VNF to Test TCP Throughput in a Call Center Environment
Contact Us
+1 844 GO VIAVI
(+1 844 468 4284)
To reach the Viavi office nearest you,
visit viavisolutions.com/contacts.
© 2015 Viavi Solutions Inc.
Product specifications and descriptions in this
document are subject to change without notice.
truespeedvnf-an-tfs-nse-ae
30176079 901 0315
viavisolutions.com