Tips for Improving Your Cisco UCS Power Efficiency

White Paper
Tips for Improving
Your Cisco UCS Power
Efficiency
February 2015
What You Will Learn
This document shows you how to increase the power efficiency of your Cisco UCS equipment with recommendations and best
practices as well as guidelines for selecting hardware and BIOS options. It also provides a list of tools that you can use to validate
your power-efficiency choices.
© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information.
Tips for Improving Your Cisco UCS Power Efficiency
February 2015
Contents
Choosing Your Hardware..................................................................................3
Processor Considerations......................................................................................4
Memory Considerations.........................................................................................5
Storage Components.......................................................................................6
Power Supplies................................................................................................7
BIOS Configuration Choices.............................................................................8
BIOS Settings: No Performance Impact.................................................................8
BIOS Settings: Moderate Performance Impact.......................................................9
BIOS Settings: Maximum Energy Savings with Moderate to
Significant Performance Impact.............................................................................10
Tools to Help Determine Power Efficiency.........................................................10
Cisco UCS Power Calculator.................................................................................10
Cisco UCS Performance Manager.........................................................................11
Microsoft Windows: CPU-Z...................................................................................11
Microsoft Windows Task Manager.........................................................................11
Microsoft Windows: powercfg.exe.........................................................................11
Linux Tools.............................................................................................................11
Improved Power Efficiency...............................................................................12
For More Information........................................................................................12
© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information.
Page 2
Tips for Improving Your Cisco UCS
Power Efficiency
White Paper
February 2015
Energy management is critical to efficient
data center operation. You are constantly
balancing power consumption against
the need for availability, scalability, and
performance to lower operating costs and
support robust service levels.
Power consumption is a complex function that depends on your hardware
configuration, BIOS settings, and workload. Although optimizing power efficiency
can be an overwhelming task, Cisco is dedicated to advancing energy efficiency
in your data centers. One example of this priority is the Cisco Unified Computing
System™ (Cisco UCS®). With its unified, policy-based, and ecosystemfriendly approach, Cisco UCS helps you easily manage hardware and software
components—including power—to improve the efficiency of the entire system and
reduce operating expenses.
This document shows you how to increase the power efficiency of your Cisco UCS
equipment with recommendations and best practices as well as guidelines for
selecting hardware and BIOS options. It also provides a list of tools that you can use
to validate your power-efficiency choices.
Choosing Your Hardware
The way you configure your server hardware affects the efficiency with which
the server uses power. This section presents best practices for choosing and
configuring your server hardware to improve power utilization.
© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information. Page 3
Tips for Improving Your Cisco UCS Power Efficiency
February 2015
Processor Considerations
With every Intel® Xeon® processor release, you gain energy efficiency
improvements. Intel delivers processors in what Intel calls a tick-tock release
cycle (Figure 1). With the “tick” release, Intel updates the manufacturing process
using smaller transistors and shorter interconnect leads to reduce capacitance.
Less capacitance means less delay, which results in lower voltage and less
power consumption. With the “tock” release, Intel updates the microarchitecture,
introducing new or enhanced power management features such as low idle states,
core voltage regulation to optimize power delivery, and link power management
states for SATA and PCI Express (PCIe) bus interfaces.
45 nm
Tick
32 nm
Tock
Tick
Nehalem
Tock
22 nm
Tick
Sandy Bridge
Manufacturing Process
Technology Update Cycle
14 nm
Tock
Tick
Tock
Microarchitecture
Update Cycle
Haswell
Figure 1. Intel’s “Tick-Tock” Processor Release Cycle
For example, Figure 2 shows the relative power efficiency improvements from the
Intel Sandy Bridge processor (tock) to the Ivy Bridge (tick) and the Haswell (tock).
The power efficiency improvement from a manufacturing process change was 1.55
times greater than the baseline (Sandy Bridge). The power efficiency improvement
from a combination of manufacturing and microarchitecture changes was 1.92
times greater than the baseline. The idle power from the same microarchitecture
(Sandy Bridge and Ivy Bridge) is equivalent, but the change in microarchitecture with
Haswell resulted in a 14 percent reduction in idle power consumption.
2.0
1.2
Sandy Bridge
1.92
Ivy Bridge
1.6
1.0
Haswell
1.00
1.00
1.55
0.86
1.00
0.8
Power (Watts)
Power per Watt
0.8
1.2
0.6
0.4
0.4
0
0.2
SPECpower_ssj2008 Power Efficiency
(Higher Is Better)
0
Idle Power C6 State
(Lower Is Better)
Figure 2. Power Efficiency and Idle Power Consumption Improvements Through Updates
in Intel Xeon Processor Manufacturing and Microarchitecture Relative to the Sandy Bridge
Idle Power
© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information.
Page 4
Tips for Improving Your Cisco UCS Power Efficiency
February 2015
To optimize server efficiency, you need to understand your performance
requirements when purchasing a processor. You should select the lowest-power
processor that meets your performance, feature, and growth needs. Intel Xeon
processor families are divided into six market segments: Advanced, Standard, Basic,
High Core Count, Frequency Optimized, and Low Power. Each segment is optimized
for a specific market workload. Cisco UCS servers support all Intel processor
segment offerings.
In addition, within each segment, lower-power options are available that can provide
nearly equivalent performance while consuming less power.
We give you the capability to select the lowest-power processor option that meets
your performance and feature requirements. If one of the processors from the
Low Power segment meets your performance and feature requirements, Cisco
recommends selecting this processor segment to increase your power savings.
Memory Considerations
As with processors, several factors affect memory power consumption, including
the amount of memory, the type of memory, the way the memory is organized, the
memory technology used, the operating frequency, and the operating voltage. For a
small total memory configuration, you can consider using unregistered dual in-line
memory modules (UDIMMs). UDIMMs do not have the additional register found on
registered DIMMs (RDIMMs) and load-reduced DIMMs (LRDIMMs), resulting in less
power consumption per module. However, UDIMMs are limited in capacity, and only
two UDIMMs can be configured per memory channel
When possible, you should select DIMMs with x8 dynamic RAM (DRAM). DIMMs
with x8 DRAM access half as many DRAM devices at a time compared to
DIMMs with x4 DRAM (Figure 3). This configuration reduces the average power
consumption. For example, two 8-GB double-date-rate 4 (DDR4) 1Rx8 RDIMMs
consume less power than one 16-GB DDR4 2Rx4 RDIMMs (Figure 4). If the DRAM
organization is the same, consider higher-capacity DIMMs to reduce the number of
total DIMMs being used. Besides reducing total power consumption, reducing the
number of DIMMs per channel (DPC) can allow the memory configuration to operate
at a higher frequency and improve performance.
Front
Back
1
1Rx8
Second Rank Is Energized and Consuming
Power but Not Being Accessed
Read Access
1
2
2Rx4
Figure 3. Memory Organization Comparison
© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information.
Page 5
Tips for Improving Your Cisco UCS Power Efficiency
February 2015
One DIMM per Channel
Power (Watts per DIMM)
10
Two DIMMs per Channel
DDR4 8-GB 1Rx8 2133-MHz RDIMM
DDR4 16-GB 2Rx4 2133-MHz RDIMM
8
DDR4 32-GB 4Rx4 2133-MHz RDIMM
6
4
2
0
0
60
0
Bandwidth (GB/Second/Socket)
60
Figure 4. Power Consumption Compared to Memory Bandwidth
In addition, if you select low-voltage DIMMs that operate at 1.35 volts (V), you can
reduce total memory power by as much as 20 percent. Additional power savings
are possible through the use of DDR4 memory (Figure 5). Typical DDR3 memory
operates at 1.5 or 1.35V, whereas DDR4 memory operates at 1.2V while also
providing higher performance.
8
Core Power
I/O Power
Power (Watts)
6
4
2
0
40nm DDR3
30nm DDR3L
20nm DDR3L
20nm DDR4
Figure 5. DDR4 DIMMs Consume 25 to 30 Percent Less Power
Storage Components
If you can use fewer, higher-capacity storage devices, you can reduce the total
number of storage devices as well as your server’s total power consumption.
However, you must balance your power savings against the performance impact on
workloads sensitive to the number of I/O operations per second (IOPS). Although
© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information.
Page 6
Tips for Improving Your Cisco UCS Power Efficiency
February 2015
solid-state drive (SSD) capacity can be limited, the drives have exceptional
performance and typically consume 3 to 6 watts (W) less power than equivalentcapacity hard-disk drives (HDDs). If top-end performance is not necessary for your
particular workload, you should select the HDD with the lowest operating speed. A
HDD operating at 10,000 rpm typically consumes 1 to 4 watts less power than an
equivalent HDD operating at 15,000 rpm.
When possible, select software RAID, which eliminates the need for PCIe hardware
RAID adapters and reduces your server power consumption by 8 to 17 watts. The
Cisco UCS product portfolio includes FlexFlash technology, a low-power securedigital (SD) card that allows you to boot locally and eliminates the need for hardware
RAID controllers and HDDs.
Power Supplies
The efficiency with which power supplies convert input power to output varies
based on input voltage and load. It may seem reasonable to populate your server
with high-capacity power supplies. However, to reduce conversion losses,
we recommend that you right-size the power supply capacity for the selected
hardware configuration to optimize power-supply efficiency. Use the Cisco UCS
Power Calculator at http://ucspowercalc.cisco.com to right-size your power supply
selection. Typically, power supplies are more efficient when operating at a higher
percentage of their rated output. Thus, selecting a lower-capacity power supply
(right-sizing) increases the load per power supply. As shown in Figure 6, the optimal
choice for the sample configuration is two power supply units (PSUs) running at 30
percent load, instead of four PSUs running at 15 percent load.
Power Savings From 2 PSUs at 30% Load Compared to 4 PSUs at 15% Load
95
Power Efficiency (Percent)
93
91
Recommended Operating Zone
89
87
85
10
20
30
40
50
60
70
80
90
100
Load (Percent)
Figure 6. Power Supply Optimal Power Savings Operating Zone
© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information.
Page 7
Tips for Improving Your Cisco UCS Power Efficiency
February 2015
The Cisco UCS portfolio includes power supplies that support both high-line (200
to 240 volts) and low-line (100 to 120 volts) inputs. If you can, use high-line input.
High-line operation reduces conversion losses by up to 1 to 2 percent. Figure 7
shows the efficiency of the high-line compared to the low-line input voltages of a
typical Platinum-rated Cisco UCS power supply.
95
Power Efficiency (Percent)
93
}
91
Average Power Efficiency Difference:
Approximately 1 to 2 Percent
89
200-240 Volt Input
87
100-120 Volt Input
85
10
20
30
40
50
60
Load (Percent)
70
80
90
100
Figure 7. Power Efficiency Curve Differences Between High-Line and Low-Line Input
Voltages
BIOS Configuration Choices
The way you configure your BIOS options also affects the efficiency with which your
server uses power. This section presents some best practices for configuring your
BIOS options to improve power utilization in your servers. These general guidelines
are not intended as an all-inclusive list of BIOS tuning recommendations. For a
complete list of BIOS tuning recommendations, consult the platform-specific BIOS
tuning guides documented in the For More Information section at the end of this
document.
BIOS Settings: No Performance Impact
When you attach USB devices to your server, particularly a mouse and a keyboard,
they can affect the processor’s package and core C-state residency, which
affects the amount of power consumed during idle periods. If you are not using
these devices, you should disable them in the BIOS to reduce overall idle power
consumption. You can do this without affecting performance.
Whereas the traditional, manual method of setting BIOS parameters is time
consuming and error prone, Cisco UCS uses BIOS policies that automate the
process of configuring BIOS parameters, allowing you to easily deploy BIOS changes
to many servers or a single server.
Table 1 lists several Cisco UCS BIOS settings that should be disabled to reduce
server power consumption.
© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information.
Page 8
Tips for Improving Your Cisco UCS Power Efficiency
February 2015
Table 1. Cisco UCS BIOS Settings to Adjust, with No Performance Impact, for Optimal Power
Efficiency
Setting
Description
SATA ports and mode
Disable if not used
Onboard server configuration utility (SCU) support
Disable if not used
USB ports: External
Disable if not used
USB ports: Internal
Disable if not used
PCIe slot
Disable if not used
BIOS Settings: Moderate Performance Impact
Table 2. Cisco UCS BIOS Settings That Improve Power Efficiency with Minimal to Moderate
Performance Impact
Setting
Suggested State
C6
C6 is the deep power conservation core idle state and is enacted on a
per-core basis.
Enable
C1E
The C1 idle state occurs when all hardware threads supported by a core
have processed a HALT instruction.
Enable
Energy performance
This setting is the energy efficiency policy through which the OS can
provide hints to the power control unit (PCU) through a model-specific
register (MSR) about the desired power-to-performance trade-off.
The PCU can use these hints to manipulate the internal behavior of the
processor and determine whether to target higher performance or greater
power savings.
Balanced Performance
P-state coordination
This is a setting in which each core shares the same voltage. Because of
this sharing, coordination is required to reconcile P-state requests from
logical processors. There are three models of P-state resolution defined by
Advanced Configuration and Power Interface (ACPI): SW_ANY, SW_ALL,
and HW_ALL. (The Haswell server processor has specific support for
SW_ANY and HW_ALL, but does not prevent the use of SW_ALL.) The
selection of a specific P-state coordination type (e.g., HW_ALL, SW_ALL
or SW_ANY) is how the BIOS communicates the P-state support model
to the Operating System Power Management (OSPM). In SW_ANY mode,
the OS power manager is responsible for coordinating the P-state among
logical processors in a package. SW_ANY is the preferred setting for OS
deployments that are biased toward power savings, such as Microsoft
Windows, and HW_ALL is the preferred setting for OS deployments that
are biased toward performance, such as Linux.
SW_ANY
LV DDR mode This setting determines whether the system prioritizes
low voltage or high-frequency memory operations. Power Savings Mode
prioritizes low voltage memory operations over high-frequency memory
operations. This mode may lower the frequency of memory operations to
keep the voltage low.
Power Savings Mode
© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information.
Page 9
Tips for Improving Your Cisco UCS Power Efficiency
February 2015
Before you start adjusting these settings, you need to determine the trade-off
between power consumption and performance requirements for your workload.
Table 2 shows the BIOS settings that will improve power utilization, but with minimal
to moderate performance impact.
BIOS Settings: Maximum Energy Savings with Moderate to Significant
Performance Impact
Some BIOS settings deliver extremely efficient power utilization, but the trade-off
is lower performance. If you have workloads that don’t require high performance to
function well, these settings can give you the most power savings (Table 3).
Table 3. Cisco UCS BIOS Settings That Provide Maximum Power Efficiency with a Moderate to
Significant Performance Impact
Setting
Description
Intel Turbo Boost Technology
This setting determines whether the processor
automatically increases its frequency if it is running
below power, temperature, or voltage specifications.
If this setting is disabled, the processor never
increases its frequency automatically.
Disable to reduce peak power.
Number of enabled cores
This setting determines whether the maximum
number of CPU cores are activated.
Reduce the number of active physical cores to
reduce power use.
Energy performance policy
The OS can provide hints to the PCU through an MSR
about the desired ratio of power to performance. The
PCU can use these hints to manipulate the internal
behavior of the processor and determine whether to
target higher performance or better power savings.
Choose Energy Efficient to prioritize power savings
over performance.
Energy Efficient
DDR speed
This setting determines the frequency of memory
operations.
Set to a lower frequency to reduce power use.
Port link speed
The setting determines PCIe link frequency.
Review for each adapter; set to lower speeds
to reduce power use.
Tools to Help Determine Power Efficiency
Cisco UCS Power Calculator
You can use the Cisco UCS Power Calculator to evaluate power savings from
the hardware configurations discussed previously: lower-power processors,
higher-density DIMMs, UDIMMs, higher-density storage, SDDs, right-sized power
supplies, etc. The Cisco UCS Power Calculator provides you with power estimates
based on a given hardware configuration and target workload. It is most helpful in
configuration planning to right-size your Cisco UCS deployment for the price, power,
© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information.
Page 10
Tips for Improving Your Cisco UCS Power Efficiency
February 2015
and performance that you need for your applications. You can perform what-if
comparisons before you make your purchase.
The Cisco UCS Power Calculator is based solely on the power profile of the
hardware being configured and assumes that all BIOS settings are configured
for maximum performance. Depending on your workload and both firmware and
operating system settings, you may be able to obtain even greater power efficiency
than that predicted by the calculator.
Cisco UCS Performance Manager
You can use the Cisco UCS Performance Manager to evaluate the performance
impact of power-saving configurations such as BIOS settings. This tool allows you to
quantify the impact of power-saving features on performance.
Microsoft Windows: CPU-Z
You can use the CPU-Z application, by CPIUD, or an equivalent application within
Microsoft Windows Server, to gather system information to verify your hardware
configuration and BIOS settings. Although this tool doesn’t tell you what your power
efficiency is, it offers an excellent way to verify your configuration.
Microsoft Windows Task Manager
You can use the Microsoft Windows Task Manager to verify that your workloads are
preforming within the range you desire and expect after you change BIOS power
settings that may affect performance. If the performance isn’t what you want, you
can readjust system BIOS settings to increase performance.
Microsoft Windows: powercfg.exe
You can use the powercfg.exe application in Microsoft Windows Server to control
power settings and configure your server’s default hibernate and standby modes to
save power. You can also use the –energy option to analyze the system for common
energy-efficiency problems, such as a selective suspend process that prevents the
processor from entering a low power state.
Linux Tools
Standard Linux comes with a comprehensive set of simple yet useful applications
such as top and free that can be used to validate the platform under test and
monitor system performance. In addition, each Linux distribution often delivers its
own enhanced tools. The Linux community has numerous open source projects
ongoing, with each addressing different management concerns and capabilities.
This software—for example, yum or apt-get—can be installed separately by the
package manager, or it can be downloaded and installed manually.
These are some of the more useful Linux applications:
• The cpuinfo and meminfo commands retrieve processor and memory information
installed in the system.
• The top command provides a dynamic real-time view of a running system, similar
to Microsoft Windows Task Manager.
• The free command displays the amount of free and used physical and swap
memory in a system, as well as buffers used by the kernel.
© 2015 Cisco and/or its affiliates. All rights reserved. This document is Cisco Public information.
Page 11
Tips for Improving Your Cisco UCS Power Efficiency
February 2015
• The dmidecode tool can dump a system’s desktop management interface (DMI),
or SMBIOS, table contents, such as BIOS, system, baseboard, chassis, processor,
memory, cache, connector, and slot information, with great detail.
• The lm_sensors tool is associated with the kernel driver for monitoring hardware
health, including temperature, fan speed, and voltage level, and other information
about the running hardware.
Improved Power Efficiency
With the combination of Cisco UCS architecture and adjustments in hardware
selection and BIOS settings, you can achieve the optimal balance between
performance and power consumption to get the most from your investment. Power
management can be challenging. For example, changing BIOS settings to tune
power and performance is difficult with traditional architecture. Cisco UCS service
profiles and BIOS policies make it easy to optimize power management settings.
Cisco wants you to achieve the best results for your system by understanding the
trade-offs and best practices for improving the overall efficiency of your data center
resources.
For More Information
• Power Management in the Cisco Unified Computing System: An Integrated
Approach
• Managing Power in Cisco UCS
• Power Management States: P-States, C-States, and PC-States
• Data Center Power and Cooling
• Best Practices for Setting BIOS Parameters for Performance
• Cisco UCS Power Efficiency Beats HP, IBM and Dell Solutions
• Cisco Unified Computing System BIOS Settings
• BIOS Parameters by Server Model
• Configuring BIOS Settings
• Cisco UCS Power Manager
Americas Headquarters
Cisco Systems, Inc.
San Jose, CA
Asia Pacific Headquarters
Cisco Systems (USA) Pte. Ltd.
Singapore
Europe Headquarters
Cisco Systems International BV Amsterdam,
The Netherlands
Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco Website at www.cisco.com/go/offices.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this
URL: www.cisco.com/go/trademarks. Third party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership
relationship between Cisco and any other company. (1110R)
LE-43505-00 02/15