Solution Brief November 2014 Highlights

Cisco UCS Integrated Infrastructure for
Big Data with Splunk Enterprise Delivers
Operational Visibility and Digital Intelligence
Highlights
Comprehensive Integrated
infrastructure
• Cisco UCS® Integrated Infrastructure
for Big Data offers industry-leading
performance, capacity, and scalability
for Splunk Enterprise deployments.
Real-Time Operational Intelligence
• Optimized to run on Cisco Unified
Computing System™ (Cisco UCS),
Splunk Enterprise monitors and
analyzes data from any source,
including customer clickstreams and
transactions, network activity, and call
records, turning machine-generated
data into business insight.
Powerful Search, Analysis, and
Visualization
• Splunk Enterprise provides an easy,
fast, and secure way to analyze the
massive streams of data generated
by IT systems, security devices, and
technical infrastructure.
Built on Cisco UCS Advantages
• The architecture offers unified fabric,
unified management, and advanced
monitoring capabilities.
• Consistent and rapid deployment
using Cisco UCS service profiles
delivers out-of-the-box performance.
Architectural Scalability
• The Cisco UCS with Splunk
architecture is designed to grow to
its maximum size without the need
to add complex layers of switching
infrastructure.
Solution Brief
November 2014
Cisco UCS® Integrated Infrastructure for
Big Data with Splunk Enterprise delivers a
scalable unified infrastructure platform for
operational intelligence.
Today’s data center has evolved into a complex mix of layered and interconnected
systems with blended boundaries to support modern applications. When problems
arise, finding the root cause or gaining visibility across the infrastructure to
proactively identify and prevent outages is a huge challenge for modern enterprises.
Meanwhile, virtualization and cloud infrastructure introduce additional complexity and
create an environment that is more difficult to control and manage.
Traditional tools for managing and monitoring IT and security infrastructure are out
of step with environments that are constantly changing. These tools are inflexible,
costly, less capable, usually not scalable, and not consciously designed for the
complexity of today’s environments and application demands. Designed for
individual specific IT functions, traditional tools do not work across multiple data
center technologies to help solve problems. In addition, their monitoring approaches
are often based on filtering and summarization. When problems arise, they
typically lack the capability to provide targeted, detailed analysis of IT and security
data. Traditional monitoring tools built on relational databases cannot handle the
complexity or massive scale of today’s machine data.
The Splunk Enterprise Advantage
Splunk Enterprise is an industry-leading platform for machine data. Machine data
is one of the fastest-growing and most complex types of big data. It is also one
of the most valuable, containing a definitive record of user transactions, customer
activity, sensor readings, machine behavior, security threats, and fraudulent activity.
© 2014 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.
Cisco UCS Integrated Infrastructure for Big Data with Splunk Enterprise
Delivers Operational Visibility and Digital Intelligence
It incorporates data not just from the
computing, networking, and storage
devices that power applications, but
also from security devices and the
technical infrastructure—power and
cooling resources—that enable the IT
infrastructure to operate.
Splunk Enterprise provides a fast,
easy, and secure way to analyze the
massive streams of machine data
generated by your IT systems and
technical infrastructure, whether its
physical, virtual, or in the cloud. It
collects, indexes, and harnesses live
data generated from almost any source,
format, or location, including packaged
and custom applications, application
servers, web servers, databases,
networks, virtual machines, hypervisors,
and operating systems—without
requiring custom parsers, adapters, or a
back-end database.
After data is indexed, you can correlate
complex events that span diverse data
sources and use Splunk’s powerful
search, analysis, and visualization
capabilities. Splunk Enterprise provides
you with a real-time understanding
of what happened, why it happened,
and what is happening across IT
services, systems, and infrastructure.
Gain operational intelligence with realtime visibility and critical insights into
customer experience, transactions, and
other important business metrics.
The core components of a Splunk
Enterprise deployment include
Splunk indexers, search heads, and
forwarders.
• Splunk indexers are well suited to
the computing and storage capacity
of Cisco UCS C220 M4 and C240
M4 Rack Servers. In addition to
rapidly writing data to disk, indexers
do much of the work involved in
performing searches: reading data
on the disk, decompressing the data,
extracting knowledge, and reporting
results. Therefore, when you
increase the scale of data volumes,
you should add more indexers.
These indexers will help handle
the larger volumes of data, reduce
contention for resources during
© 2014 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.
searches, and accelerate search
performance.
• Search heads search for information
across indexers and are usually both
CPU and memory intensive. Powered
by Cisco UCS C220 and C240 M4
Servers, the large number of Intel®
Xeon® processor cores help Splunk
Enterprise deliver better search
performance.
• Forwarders collect and forward data
to indexers. Forwarders are usually
not resource intensive.
The system resources needed to
enable search and index performance
depend on both the volume of data
being indexed and the search load.
To help ensure that the infrastructure
meets user demands, Splunk Enterprise
is designed to scale horizontally.
If additional search or indexing
performance is needed, a search
head or an indexer system can simply
be added to the architecture without
disrupting operations (see Table 2 later
in this document). This capability allows
Splunk Enterprise to easily scale from
Page 2 of 6
Cisco UCS Integrated Infrastructure for Big Data with Splunk Enterprise
Delivers Operational Visibility and Digital Intelligence
a solution that indexes hundreds of
gigabytes of data to one that indexes
petabytes of data.
Cisco UCS Integrated
Infrastructure for Splunk
Analytics Platform
Given the capability of Splunk
Enterprise to scale to collect, index,
and report on terabytes of data
across an entire data center, a highly
scalable, reliable, and easy-to-manage
infrastructure is critical. To address this
need, Cisco collaborated with Splunk
Certified Architects to analyze the
requirements of Splunk Enterprise. The
result of this collaboration is a highly
tuned version of Cisco UCS Integrated
Infrastructure for Big Data.
Cisco UCS Integrated Infrastructure
for Big Data is the third generation
of the Cisco Common Platform
Architecture (CPA) for Big Data. This
Cisco UCS Integrated Infrastructure
solution is a scalable architecture
designed to meet a variety of scale-
out application demands with high
performance, massive capacity, high
scalability, and smooth data and
management integration capabilities.
The latest version extends this popular
Cisco® solution with improvements in
performance and capacity delivered by
the Intel Xeon processor E5-2600 v3
product family.
Cisco UCS Integrated Infrastructure
for Big Data is built using the following
components:
• Cisco UCS 6200 Series Fabric
Interconnects establish a
single point of connectivity and
management for the entire system.
The fabric interconnects provide
high-bandwidth, low-latency
connectivity for Cisco UCS servers,
with integrated, unified management
for all connected devices provided
by Cisco UCS Manager. Deployed in
redundant pairs, Cisco UCS fabric
interconnects offer full active-active
redundancy, high performance,
© 2014 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.
and the exceptional scalability
needed to support the large number
of servers that are typical in clusters
serving big data applications. Cisco
UCS Manager enables rapid and
consistent server configuration
using Cisco UCS service profiles,
advanced monitoring, and
automation of ongoing system
maintenance activities across the
entire cluster as a single operation.
• Cisco UCS C220 M4 and Cisco
UCS C240 M4 Servers are
enterprise-class systems that
support a wide range of computing,
I/O, and storage-capacity demands
in compact designs. The servers
incorporate the Intel Xeon processor
E5-2600 v3 product family, nextgeneration DDR4 memory, and 12Gbps SAS throughput, delivering
significant performance and
efficiency gains over the previous
generation of servers. The servers
use dual Intel Xeon processor E52600 v3 series CPUs and support
up to 768 GB of main memory (128
Page 3 of 6
Cisco UCS Integrated Infrastructure for Big Data with Splunk Enterprise
Delivers Operational Visibility and Digital Intelligence
or 256 GB is typical for big data
applications) and a range of disk
drive and SSD options. Cisco UCS
virtual interface cards (VICs) are
optimized for high-bandwidth and
low-latency cluster connectivity, with
support for up to 256 virtual devices
that are configured on demand
through Cisco UCS Manager.
Table 1. Splunk Enterprise Single-Instance Reference Architectures
High Retention
(Single Instance)
High Performance
(Single Instance)
Indexer
Cisco UCS C240 M4 Rack
Server with:
• 2 Intel Xeon processor E52680 v3 CPUs (24 cores)
• 256 GB of memory
• Cisco 12-Gbps SAS modular
RAID controller with 2-GB
flash-backed write cache
• Cisco UCS VIC 1227
• 24 1.2-TB 10K SAS drives in a
RAID 10 configuration
• 2 120-GB SSD for the
operating system
Cisco UCS C240 M4 Rack
Server with:
• 2 Intel Xeon processor E52680 v3 CPUs (24 cores)
• 256 GB of memory
• Cisco 12-Gbps SAS modular
RAID controller with 2-GB
flash-backed write cache
• Cisco UCS VIC 1227
• 6 800-GB SSD-EP in a RAID 5
configuration;
• 2 1.2-TB 10K SAS drives for
the operating system
Recommended
indexing
capacity
Up to 250 GB per day
(4-month retention capacity)
Up to 250 GB per day
(1-month retention capacity)
Sample
retention
capacity
1 year
(80 GB per day indexing
capacity)
3 months
(80 GB per day indexing
capacity)
Total storage
capacity
14.4 TB
3.9 TB
Splunk index
capacity
At 2:1 compression:
28.8 TB (projected)
At 2:1 compression:
7.8 TB (projected)
Use cases
• Users requiring fast
performance with a long data
retention time
• Security, operations, and
business intelligence use
cases that require extremely
fast response times
• Multiple concurrent searches with
extremely fast response times
Cisco UCS Reference
Architectures for Splunk
Enterprise
Four Cisco UCS reference architectures
for Splunk are based on Cisco UCS
Integrated Infrastructure for Big Data,
with CPU and I/O subsystems tuned
to address the specific resource
requirements of Splunk Enterprise.
Each reference architecture is based
on a Cisco UCS instance with either
Cisco UCS C220 M4 or C240 M4 rack
servers. As Tables 1 and 2 show, the
architectures vary in disk capacity and
performance and in the distribution
of Splunk Enterprise components
across servers. Note that capacity and
retention are inversely related, and
a smaller indexing volume enables a
greater retention capacity.
• High Retention (Single Instance)
provides high data retention
capabilities for Splunk Enterprise
deployments requiring a single
server.
• High Performance (Single Instance)
provides faster I/O performance for
high-performance Splunk Enterprise
deployments requiring a single
server.
• Distributed Deployment with High
Capacity is designed with highperformance and high-capacity
Cisco UCS C240 M4 servers
© 2014 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.
as indexers, and equally highperforming Cisco UCS C220 M4
servers as search heads.
Page 4 of 6
Cisco UCS Integrated Infrastructure for Big Data with Splunk Enterprise
Delivers Operational Visibility and Digital Intelligence
Table 2. Splunk Enterprise Distributed Reference Architectures
Distributed Deployment with High Capacity
Distributed Deployment with High Performance
Indexer
16 Cisco UCS C240 M4 Rack Servers, each with:
• 2 Intel Xeon processor E5-2680 v3 CPUs (24 cores)
• 256 GB of memory
• Cisco 12-Gbps SAS modular RAID controller with
2-GB flash-backed write cache
• Cisco UCS VIC 1227
• 24 1.2-TB 10K SAS drives in a RAID 10
configuration
• 2 120-GB SSD for the operating system
16 Cisco UCS C220 M4 Rack Servers, each with:
• 2 Intel Xeon processor E5-2680 v3 CPUs (24 cores)
• 256 GB of memory
• Cisco 12-Gbps SAS modular RAID controller with
2-GB flash-backed write cache
• Cisco UCS VIC 1227
• 6 800-GB SSD-EP in a RAID 5 configuration;
• 2 1.2-TB 10K SAS drives for the operating system
Search head
3 Cisco UCS C220 M4 Rack Servers, each with:
• CPU, memory, RAID controller, and Cisco UCS VIC configuration as above
• 2 600-GB 10K Small Form Factor (SFF) SAS drives
Administration and
master nodes
2 Cisco UCS C220 M4 Rack Servers, each with:
• 2 Intel Xeon processor E5-2620 v3 CPUs (12 cores)
• 128 GB of memory
• Cisco 12-Gbps SAS modular RAID controller with 2-GB flash-backed write cache
• Cisco UCS VIC 1227
• 2 600-GB 10K SFF SAS drives
Networking
2 Cisco UCS 6296UP 96-Port Fabric Interconnects
Recommended
indexing capacity
Up to 4 TB per day
(4-month retention capacity)
Up to 4 TB per day
(1-month retention capacity)
Recommended
indexing capacity
with replication
Up to 2 TB per day
Up to 2 TB per day
Sample retention
capacity
1 year
(1.25 TB per day indexing capacity)
3 months
(1.25 TB per day indexing capacity)
Total storage capacity
230.4 TB
62.4 TB
Splunk index capacity
At 2:1 compression:
460.8 TB (projected)
At 2:1 compression:
124.8 TB (projected)
Use cases
Enterprises requiring longer data retention
Enterprises needing to support a large number of
concurrent users that require faster response times
Servers
21 (39 rack units [39RU])
21 (23RU)
Scalability
Additional search heads
1 to 16 additional indexers
Additional search heads
1 to 16 additional indexers
© 2014 Cisco Systems, Inc. All rights reserved. This document is Cisco Public Information.
Page 5 of 6
Cisco UCS Integrated Infrastructure for Big Data with Splunk Enterprise
Delivers Operational Visibility and Digital Intelligence
• Distributed Deployment with High
Performance is designed with highperformance Cisco UCS C220
M4 servers offering computationdense indexers and equally highperforming Cisco UCS C220 M4
servers as the search heads.
Distributed System Scalability
In the distributed architectures, indexers
and search heads can be configured
in a clustered or nonclustered mode.
You can increase the scale of the
architecture by adding search heads
and indexers to up to 80 servers
without the need for any additional
networking infrastructure. Splunk
Enterprise supports clustering for both
search heads and indexers subject to
the following guidelines:
• A search head cluster is a group of
interchangeable and highly available
and Splunk Enterprise search
heads. By increasing concurrent
user capacity and by eliminating the
existence of a single point of failure,
search head clusters reduce the total
cost of ownership (TCO). For failover
with clustering, three search heads
are required.
configured to replicate each other’s
data so that the indexes of the
system become highly available. By
maintaining multiple, identical copies
of indexes, clusters prevent data loss
while promoting data availability for
searching.
Achieve Massive Scalability
of Splunk Enterprise with
Cisco UCS Integrated
Infrastructure
Splunk Enterprise delivers bestin-class operational visibility and
digital intelligence by monitoring all
machine-generated data and making it
accessible, usable, and valuable across
the organization. Cisco UCS Integrated
Infrastructure for Big Data, with its
computing, storage, connectivity, and
unified management features, simplifies
the deployment and offers dependable,
scalable integrated infrastructure that
delivers predictable performance
and high-availability for your Splunk
Enterprise platform with lower TCO.
• Indexer clusters consist of groups
of Splunk Enterprise indexers
The Cisco UCS reference
architectures for Splunk Enterprise
support the massive scalability that
Splunk deployments demand. The
reference architectures described
Americas Headquarters
Cisco Systems, Inc.
San Jose, CA
Asia Pacific Headquarters
Cisco Systems (USA) Pte. Ltd.
Singapore
in this document support up to 80
servers with a pair of 96-port fabric
interconnects. Up to 160 servers in
a single Cisco UCS domain can be
supported by incorporating Cisco
Nexus® 2232PP 10GE Fabric Extenders
into the network fabric. Multiple Cisco
UCS domains—up to thousands of
servers—can be supported using Cisco
Nexus 9000 or 7000 Series Switches.
For More Information
For more information about Cisco UCS,
visit http://www.cisco.com/go/ucs
For more information about Splunk
Enterprise, visit http://www.splunk.com
For more information about the Cisco
UCS SmartPlay program, visit http://
www.cisco.com/go/smartplay
For more information about Cisco UCS
big data solutions, please visit http://
www.cisco.com/go/bigdata
For more information about Cisco UCS
Integrated Infrastructure for Big Data,
visit http://blogs.cisco.com/datacenter/
cpav3
Europe Headquarters
Cisco Systems International BV Amsterdam,
The Netherlands
Cisco has more than 200 offices worldwide. Addresses, phone numbers, and fax numbers are listed on the Cisco Website at www.cisco.com/go/offices.
Cisco and the Cisco logo are trademarks or registered trademarks of Cisco and/or its affiliates in the U.S. and other countries. To view a list of Cisco trademarks, go to this
URL: www.cisco.com/go/trademarks. Third party trademarks mentioned are the property of their respective owners. The use of the word partner does not imply a partnership
relationship between Cisco and any other company. (1110R) LE-44701-00 11/14