Overview of the Log Correlation Engine Splunk Client November 20, 2014

Overview of the Log Correlation
Engine Splunk Client
November 20, 2014
(Revision 3)
Table of Contents
Introduction ......................................................................................................................................... 3
Standards and Conventions........................................................................................................................... 3
Configuring Log Correlation Engine Splunk Client ......................................................................... 3
Importing the Edited Policy File ........................................................................................................ 7
Configuring Splunk Indexer to Forward Data ................................................................................... 9
Data Comparison .............................................................................................................................. 11
For More Information ........................................................................................................................ 14
About Tenable Network Security ..................................................................................................... 15
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
2
Introduction
Tenable’s Log Correlation Engine unifies vulnerability collection and event analysis data through SecurityCenter, which
provides easy-to-use dashboards to display multiple data points in a centralized view. Organizations that choose to send
Splunk logs to the LCE have a unique advantage in that Splunk data is normalized by LCE and can be included for
automatic anomaly detection, discovering assets, and additional vulnerability information including botnet and malware
detection.
The Log Correlation Engine Splunk Client has one purpose: to forward data that Splunk collects to the LCE server. Once
the data reaches the LCE server, the data is reviewed and normalized so it can be queried in SecurityCenter. The scope
of this client can vary depending on what data is being forwarded from Splunk to the LCE Splunk Client.
This document is intended to assist with basic configuration of the LCE Splunk Client. It will also briefly cover some
configuration steps that are required on the Splunk Indexer.
A working knowledge of Splunk, SecurityCenter, and LCE operation and architecture is assumed. Familiarity with general
log formats from various operating systems, network devices, and applications, as well as a basic understanding of
Linux/Unix, is also assumed.
Standards and Conventions
Throughout the documentation, filenames, daemons, and executables are indicated with a courier bold font such as
gunzip, httpd, and /etc/passwd.
Command line options and keywords are also indicated with the courier bold font. Command line examples may or
may not include the command line prompt and output text from the results of the command. Command line examples will
display the command being run in courier bold to indicate what the user typed while the sample output generated by
the system will be indicated in courier (not bold). Following is an example running of the Unix pwd command:
# pwd
/opt/local/lce
#
Important notes and considerations are highlighted with this symbol and grey text boxes.
Tips, examples, and best practices are highlighted with this symbol and white on blue text.
Configuring Log Correlation Engine Splunk Client
The first step is to install and then configure the LCE Splunk Client on a RHEL 5 or 6 32/64-bit host. The client can be
installed on the same server as Splunk, if desired. Throughout this example, the LCE Splunk Client resides on the same
server as Splunk. Choose the correct installation package for the server OS and install the package. An example of the
installation is shown below:
[root@LCE_splunk tmp]# rpm -ivh lce_splunk-4.2.0-el6.x86_64.rpm
Preparing...
########################################### [100%]
1:lce_splunk
########################################### [100%]
Please run /opt/lce_splunk/set-server-ip.sh to configure your LCE server's IP and
port.
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
3
After the installation is complete, execute the set-server-ip.sh script. The LCE server IP address and port that the
client communicates with the LCE server will be required. An example is shown below:
[root@LCE_splunk lce_splunk]# /opt/lce_splunk/set-server-ip.sh
Enter the new desired LCE server IP or hostname.
>>
192.168.1.14
Enter the new desired LCE server port [31300].
>>
31300
Updating LCE Server IP from 192.168.1.140 to 192.168.1.14... Updating LCE Server Port
from 31300 to 31300...
Done
Stopping LCE Splunk Client daemon
Starting LCE Splunk Client daemon
[
[
OK
OK
]
]
After the LCE Splunk Client configuration is complete, access SecurityCenter as the admin user and choose “Resources”
from the menu options, followed by “LCE Clients”.
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
4
Next, select the host that has the LCE Splunk Client installed, and choose “Assign Policy”:
Once the “Assign Policy” window opens, scroll down, select the “default_rhel_lcesplunk.lcp” file, and choose “Export”.
Save the file and then open it in a text editor.
The default policy downloaded will be similar to the one shown below:
<?xml version="1.0" encoding="UTF-8" standalone="no" ?>
<options xmlns:xi="http://www.w3.org/2003/XInclude">
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
5
<!-- Splunk client log messages are written to a file named according to the date
in
the directory specified below. -->
<log-directory>/opt/lce_splunk/</logdirectory>
<!-- The splunk client listens for connections from Splunk Indexers at the port
specified
below. Each Splunk Indexer in the deployment should be configured to forward
data
to this location. -->
<listen-port>9800</listen-port>
<!-- The splunk client will only accept connections from servers authorized with
the
splunk-server keyword. -->
<splunk-server>127.0.0.1</splunk-server>
<!-- When the following line is uncommented, debugging is enabled in the client. >
<!-- client-debug>1</client-debug -->
<!-- The heartbeat-frequency option defines the number of seconds between each
pair
of client heartbeat messages that are sent to the server. -->
<heartbeatfrequency>300</heartbeat-frequency>
<!-- The LCE client provides the option of periodically sending a log file
containing
performance statistics to the LCE server. The following option determines the
number of minutes between each performance statistics report. When the next
line
is commented out or removed, performance reporting is disabled. -->
<statistics-frequency>60</statistics-frequency>
<!-- LCE clients can compress log data prior to sending it to the LCE server,
saving bandwidth.
For debugging purposes, event packet compression may be disabled, but this
will
increase the bandwidth required to send data from LCE clients to the LCE
server.
Setting the following option to 0 will disable compression only during
transmission. -->
<compress-events>1</compress-events>
</options>
To configure the LCE Splunk Client to receive messages from the Splunk Indexer, a “listen” port must be chosen. The
default listen port is 9800, and it will be left unchanged from the default in this example.
<listen-port>9800</listen-port>
If the splunk-server IP address is different from the local host IP address that is in the default policy, it can be changed
by editing the splunk-server tag. The local host IP address works for this example since the LCE Splunk Client is
located on the Splunk Indexer.
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
6
<splunk-server>127.0.0.1</splunk-server>
If you install the LCE Splunk Client on the Splunk Indexer, changes to the LCE Splunk Client policy file will not be required.
Importing the Edited Policy File
If the LCE Splunk Client policy file required modification, follow the steps below to upload and apply the modified policy.
Select the “lcesplunk” client from the list of clients, and choose “Assign Policy” from the “LCE Clients” menu.
The “Import LCE Client Policy” menu will be displayed. Select “Browse” to start the policy upload process.
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
7
The next menu for file upload will be displayed. Select “Browse” to locate the policy file, and after the file has been
chosen, click “Upload”.
Finally, select “Import” to complete the policy upload.
Choose the policy from the “Assign Policy” menu, and select “Assign”.
The new policy will now be in effect on the chosen LCE Splunk Client.
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
8
Configuring Splunk Indexer to Forward Data
To allow the Splunk Indexer to forward data to the LCE Splunk Client, first choose “Settings” from the main menu of the
Splunk Indexer, and select “Forwarding and receiving”.
Next, select “Add new” from the “Configure forwarding” option.
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
9
Enter the IP address of the Splunk client along with the port number as shown below and select “Save”.
Once complete, the entry will be displayed as shown below in the “Forward data” section of Splunk.
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
10
To complete the configuration of Splunk, access the Splunk Indexer as the root user and edit the file
“/opt/splunk/etc/system/local/outputs.conf”. The lines added to the file are highlighted below:
[tcpout]
defaultGroup = default
disabled = 0
indexAndForward = 1
[tcpout-server://LCE_IP_OR_Hostname:9800]
[tcpout:default]
disabled = 0
server = LCE_IP_OR_Hostname:9800
sendCookedData = false
A restart of the Splunk services will be required after the “outputs.conf” file has been edited.
Data Comparison
An example of the data shown in Splunk is shown below. The example shown contains search results for a Cisco ASA
firewall. The exact search used narrowed the results to “sourcetype=syslog”, and matched the text string “%ASA”.
The same type of log information is available in SecurityCenter. When a user logs into SecurityCenter, there can be
multiple dashboards available that display pertinent information for that user. It is possible to set a specific collection of
dashboards as the default view in SecurityCenter. Examples of dashboards that can be created for events that are
collected by the LCE Splunk client are shown below.
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
11
The dashboard shown above contains the dashboard named “NormalizedEvent Types Collected by Splunk”. Select the
arrow “>” beside that dashboard to view all the information available.
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
12
The dashboard above includes the Cisco ASA Firewall events and all event types in a normalized format that is easy to
interpret. There are several views that you can select from the “Event Analysis” window that can be displayed by
selecting “Normalized Event Summary”. A similar view can be displayed in Splunk by selecting the “Raw Syslog
Events” option.
It is also possible to filter the “Normalized Event Summary” along with any other summary view by selecting the double
Arrow “>>” at the top left of the window. The text string “%ASA” used in the Splunk search above could be entered under
“Syslog Text”.
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
13
For More Information
Tenable has produced a variety of documents detailing the LCE’s deployment, configuration, user operation, and overall
testing. These documents are listed here:

Log Correlation Engine 4.2 Architecture Guide – provides a high-level view of LCE architecture and supported
platforms/environments.

Log Correlation Engine 4.4 Administrator and User Guide – describes installation, configuration, and operation of
the LCE.

Log Correlation Engine 4.4 Quick Start Guide – provides basic instructions to quickly install and configure an LCE
server. A more detailed description of configuration and management of an LCE server is provided in the “LCE
Administration and User Guide” document.

Log Correlation Engine 4.2 Client Guide – how to configure, operate, and manage the various Linux, Unix,
Windows, NetFlow, OPSEC, and other clients.

LCE 4.4 High Availability Large Scale Deployment Guide – details various configuration methods, architecture
examples, and hardware specifications for performance and high availability of large scale deployments of
Tenable's Log Correlation Engine (LCE).

LCE Best Practices – Learn how to best leverage the Log Correlation Engine in your enterprise.

Tenable Event Correlation – outlines various methods of event correlation provided by Tenable products and
describes the type of information leveraged by the correlation, and how this can be used to monitor security and
compliance on enterprise networks.

Tenable Products Plugin Families – provides a description and summary of the plugin families for Nessus, Log
Correlation Engine, and the Passive Vulnerability Scanner.

Log Correlation Engine 4.2 Log Normalization Guide – explanation of the LCE’s log parsing syntax with extensive
examples of log parsing and manipulating the LCE’s .prm libraries.

Log Correlation Engine 4.4 TASL Reference Guide – explanation of the Tenable Application Scripting Language
with extensive examples of a variety of correlation rules.

Log Correlation Engine 4.0 Statistics Daemon Guide – configuration, operation, and theory of the LCE’s statistic
daemon used to discover behavioral anomalies.

Log Correlation Engine 3.6 Large Disk Array Install Guide – configuration, operation, and theory for using the LCE
in large disk array environments.

Example Custom LCE Log Parsing - Minecraft Server Logs – describes how to create a custom log parser using
Minecraft as an example.
Documentation is also available for Nessus, the Passive Vulnerability Scanner, and SecurityCenter through the Tenable
Support Portal located at https://support.tenable.com/.
There are also some relevant postings at Tenable’s blog located at http://www.tenable.com/blog and at the Tenable
Discussion Forums located at https://discussions.nessus.org/community/lce.
For further information, please contact Tenable at [email protected], [email protected], or visit our web site at
http://www.tenable.com/.
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
14
About Tenable Network Security
Tenable Network Security provides continuous network monitoring to identify vulnerabilities, reduce risk, and ensure
compliance. Our family of products includes SecurityCenter Continuous View™, which provides the most comprehensive
and integrated view of network health, and Nessus®, the global standard in detecting and assessing network data.
Tenable is relied upon by more than 24,000 organizations, including the entire U.S. Department of Defense and many of
the world’s largest companies and governments. We offer customers peace of mind thanks to the largest install base, the
best expertise, and the ability to identify their biggest threats and enable them to respond quickly.
For more information, please visit tenable.com.
Copyright © 2014. Tenable Network Security, Inc. All rights reserved. Tenable Network Security and Nessus are registered trademarks of Tenable Network Security, Inc.
15