Installation and User Guide
Transcription
Installation and User Guide
Installation and User Guide LogWatch Knowledge Module for BMC PATROL® Arackal Digital Solutions 240 Duncan Mill Road, Suite 301 Toronto, Ontario, Canada M3B 1Z4 Phone: 1-877-437-4933 or 416-703-1211 Fax: 416-703-2544 LogWatch Knowledge Module for BMC PATROL® Installation and User Guide Covering Version 3.2.1 Jan 14, 2005 Document revision 1.8 This document is published by Arackal Digital Solutions Copyright © 2002-2004 Arackal Digital Solutions All rights reserved Arackal Digital Solutions 240 Duncan Mill Road, Suite 301 Toronto, Ontario Canada M3B 1Z4 BMC Software, the BMC Software logos and all other BMC Software product or service names are registered trademarks or trademarks of BMC Software, Inc. in the USA and in other select countries. Contacting Arackal Digital Solutions You can contact our support department via e-mail, fax or phone. Website: http://www.arackal.com E-mail: [email protected] Telephone: (416) 703-1211 or 1-877-437-4933 Fax: (416) 703-2544 Contents CONTENTS....................................................................................................................................................III FIGURES......................................................................................................................................................... V TABLES..........................................................................................................................................................VI ABOUT THIS GUIDE..................................................................................................................................... 1 WHO SHOULD READ THIS GUIDE ................................................................................................................... 1 HOW THIS GUIDE IS STRUCTURED ................................................................................................................. 1 MARGIN NOTE ICONS ..................................................................................................................................... 2 RELATED PUBLICATIONS ................................................................................................................................ 2 CHAPTER 1. INTRODUCTION ................................................................................................................... 3 FEATURES ....................................................................................................................................................... 3 CHAPTER 2. GETTING STARTED............................................................................................................. 5 SYSTEM AND SOFTWARE VERIFICATION......................................................................................................... 5 INSTALLATION ................................................................................................................................................ 5 Console and Agent are on the same machine ............................................................................................ 6 Console and Agent are on different machines........................................................................................... 7 Installing Your License.............................................................................................................................. 8 CHAPTER 3. SETTING UP A LOG FILE MONITOR ............................................................................ 10 ADDING A LOG FILE MONITOR ..................................................................................................................... 10 DEFINING A SIMPLE LOG FILE MONITOR ...................................................................................................... 12 DEFINING THE SEARCH STRINGS TO MONITOR FOR ....................................................................................... 15 Notification Levels................................................................................................................................... 16 DEFINING THE EMBEDDED RECOVERY ACTIONS .......................................................................................... 20 DEFINING THE LOG FILE MONITORING SCHEDULE ....................................................................................... 21 MODIFYING A LOG FILE MONITOR ............................................................................................................... 23 DELETING LOG FILES FROM THE MONITORED LIST ...................................................................................... 26 CHAPTER 4. ADVANCED TOPICS........................................................................................................... 29 KM HIERARCHY MODEL .............................................................................................................................. 29 WORKING WITH GROUPS .............................................................................................................................. 30 MONITORING BY COMMAND EXECUTION ..................................................................................................... 31 Macro Variables for the Monitoring Command...................................................................................... 33 HOW TO MONITOR NT/2000 EVENT LOGS ................................................................................................... 34 Simply dumping an event log................................................................................................................... 35 Organization of the log dump.................................................................................................................. 35 Range of data to dump............................................................................................................................. 35 EMBEDDED RECOVERY ACTIONS ................................................................................................................. 37 Macro Variables for Embedded Recovery Actions.................................................................................. 38 GENERATED PEM EVENTS ........................................................................................................................... 39 Case 1: String Match............................................................................................................................... 40 Case 2: Timeout....................................................................................................................................... 40 Case 3: Menu Command ......................................................................................................................... 41 Case 4: Group Member ........................................................................................................................... 41 Case 5: No data added ............................................................................................................................ 41 Case 6: Log File Found.......................................................................................................................... 42 Case 7: Log File Not Found.................................................................................................................... 42 Case 8: Mutual Exclusion List Match ..................................................................................................... 42 USING THE “ALL MATCHING FILES” MONITORING OPTION ......................................................................... 44 READING THE FILE FROM THE BEGINNING AT A SPECIFIED DATE/TIME .......................................................... 45 MATCHING USING MUTUAL EXCLUSION LIST OPTION.................................................................................. 47 HOW TO MONITOR PROTECTED FILES .......................................................................................................... 48 DISPLAYING THE CONTENTS OF A PROTECTED FILE ..................................................................................... 49 THE LOGWATCH INI FILES .......................................................................................................................... 50 Changing the Location of the INI Files Directory................................................................................... 50 CHAPTER 5. MENU SUMMARY.............................................................................................................. 52 LOGWATCH MENU COMMANDS:.................................................................................................................. 52 CHAPTER 6. INFOBOX ITEM SUMMARY............................................................................................ 55 LOGWATCH INFOBOX ITEMS: ...................................................................................................................... 55 CHAPTER 7. PARAMETER SUMMARY ................................................................................................. 57 LOGWATCH PARAMETERS: .......................................................................................................................... 57 PARAMETER NAME ....................................................................................................................................... 57 CHAPTER 8. FIELDS SUMMARY............................................................................................................. 62 APPENDIX A: INI CONFIGURATION FILE AND EXPLANATION .................................................. 67 Name/Value Pair Formats....................................................................................................................... 68 INDEX............................................................................................................................................................. 69 Figures FIGURE 1: LOGWATCH CONFIGURATION WIZARD WINDOW ........................................................................... 10 FIGURE 2: DEFINITION DETAILS WINDOW ....................................................................................................... 12 FIGURE 3: SEARCH STRINGS DEFINITION WINDOW ......................................................................................... 15 FIGURE 4: SEARCH STRINGS ATTRIBUTES WINDOW ........................................................................................ 17 FIGURE 5: SECTION OF SEARCH STRINGS ATTRIBUTES WINDOW WITH STRING LIST ....................................... 19 FIGURE 6: EMBEDDED RECOVERY ACTIONS WINDOW .................................................................................... 20 FIGURE 7: SCHEDULE WINDOW ....................................................................................................................... 22 FIGURE 8: MODIFY LOG FILE WINDOW ........................................................................................................... 23 FIGURE 9: CONFIRMATION WINDOW 1............................................................................................................. 26 FIGURE 10: DELETE LOG FILE WINDOW .......................................................................................................... 26 FIGURE 11: LOGWATCH HIERARCHY MODEL .................................................................................................. 29 FIGURE 12: READ DATA SETTINGS WINDOW................................................................................................... 45 FIGURE 13: READ DATA SETTINGS WINDOW EXAMPLE .................................................................................. 46 Tables TABLE 1: LOGWATCH MONITORING COMMAND MACRO VARIABLES ............................................................. 33 TABLE 2: LOGWATCH EMBEDDED RECOVERY ACTION MACRO VARIABLES ................................................... 38 TABLE 3: LOGWATCH NOTIFICATION LEVEL TO PEM EVENT TYPE MAPPINGS .............................................. 39 TABLE 4: SUMMARY OF MENU COMMANDS ..................................................................................................... 53 TABLE 5: LOGWATCH INFOBOX ITEMS ............................................................................................................ 56 TABLE 7: ALARMSTRINGSMATCHED PARAMETER ........................................................................................... 57 TABLE 8: CURRENTSIZE PARAMETER .............................................................................................................. 57 TABLE 9: GROWTHRATE PARAMETER.............................................................................................................. 58 TABLE 10: LINESNOTMATCHED PARAMETER .................................................................................................. 58 TABLE 11: LOGFILESTATUS PARAMETER ........................................................................................................ 58 TABLE 12: NOTIFYSTRINGSMATCHED PARAMETER......................................................................................... 59 TABLE 13: OKSTRINGSMATCHED PARAMETER .............................................................................................. 59 TABLE 14: SIZEOFLINESMATCHED PARAMETER ............................................................................................ 59 TABLE 15: TIMEBETWEENUPDATES PARAMETER ........................................................................................... 59 TABLE 16: TOTALSTRINGSMATCHED PARAMETER .......................................................................................... 60 TABLE 17: WARNSTRINGSMATCHED PARAMETER .......................................................................................... 60 TABLE 6: WATCHERLOGCOLL PARAMETER ..................................................................................................... 60 TABLE 18: EXTRAFILESLIST PARAMETER ........................................................................................................ 60 TABLE 19: FIELDS FOR DEFINITION DETAILS WINDOW .................................................................................... 62 TABLE 20: FIELDS FOR READ DATA SETTINGS WINDOW ................................................................................. 63 TABLE 21: FIELDS FOR SEARCH STRINGS WINDOW ......................................................................................... 63 TABLE 22: FIELDS FOR SEARCH STRING DEFINITION WINDOW ........................................................................ 64 TABLE 23: FIELDS FOR EMBEDDED RECOVERY ACTIONS WINDOW ................................................................. 65 TABLE 24: FIELDS FOR LOG FILE SCHEDULE WINDOW .................................................................................... 66 About This Guide This guide provides details on the installation, configuration, and use of the LogWatch Knowledge Module (KM) for BMC PATROL®. It outlines the application classes, menus, InfoBox commands, and parameters provided by the KM. This guide also provides details on performing the installation on both NT/2000 and UNIX platforms and how to configure the KM. It is useful to note that all screen shots present in this guide were taken from the Windows NT/2000 version of the KM. There should be no functional differences between these screens and those found in the UNIX version. Who Should Read This Guide This guide is intended for use by system and application administrators who are responsible for installing, and configuring PATROL and the LogWatch KM. It is somewhat technical in nature and assumes a fundamental knowledge of the Patrol architecture as well as familiarity with knowledge modules, console navigation, and agent configuration. How This Guide Is Structured Chapter Chapter 1. Introduction Chapter 2. Getting Started Chapter 3. Setting up a Log File Monitor Chapter 4. Advanced Topics Chapter 5. Menu Summary Chapter 6. InfoBox Item Summary Chapter 7. Parameter Summary Chapter 8. Fields Summary Appendix A: INI Configuration File Contents Introduces the Knowledge Module Provides information about system requirements to use the KM, installation and licensing procedures. How to use the KM and configuring Log File monitors Tips and tricks to get the most out of the KM Outlines the menu commands provided by the KM Outlines the InfoBox items provided by the KM Provides an overview of the parameters provided by the KM Offers description for all fields provided in wizard windows that define a Log file Offers a sample configuration file for reference LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 1 Margin Note Icons This manual makes use of notes in the left margin for presenting information that is useful or pertinent to the current discussion. The icons associated with these notes have the following meanings: Reference Example Note Warning New A cross-reference to material found elsewhere in this manual. An example of what was most recently discussed. A note, or information of general interest. Warns that caution should be exercised when performing the associated actions. Indicates new or enhanced functionality that was not present in the previous release. Related Publications All of the PATROL Installation Manuals, Release Notes, and Knowledge Module Guides are related to, and may be referenced in this guide. The NT/2000 or UNIX Console User Guide from BMC should be used as a reference for detailed information relating to the loading of Knowledge Modules, the setting of agent accounts, and items such as configuring the Agent access control list. Any suggestions made within this guide are based upon the expertise of the author. Any suggestions or comments regarding this manual or the product should be directed to [email protected]. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 2 Chapter 1. Introduction The first in a complete line of BITWatch products, LogWatch has all of the features required by PATROL users to monitor application and system log files across their UNIX and Windows NT/2000 based enterprise. Almost every application running within your enterprise writes error and debug information to a Log File. System and application administrators usually resort to creating scripts to parse these files with the intent of automatically emailing any problems to interested parties. This approach generally starts out well, but runs into the following issues: • There are many log files to monitor per server, requiring considerable effort to develop the scripts for each individual Log File. • Ongoing maintenance of the scripts usually becomes an issue, as changes need to be made to the scripts after they are built. • Administrators do not have the time to properly write and debug the scripts. • Having the scripts trigger recovery actions requires relatively complex script creation. • Cross-platform support is required, requiring more work. • The scriptwriter may leave the company or go on vacation, so support becomes an issue. By using the LogWatch KM for PATROL, the headaches associated with monitoring many Log Files on many different servers, as well as getting the events delivered to the appropriate person, are reduced. Features Through an easy-to-navigate set of dialog boxes, anyone having access to a PATROL Console and appropriate security privileges can define, edit, and delete log files from the monitored list. Each Log File instance has its own configurable search criteria and optional recovery actions. Major features include: • Simple, wizard based configuration, with all configuration information being stored in an external ASCII INI file. • Four levels of string match notification (ALARM, WARN, NOTIFY and OK) with an OK match automatically resetting any current ALARM or WARN condition. • Multiple string matches per notification type. • Optional pop-up notification windows upon string matches activated when a search string generates a change of state. • Improved per string search criteria allowing for individual string settings for case sensitive and match inversion • Support for Match N times within M minutes criteria. • The ability to set time ranges for each individual day of the week for which the KM should monitor a given Log File. • Supports grouping of related Log Files (fully compatible with ProcWatch and AgentWatch groups) with the sending of Group based events. • Polling intervals that can be defined on a per-Log File basis. • The ability to produce an ALARM if no new data is added to the Log File within a configurable period of time. • The ability to set a different icon for each instance. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 3 Reference Embedded recovery actions are covered further in Page 37. New licensing: You will need to request new licenses if you are upgrading from a LogWatch version that is less than version 3. • The capability to easily assign a single "embedded" OS recovery action to each state, without the need to modify and redistribute the KM file. • The ability to assign and execute a single “No Match” OS recovery action for every new line added to the Log File that does not match any search string criteria. • The KM has been coded in a completely OS independent manner (no need for external binaries like tail or cat). • Simple integration into any existing PATROL/Helpdesk event management system. • The ability to specify that all the files that match a pattern in a specified directory should be monitored. • Useful Info Boxes displaying pertinent per-Log File statistics and configuration information. • The ability to assign a monitoring command (command to be executed per instance that generates the file to be monitored). • The ability to monitor NT/2000 event logs using the monitoring command feature in combination with utilities like ELDump (application that dumps NT/2000 event logs into text files). • The ability to parse: only new data added, entire log file or a combination of the previous (parse new data and parse entire file at a given date/time). • The ability to use comparison types in the search string definition, other than just regular expression matching. • The ability to define a mutual exclusion list of strings per log file to be used as the search condition. • New MUXListMatch event class that provides the ability to know when a line of data in the logfile does not match any of the strings (PSL. Regular Expressions) that are part of the Mutual Exclusion List. • StringMatchFound event description now provides the pattern matched. • LogFileFound event class that provides the ability to know when a logfile does not exist. • LogFileNotFound event class that provides the ability to know when a previously non-existing file is created. • Parameters that provide extra information about number of string matches: AlarmStringsMatched, NotifyStringsMatched, OKStringsMatched and WarnStringsMatched • The ability to know the delta time for logfile updates, provided by the TimeBetweenUpdates parameter. • The ability to define a column, row or column/row range per log file instance to be used for the search conditions. • The ability to define column separator other than white spaces. • Ability to use the previous line to the matched use in the Embedded Recovery Actions. • The ability to specify when to execute an Embedded Recovery Action. • Licensing has changed. If you are an existing LogWatch user wanted to upgrade you will need a new license to activate LogWatch. If your support contract is up to date, you are entitled to a free upgrade. For further details and to request a license please contact our support department at: [email protected] LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 4 Chapter 2. Getting Started This chapter provides a checklist of items to consider before installing the LogWatch KM. The steps to follow for installation in both the Windows NT/2000 and UNIX environments are also provided. System and Software Verification The LogWatch Knowledge Module is written completely in PSL and therefore is completely platform independent. LogWatch has been extensively tested on Windows NT/2000 and UNIX platforms using Patrol version 3.4 and higher. This KM will not currently run on OpenVMS, MVS, or OS/2 platforms. Supported OS: Windows NT/2000 Solaris 2.7, 2.8, 2.9 HP-UX 11.11 AIX 4.3 Tested on following Patrol versions: Patrol 3.4 Patrol 3.5 Resource requirements: • Disk space required for install: 450k (UNIX), 400k (Windows NT/2000) • Estimated disk space required/day: 5 instances, no annotations 120k • Estimated memory required per instance: 4k Although the Knowledge Module will run anywhere, the format for PATROL icons and parameter help files is different between Windows NT/2000 and the different flavours of UNIX. To combat this, there are two separate distributions available - a self-extracting archive for Windows NT/2000, and a compressed tar file for UNIX. Installation Installing the LogWatch KM involves: 1. Copying the distribution files to the appropriate places in the Patrol directory hierarchy. 2. Loading the KM into your PATROL Console. 3. Installing the KM license in the corresponding Agent(s). 4. Configuring the Agent to load the KM automatically upon startup (optional). The process to follow to perform these steps is different depending on where the PATROL Console and PATROL Agent are installed. This section will explain in detail how to install the KM if: Console and Agent are on the same machine. Console and Agent are on different machines. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 5 Console and Agent are on the same machine 1. Run the distribution self-extracting ZIP archive for Windows NT/2000 or uncompress and untar the distribution for UNIX as follows: Unpacking the Windows NT/2000 distribution Run the distribution's self-extracting ZIP archive (Logwatch_<ver#>_win_dist.exe), where <ver#> is the version for the current distribution. Press the Unzip button. When prompted with Unzip to Folder, type in the path to PATROL_HOME folder, the path where patrol is installed (e.g. D:\Program Files\BMC Software\Patrol3). All LogWatch files required for both the NT/2000 Patrol Agent and the NT/2000 Console will be installed in their proper locations in your Patrol directory hierarchy. Unpacking the UNIX distribution Copy the LogWatch_<ver#>_unix_dist.tar.Z file into the $PATROL_HOME folder, the folder where patrol is installed (e.g. /opt/bmc/Patrol3.4, /opt/bmc/Patrol3.5). Uncompress and untar the distribution using: <prompt>% uncompress LogWatch_<ver#>_unix_dist.tar.Z <prompt>% tar -xvf LogWatch_<ver#>_unix_dist.tar All LogWatch files required for both the UNIX Patrol Agent and the UNIX Console will be installed in their proper locations in your Patrol directory hierarchy. 2. Load the LOGWATCH_LOAD KML file by selecting the Load KM … menu item from the File menu on your Patrol Console and picking the LOGWATCH_LOAD.kml file. 3. If you have not yet obtained/installed a valid license, you will see the following message displayed in your Patrol Console: ERROR (LOGWATCH)==>pconfig license variable "/LOGWATCH/License" not found. You will not see any LogWatch icons in the Patrol Console as long as you have an invalid license, but the BITWATCH container icon will be present (the BITWATCH icon is orange with a magnifying glass in it). Simply request a demo license by e-mailing [email protected] and install the license as outlined in the “Installing Your License” section below. 4. If you want to configure the PATROL Agent to load the KM automatically upon startup, select the Add to Preloaded KMs menu item from the Admin menu of LOGWATCH_SETUP icon, this will modify the Agent’s /AgentSetup/preloadedKMs configuration variable and LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 6 adds “LOGWATCH_LOAD.kml” to it for you. The PATROL Agent must be restarted for the changes to take effect. Note that the username/password combination required for this step refers to a user having configuration privileges on the local PATROL Agent Console and Agent are on different machines If the PATROL Console and Agent are on separate machines, the installation should be performed twice – once on each machine. 1. Run the distribution self-extracting ZIP archive for the Console and the Agent on Windows NT/2000 or uncompress and untar the distribution for the Console and the Agent on UNIX as follows. Unpacking the Windows NT/2000 distribution Run the distribution's self-extracting ZIP archive (Logwatch_<ver#>_win_dist.exe), where <ver#> is the version for the current distribution. Press the Unzip button. When prompted with Unzip to Folder, type in the path to where Patrol is installed (PATROL_HOME) (e.g. D:\Program Files\BMC Software\Patrol3). The LogWatch files are now installed into their appropriate places in the PATROL directory hierarchy. Note: You must do this on the computer where the Console is running and for every Windows NT/2000 computer where you wish to run the KM. Unpacking the UNIX distribution Copy the LogWatch_<ver#>_unix_dist.tar.Z file into your $PATROL_HOME directory. Uncompress and untar the distribution using: <prompt>% uncompress LogWatch_<ver#>_unix_dist.tar.Z <prompt>% tar -xvf LogWatch_<ver#>_unix_dist.tar The LogWatch files are now installed into their appropriate places in the PATROL directory hierarchy. Note: You must do this on the computer where the Console is running and for every UNIX machine where you wish to run the KM. 2. On the PATROL Console, load the LOGWATCH_LOAD KML file by selecting the menu item Load KM … from the File menu and picking the LOGWATCH_LOAD.kml file. 3. If you have not yet obtained/installed a valid license, you will see the following message displayed in your Patrol Console: ERROR (LOGWATCH)==>pconfig license variable "/LOGWATCH/License" not found. You will not see any LogWatch icons in the Patrol Console as long as you have an invalid license, but the BITWATCH container icon will be present (the BITWATCH icon is orange with a magnifying glass in it). Simply obtain a demo license by e-mailing LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 7 [email protected], and install the license as outlined in the “Installing Your License” section below. 4. If you want to configure the PATROL Agent to load the KM automatically upon startup, select the Add to Preloaded KMs menu item from the Admin menu of LOGWATCH_SETUP icon, this will modify the Agent’s /AgentSetup/preloadedKMs configuration variable and adds “LOGWATCH_LOAD.kml” to it for you. The PATROL Agent must be restarted for the changes to take effect. Note that the username/password combination required for this step refers to a user having configuration privileges on the remote PATROL Agent Alternative Approach: The above steps must be done for each Agent you have where you want to install the KM. Alternatively, you can install the LogWatch KM onto your local machine, load the LOGWATCH_LOAD.kml file into your console, connect to each Agent as a developer, and committing the currently loaded KMs out to all the Agents connected. This will also commit out any KMs, bins, psls, and libs that you have recently modified, so care must be taken if this alternative approach is used. Note also that Console installations will not be updated by the commit; these must be done using the manual method outlined above. Installing Your License Note Invoking the appropriate menu command from the new BITWATCH KM icon now allows for one-step license key installation. Installing your license must be done before the LOGWATCH KM will start. Follow the simple steps below to install your license. 1. Select the Install LogWatch License menu from the BITWATCH icon (the orange icon with the magnifying glass in it) 2. You will be prompted to enter a username and password. If you have the PATROL Agent's accessControlList configuration variable set, enter the username and password of the user allowed configuring the PATROL Agent when prompted. Otherwise, just enter any username/password combination. 3. You will then be shown your Company name, as it appears in your PATROL license, and be prompted for the license key. If you are currently using a Demo copy of PATROL, the Company name shown will reflect this. 4. Enter the license key as it appears in the Email or FAX that you received. Press the OK button when done. 5. If the license is successfully written into the PATROL Agent’s configuration database, a dialog box will appear shortly indicating that the license has been installed. If not, note the error returned and try it again (this will happen if the username/password combination you’ve entered is incorrect). Contact our support department at [email protected] if you can't resolve the license error. 6. The KM will automatically check the license shortly after and the LOGWATCH and LOGWATCH_SETUP icons will appear underneath the BITWATCH icon on your PATROL Console. Where to go from here… To… Adding, modifying, deleting Log File monitors … LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved Proceed to… Pg. 10 Chapter 3. Setting up a Log File Monitor 8 How to get the most out of the KM … Descriptions of the menus provided and related functions … Detailed explanation for each of the InfoBox fields … Attributes and meaning of the KM Parameters … Description of the fields for each wizard window including default values and format expected … LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved Pg. 28 Chapter 4. Advanced Topics Pg. 52 Chapter 5. Menu Summary Pg. 54 Chapter 6. InfoBox Item Summary Pg. 57 Chapter 7. Parameter Summary Pg. 62 Chapter 8. Fields Summary 9 Chapter 3. Setting up a Log File Monitor This chapter provides step-by-step instructions on how to use the LogWatch KM to monitor your log files. It walks you through the entire wizard and includes sections for each wizard page. You will find instructions for adding, modifying and deleting log file monitors as well. Adding a Log File Monitor Note BMC recommends a maximum of 50 instances per KM. Note By default, the LW configuration files will be located in $PATROL_HOME/config (UNIX) or %PATROL_HOME/config % (Windows) and will have a name resembling lw<port#>-<instance>.ini (the <port#> portion will be replaced with the port number that your Agent is running on the <instance> portion is replaced by the instance label. You start monitoring a Log File by adding it to the LogWatch KM's monitored list. The LogWatch KM will allow you to add up to 200 Log File monitors, but various elements including memory, CPU, and polling frequency may reduce this limit. To add a Log File monitor, you typically define the monitor details, the search strings, the recovery actions and schedule. Each of these steps is described in below in its corresponding section. 1. Right-click your mouse on the LOGWATCH_SETUP icon and select the Logfile Maintenance menu pick from the Admin Menu. When prompted for a username/password combination, enter in the information for a user that has write permissions for the LogWatch configuration files. 2. If you have entered the username/password combination correctly, the KM will display the LogWatch Configuration Wizard Window, shown below in Figure 1: LogWatch Configuration Wizard Window. Figure 1: LogWatch Configuration Wizard Window Otherwise, a popup message will be displayed indicating the condition. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 10 New for v3! The KM creates one configuration file per process(es) definition. Starting with LogWatch version3, the KM uses a configuration (INI) file per logfile definition. If the INI file did not exist before you loaded the LogWatch KM into the Patrol Agent, the KM will create it when the log file monitor is defined, so it will be owned by the Patrol Agent’s /AgentSetup/defaultAccount, which is most likely patrol. For any pre-existing configuration files generated by LogWatch previous to version 3, the KM will backup the original ini file and try to convert into many mini-INIs. If you have any issues with the conversion feel free to contact our support department at [email protected]. Note: do not delete the old ini file backup until all your definitions have been split into mini-INIs successfully. 3. Ensure that the Add a new Log File definition action is selected and press the Next > button. After a short wait, the LogWatch Configuration Wizard Window will be replaced with the Definition Details Window, shown in Figure 2. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 11 Defining a Simple Log File Monitor This section provides the steps for setting the basic Log File attributes from the Definition Details Window. It assumes you followed the previous steps to get to the Definition Details Window, shown in Figure 2. Figure 2: Definition Details Window 1. Enter the full name of a file you wish to monitor into the File Name field (this name must include both the path and the name, with the path separation character used being the appropriate one for the OS). LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 12 The wildcard character (“*”) is allowed in the filename if you: New Logfile names can now contain wildcards Example Acceptable File Names for UNIX and WinNT want to monitor a log file whose name changes in some random manner. OR want to monitor all the files matching the specified pattern within a specified directory. File Name: /var/adm/messages or /var/log/syslog* File Name: c:\pagefile.sys or c:\temp\test*.log 2. If you want the KM to generate the log file dynamically as result of a command execution, assign a value to: Command to generate this file. If this field is not blank, the LOGWATCH KM will execute the given command every poll cycle. The resulting file will be parsed for string matches upon command completion. If the command requires a time range you can use the %FROM and %TO macros. For more details see the Monitoring by Command subsection on page 31. Note The command functionality cannot be combined with the file patterns or matching files monitoring options outlined above. Note Special characters (i.e. All "/", "\", and ":") are removed from the label prior to being saved. They are converted to "_" characters. 3. If you want to monitor all the files matching a specified pattern then enable the Monitor all matching files toggle button. If this option was enabled in the c:\temp\test*.log example above, all files in c:\temp that matched test*.log (such as test1.log, testsimple.log, etc) would have Logfile monitors automatically created for each of them. If this option is selected, the instance that you are currently configuring becomes a master instance and all matching monitors that are created will have the same monitoring attributes as this master. 4. If you wish, enter a name to use as the label for the instance into the Label field. This label will be placed under the Log File monitor’s icon on the Patrol Console and will be used in any PEM events that are sent out from the PATROL Agent (see the section entitled "Generated PEM Events" for more information on PATROL events). If you do not supply a label, the label Log’s File Name will be used with any special characters removed. The following cases are exceptions where the value in the Label field is ignored and a special label is created: If the monitor is a master, the label assigned follows the pattern: “MASTER(FileName)”, where FileName is the value set into the File Name field with any special characters removed. If the monitor is a child of a master the label assigned is “FileName”, where FileName is the actual name of the file that was found to match with any special characters removed. For more information about master and children instances, refer to Chapter 4. Advanced Topics, section Using the “All Matching Files” Monitoring Option 5. You may also change the icon that this Log File monitor uses for display on your Patrol console. By default, the "log" icon is used, but you are free to use any icons that are present in the $PATROL_HOME/lib/images (UNIX) or %PATROL_HOME%/lib/images (Windows) directory on your console host. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 13 Note that the icons in this directory have extensions that you should not include in the text you put into the Icon configuration field. The actual name of the "log" icon is "log_ok.bmp" (for Windows or "log_ok.xpm" if your console is installed on a UNIX server), but only the "log" part should be typed into the field. Example Acceptable icon names Icon: cron Icon: application (real name in the images directory: cron_ok.bmp) (real name in the images directory: application_ok.xpm) 6. If you wish, enter a name to use as the group for the instance into the Group Name field. You can use the group to assemble related Log File monitors underneath a single container in your PATROL Console. If the Group Name is not specified the instance will be created and displayed under the LOGWATCH application class. 7. By default the Auto-reset to OK Timer (mins:) is disabled and displays 0 minutes. You may want to change this value if you want the Log File monitor state to go back to OK automatically after a specific period of time. The "Generated PEM Events" section found later in this document provides information on PEM events that are sent when this timer fires. 8. By default the Alarm if no data Timer (mins:) is disabled and displays 0 minutes. You may want to change this value if you want the Log File monitor to go to ALARM state automatically if no new data is added to the Log File within a certain amount of time. The "Generated PEM Events" section found later in this document provides information on PEM events that are sent when this timer fires. Note: the default behavior is to monitor only new data added to the file every poll cycle. 9. If you want to change the data parsed by the KM, make your selection using the Each poll cycle, perform search on menu. 10. Ensure that all the information is correct and press the Next > button. After a short wait, the Search Strings Definition Window, shown in Figure 3 will replace the Definition Details window. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 14 Defining the Search Strings to monitor for This section provides the steps for adding, modifying and deleting strings from the list of strings that the KM will monitor for within the specified Log File. It assumes you followed the previous steps to get to the Search Strings Definition Window, shown below in Figure 3. LogWatch can generate 4 separate notification levels (ALARM, WARN, NOTIFY and OK) whenever a string match occurs. Several strings can be defined to generate the same notification level if necessary. The Search Strings Definition Window allows definition, modification and deletion of search strings. Figure 3: Search Strings Definition Window LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 15 The previous figure displays 2 groups of special fields: • The first group has 3 special fields: Column, Column Delimiter and Row. These fields can be defined only one per log file instance. They can be used to search only specific fragments of the log file. By default the KM verifies all the new data added to a file, but if you know the line(s) and/or columns you want to look for (in case the monitored file) has a pattern, you can use the Column/Row fields to filter only given chunks of the data. • The second group has the Consider these strings as being mutually exclusive Field. This field is unchecked by default, once it is checked it applies to all the string definitions for the given log file instance by making all the string definitions for the log file part of a Mutual Exclusion List. Mutual Exclusion list matches only compares strings (PSL Regular Expressions), ignores string individual definitions, etc. For more details see Matching using Mutual Exclusion List Option on page 47. 1. To add a search string, make sure the Add operation is selected and press the Next > button. After a short wait the Search String Attributes Window is displayed as in Figure 4. 2. By default a string is active, and you will see the String is Active toggle button checked. If you want to disable this particular string for any reason, uncheck the Active checkbox. Note Duplicate strings are not allowed within a single log file. Important! Previous versions of LogWatch allowed the use of empty strings to denote “match everything”. This version requires a regular expression such as “.*” to match everything. 3. If you want the search to be case sensitive, you should check the String is Case Sensitive toggle button. 4. If you want to invert the result of the search, you should check the Invert Search toggle button. 5. Type the string you want the KM to monitor for in the Search String field. A nonempty string must be defined. Strings can contain any regular expressions and combination of characters accept the sequence \; (the KM uses this combination internally). Duplicated strings within the same log file are not allowed. Search String: “error# *” Search String: “ shutdown “ Search String: “.*” (will match on “error#1”, “ error#35”, “ error#n”) (will match on “shutdown –y –g0 –i0”, “shutdown “) (will match on any string) 6. Select the state (ALARM, WARN, NOTIFY, OK) you want the KM to change into, if a string match occurs, by selecting the respective option from the State Change produced if string matches radio button. For the purpose of this example, we will select Warn. The four notification levels enable you the easily and effectively monitor your Log Files and provide alerts on only the string matches that demand them. Notification Levels There are four (4) notification levels: ALARM, WARN, NOTIFY, OK. Each of these serves a slightly different purpose, as outlined below: LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 16 Figure 4: Search Strings Attributes Window Note: alarm change of state is not generated if Mutual Exclusion List is enabled for the log file instance. • Alarm: Any new text coming into the Log File that matches the ALARM level monitoring attributes will change the state of the Log File instance to ALARM. A PEM event will be generated indicating the ALARM level match and the Embedded recovery action for the ALARM level will be executed. If further string matches are found and the Log File instance is still in ALARM, a state change will not occur, but the PEM event will still be sent and the Embedded recovery action will be executed again. Note: warn change of state is not generated if Mutual Exclusion List is enabled for the log file instance. • Warn: Any new text coming into the Log File that matches the WARN level monitoring attributes, will change the state of the Log File instance to WARN. A PEM event will be generated indicating the WARN level match and the Embedded recovery action for the WARN level will be executed. If further string matches are found and the Log File instance is still in WARN, a state change will not occur, but the PEM event will still be sent and the LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 17 Embedded recovery action will be executed again. If an OK or ALARM level match occurs subsequent to the WARN match, the state will be changed to reflect the new match level. Note: when Mutual Exclusion List is enabled the event generated is always a Notify event. Note: OK change of state is not generated if Mutual Exclusion List is enabled for the log file instance. • Notify: This level is used when you want to be aware of something occurring, but you do not want the Log File instance to change state. When new text coming into the Log File matches the NOTIFY level monitoring attributes, a PEM event will be generated and the Embedded recovery action for the NOTIFY level will be executed. • OK: Any new text coming into the Log File that matches the OK level monitoring attributes will change the state of the Log File instance to OK. A PEM event will be generated indicating the OK level match, and the Embedded recovery action for the OK level will be executed. Any new WARN or ALARM level match will cause the Log File instance to change to the appropriate state, as expected. 7. You will want to change the value of the Consecutive matches to state field if you want the change of state produced (WARN in this case) to happen only after finding N matches. If you select 5 for example, the WARN state condition will be generated only after 5 matches of the string are found within the Log File being monitored. Consecutive matches to state is set to 1 by default. 8. The value of the Matches must occur within(mins) field is set to the number of minutes in which the N Consecutive matches to state must occur. Continuing our example, setting this value to 2, would result in the WARN state condition being generated only after 5 consecutive matches of the string are found in the Log File within a period of 2 minutes. Matches must occur within(mins) is set to 1 minute by default. Note Lots of string matches result in lots of popup boxes! Note Some settings are ignored if the string is part of the Mutual Exclusion List (Mutual Exclusion is ON) 9. The Annotate in graph? option is used to indicate whether or not an annotation is generated every time a match is produced for the given string. If you don’t want an annotation to be generated uncheck it. Annotations hold the contents of the line that generated the state change and can be viewed by double clicking on the “*” symbol in the LogFileStatus parameter graph for the given monitor instance. 10. The When to PopUp? field dictates when pop up message is to be displayed on the Patrol Console every time a string match is found. This functionality is disabled by default (never). If Mutual Exclusion is enabled for the given log file monitor, the settings (values) for the following fields are ignored: State Change produced if string matches, Consecutive matches to state, Matches must occur within(mins), Annotate in graph?, When to PopUp? For more details see Matching using Mutual Exclusion List Option on page 47. 11. Verify that all the information for the search string defined is correct and press the Next > button. After a short wait, the Search String Attributes Window is replaced by the Search Strings Definition Window, shown in Figure 5: Section of Search Strings Attributes Window with String List LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 18 The search string that you just defined is displayed in the list of Current String Definitions and will display something similar to: W,Y=>"my search string" Where W stands for “WARN” and Y indicates that this string is currently set to active. Rather than W, the state indicator can be any of “A” for ALARM, “W” for WARN, “N” for NOTIFY and “O” for OK. The active indicator can be either “Y” or “N”. The double quotes around the search string help identify blank spaces at the beginning or end of the string. Note that leading or trailing blanks in the search strings are treated as part of the string. Example Leading or trailing spaces in search strings are significant W,Y=>”shutdown ” (trailing space is significant here) W,Y=>”shutdown“ Note You can cancel a Search String definition by pressing the < Back button if you are on the Search String Attributes Window Figure 5: Section of Search Strings Attributes Window with String List Note The contents of the Log File will not be monitored until you have defined at least one active search string. 12. Using the Search Strings Definition Window, you can keep modifying the string list by Adding, Modifying or Deleting strings with the selection of the corresponding option; the Done option is always selected by default if at least one string has been defined. To finish entering search strings and proceed with the rest of the Log File definition, select the Done operation and then click the Next > button. The Embedded Recovery Actions Window will appear (shown in Figure 6). LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 19 Defining the Embedded Recovery Actions This section provides the steps for defining the Embedded Recovery Actions for each of the states (ALARM, WARN, NOTIFY and OK) that can be generated upon a string match and No Match Recovery Action. It assumes you followed the previous steps to get to the Embedded Recovery Actions Window. Figure 6: Embedded Recovery Actions Window 1. Each search string that has been defined for the current log file monitor generates a notification message whenever it is matched. LogWatch can be instructed to execute an OS command whenever a string match occurs – these OS commands are setup by defining Embedded Recovery Actions. Embedded Recovery Actions are completely separate from the PATROL Agent’s normal recovery actions and are stored in the external configuration file with the rest of your Log File definition, rather than in the KM file. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 20 Five Embedded Recovery Actions can be defined: 1. Alarm Embedded Recovery Action – if you wish type an Operating System command to be executed when an ALARM match is found. 2. Warn Embedded Recovery Action – if you wish type an Operating System command to be executed when a WARN match is found. 3. Notify Embedded Recovery Action – if you wish type an Operating System command to be executed when a NOTIFY match is found. 4. OK Embedded Recovery Action – if you wish type an Operating System command to be executed when an OK match is found. 5. No Match Embedded Recovery Action – if you wish type an Operating System command to be executed every time a new line does not match any of the search strings currently configured. The When to Execute field is provided for each Recovery Action and defines when the KM has to execute the Recovery Action, by default all the Embedded Recovery Actions are executed every time the condition is met(always). Using our same string example, we know there is a string defined that can generate a WARN condition if a match happens, to take advantage of the Embedded recovery actions, we define a Warn Embedded Recovery Action that can append a message to an output file: Example A simple Embedded Recovery Action Note Only Notify Embedded Recovery Action is executed for Mutual Exclusion Lists echo “DB is shutting down” >> /tmp/DBoutput.txt To insert dynamic match related information into the your Embedded Recovery Actions, use the built-in MACRO variables (these are discussed in the Advanced Topics section found later in this document). For your convenience the View Macro List button is provided in the wizard window, which displays a popup list of the available MACRO variables. If Mutual Exclusion is enabled for the given log file monitor the KM will generate only Notify events every time a line does not match the Mutual Exclusion List and therefore Notify Embedded Recovery Action is the only field that will be taken in consideration and executed. For more details see Matching using Mutual Exclusion List Option on page 47. 2. Click the Next > button to define the monitoring schedule for this monitor. Defining the Log File Monitoring Schedule This section provides the steps for defining the monitor schedule. It assumes you followed the previous steps to get to the Schedule Window, shown below in Figure 7: Schedule Window. 1. The Schedule Window is the last configuration window. Here you can set the Log File monitor scheduling attributes. The fields: From, To and Invert are provided for each day of the week. You can set these fields to tell the monitor specifically when it is to start and stop monitoring the associated log. The Invert checkbox indicates whether or not the Log File should “be monitored” (normal) or “not be monitored” (inverted) during the associated time range. If, for example, you wish to monitor this Log File definition on Sundays from 9:30 AM to 4:45 PM, the entire day from Monday through Friday, and not monitor from 7:00AM to 8:00 PM on LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 21 Saturday, the Schedule Window would be setup to look like the one in Figure 7: Schedule Window. Figure 7: Schedule Window By default all the From fields are 00:00:00 and all the To fields contain 23:59:59 values, meaning “all hours of the day”. Note the time fields are spinners where the digits represent hours (in 24 hour time), minutes and seconds respectively. 2. The last field in this window is the Poll Interval for the Log File monitor. The default value is 10 minutes. The shortest poll interval supported by the LogWatch KM is 30 seconds. 3. Verify the Schedule information and press the Done button to complete this Log File monitor definition. You will get a confirmation message window asking if you are sure you want to make the configured changes. Pressing the Yes button here does the following: your new definition is written to the INI file LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 22 a new instance is created on the Patrol Console and will be using the label, icon, group and other information that you have assigned to it monitoring of the various Log File parameters (i.e. Log File size, and growth rate) begins, assuming that the Active checkbox in the Definition Window was not unchecked. Note: on a busy machine, it may take some time for LogWatch Configuration Wizard to redisplay. 4. You will now be confronted with the first wizard page once again. Here you have an opportunity to add more Log Files, Modify your current monitoring attributes, Delete Existing Log Files or to quit. 5. Press the Done button to complete your definition session. Modifying a Log File Monitor This section assumes you have at least one Log File being monitored, so you can proceed to modify the monitor definition. The following steps will guide you through modifying the definition for an existing Log File monitor. The steps for configuring each of the notification levels involve the exact same procedure as the one presented below for the ALARM level. 1. Open the LogWatch Configuration Wizard Window, as outlined in the Adding a Log File Monitor section found earlier in this document. Select Modify an existing Log file definition operation and press the Next > button. After a short wait, the Modify Log File Window appears displaying the defined Log Files in the corresponding list, see Figure 8 below. The Current Log File Definitions list shows the label, the file name and group for each monitor configured. Figure 8: Modify Log File Window LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 23 2. Select the "Agent Errors" monitor from the Current Log File Definitions and click the Next > button to bring up the Definition Details Window. Example Modifying the Log File monitoring attributes 3. Modify the Log File definition to meet the following criteria: we want to set the icon to the "cube_ok.bmp" that is located in the %PATROL_HOME%/images directory look for the string "couldn’t" or the string "Error" every 60 seconds go into ALARM if we find either of these strings we have to find these at least 3 times in 5 minutes to consider this a problem worthy of ALARM status upon getting a match, email the Patrol administrator, Joe, indicating the problem we want to annotate any string matches that cause a state change to the ALARM state we only want to monitor for these strings during the week from 6:00pm and 8:00am To do this, we set the fields in the wizard windows as follows: Note Many of these fields will already be set if you have chosen to Modify an existing Log File definition In the Definition Details Window: - Active: checked - File Name: c:\patrol3\log\PatrolAgent-tor-bplanchart-3181.errs - Command to generate this file: <blank> - Monitor all matching files: <disabled> - Label: Agent Errors - Icon: cube - Group: <blank> - Auto-reset to OK Timer:0 - Alarm if no data Timer:0 - Each poll cycle, perform search on: New data only In the Search String Definition Window: - Column: <blank> - Column Delimiter: <blank> - Row: <blank> - Consider these strings as being mutually exclusive: checked In the Search String Attributes Window: (for the definition of “couldn’t”) - String is Active: checked - String is Case Sensitive: <disabled> - Invert Search: <disabled> - Type of comparison: PSL Reg. Exp. - Search String: couldn’t - State Change produced if string matches: Alarm - Consecutive matches to state: 3 - Matches must occur within(mins): 5 - Annotate in Graph?: checked - When to PopUp?: Always LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 24 In the Search String Attributes Window: (for the definition of “Error”) - String is Active: checked - String is Case Sensitive: checked - Type of comparison: PSL Reg. Exp. - Invert Search: <disabled> - Search String: Error - State Change produced if string matches: Alarm - Consecutive matches to state: 3 - Matches must occur within(mins): 5 - Annotate in Graph?: checked - When to PopUp?: Always Reference You’ll find more complicated examples of recovery actions in the “Embedded Recovery Actions” section In the Embedded Recovery Actions Window: - Alarm Embedded Recovery Action: echo “got an ALARM line match” | smtpsend –s “PATROL LogWatch Alarm” –h my_mail_server –r [email protected] - - When to Execute: Always Warn Embedded Recovery Action: <blank> When to Execute: Always Notify Embedded Recovery Action: <blank> When to Execute: Always OK Embedded Recovery Action: <blank> When to Execute: Always No Match Embedded Recovery Action: <blank> When to Execute: Always In the Schedule Window: - From( Monday through Friday): 18:00:00 - To( Monday through Friday): 8:00:00 - From( Sunday and Saturday): 00:00:00 - To( Sunday and Saturday):00:00:00 - Invert(all days of the week): <unchecked> - Poll Interval: 00:01:00 Note: Any time during the modification you can go back to the previous window by clicking the < Back button when available. After reaching the Schedule Window, click the Done button and then Yes from the Confirmation window; your changes are then written to the LogWatch INI file, and the monitoring of the Patrol agent error log file with the new configuration begins. You can cancel the Log File modification if you press No in the Confirmation Window (see Figure 9). If you press No, you may want to go back and verify/fix some values by pressing the < Back button or Cancel the entire operation and go to the LogWatch Configuration Wizard Window by pressing the Cancel button. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 25 Figure 9: Confirmation Window 1 Deleting Log Files from the Monitored List Deleting Log Files is as simple as adding them. The following steps will guide you through removing a Log File instance from the Patrol environment. 1. Open the LogWatch Configuration Wizard Window, as outlined in the "Adding a Simple Log File Monitor" section found earlier in this document. Select Delete an existing Log file definition operation and press the Next > button. 2. After a short wait, the Delete Log File Window appears displaying the currently defined Log Files monitors. Select the log file monitor you wish to delete and click Next > ; in our example we have selected the "Agent Errors" monitor. Figure 10: Delete Log File Window LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 26 Note that if you delete a master instance (i.e. a logfile monitor that has been configured with the “Monitor all Matching Files” option selected), all child monitors that have been created by the master will also be deleted. 3. After a short wait a Confirmation Window will popup. Pressing the Yes button will cause the KM to delete the specified Log File from the LogWatch INI file. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 27 Where to go from here… To… How to get the most out of the KM … Descriptions of the menus provided and related functions … Detailed explanation for each of the InfoBox fields … Attributes and meaning of the KM Parameters … Description of the fields for each wizard window including default values and format expected … LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved Proceed to… Pg. 29 Chapter 4. Advanced Topics Pg. 52 Chapter 5. Menu Summary Pg. 545 Chapter 6. InfoBox Item Summary Pg. 577 Chapter 7. Parameter Summary Pg. 622 Chapter 8. Fields Summary 28 Chapter 4. Advanced Topics This chapter presents some advanced LogWatch features and tips on getting the most out of the product. KM Hierarchy Model Once the LogWatch Knowledge Module is fully installed and configured, we have provided below in Figure 11 a hierarchical map of the relations between the application classes and instances. Figure 11: LogWatch Hierarchy Model Figure 11, below, presents graphically a list of the KMs that make up LOGWATCH, as well as a sample object hierarchy exhibited by a typical configuration. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 29 On the left side of the figure, the tree shows the relation between application classes and instances if a successful configuration: - You will notice that the LOGWATCH container is located under the BITWATCH container and, the LOGWATCH_SETUP icon is shown under LOGWATCH. In case of a first time installation when the logwatch configuration ini file does not exist or does not contain monitor definitions, this would be all the information displayed. - Given that a log file monitor has been created, an instance like the one named “file3all” is added under the LOGWATCH container. - As well, if a group has been defined, a container is created (if it does not exist) with the name specified in the group and the log file-monitoring instance is placed under that group. The “file1”monitor shown below is part of the “GROUP_CONTAINER” group. Working with Groups Starting with LogWatch version 1.2, the concept of logical groups has been realized, allowing you to assemble related Log File monitors underneath a single container in your PATROL Console. Note If you edit a Log File monitor and change its group, a new group icon will be created and the monitor will be moved to the new group. For example, say you have an application called “CustomApp” that you want to monitor. It sends debug information to a file called “/tmp/debug.out” and sends transaction information to “$APP_HOME/transaction.log”. To have both of these Log Files reside under a single application icon do the following: set up two independent Log File monitors, one for “/tmp/debug.out” and one for “$APP_HOME/transaction.log” (as outlined in Adding a Simple Log File Monitor) set the Group fields for both monitors to “CustomApp” A new application icon having the name “CustomApp” will be automatically be created on your PATROL Console and the two Log File monitors will be created under it. Note If you configure a master logfile monitor (i.e. a logfile monitor that has been configured with the “Monitor all Matching Files” option selected), to be part of a group, all child monitors that are created by the master are placed into that same group. The group feature is part of each of the BITWatch KMs – LogWatch, ProcWatch, and AgentWatch – enabling you to group related Log Files, processes, and agents together in a meaningful way; when your special group icon flashes, you know there is something wrong with the associated application. Being part of a group also affects the events that are generated by LogWatch. If a string match is found for one of the log file monitors in a group, LogWatch generates two PEM events – one indicating the match for the instance (as normal), and a second one indicating that string match for the group has been detected. This makes it much easier to perform event correlation based on the application alerts, rather than on individual log files. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 30 Monitoring by Command Execution In some cases, it is useful to be able to monitor files that are created dynamically by the execution of a command. Providing a monitoring command for the instance enables this type of monitoring. A monitoring command can be specified in the Definitions Details Window. The command is limited to OS commands only and executes once per instance every poll cycle. Any command that you would normally type on the command line (on the Patrol Agent host) can be used. Example A Monitoring Command on UNIX An example of a monitoring command on UNIX would be: du > /opt/user/dusage.txt This would result in the file dusage.txt created with the information about the disk usage every poll cycle. You could then configure LogWatch to watch for a specific directory name that is created only when a fatal error happens for an in-house application. Example A Monitoring Command on NT An example of a monitoring command on NT would be: dir /AH > c:\hiddenfiles.txt This would result in the file hiddenfiles.txt created with the information about the hidden files on c:\. You could then configure LogWatch to watch for a specific file extension that should not be hidden and, generate an alarm if there is at least one hidden file with that extension. The definition of an instance using a monitoring command adds power and flexibility to the LogWatch KM. This feature can be used to solve real life situations like the conversion of a binary file into a text file by an application that’s capable of doing so and, the utilization of LogWatch KM in order to execute the command that converts the given binary file into a text file on a regular basis and monitor the instance like any other Log File definition. Monitoring by command execution can be combined with utilities like ELDump, DumpEL or any other executable that can dump the content of the Windows NT/2000 event log into text files, adding the capability of monitoring NT/2000 Event Logs. For more about how to monitor NT/2000 Event Logs, look at the end of this subsection. Example Creating a Log File using monitoring by Command Execution In order to clarify how to monitor by command execution and take full advantage of this feature, let’s assume we want to create a Log File instance definition to meet the following criteria: we want to monitor the Windows NT/2000 event log we want to label the instance: NTSysLog look for the string "warning" every 10 minutes go into ALARM if we find the string above we want to annotate any string matches that cause a state change to the ALARM state LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 31 we want to popup a window on the console every time there is a string match that causes a state change to the ALARM state we want to monitor all the time To do this, we set the fields in the wizard windows as follows: Reference Macro variables available for the monitoring command are discussed below Reference ELDump is a free application that dumps the contents of a NT event log as text, for more info. about ELDump: http://www.ibt.ku.dk/ jesper/ELDump/def ault.htm Note: ELDump is not proprietary information of GNTS. In the Definition Details Window: - Active: checked - File Name: c:\ntsys.log - Command to generate this file: %PATROL_HOME%\lib\BIT\eldump -a %FROM={%Y%m%d%H%M%S} > c:\ntsys.log - Monitor all matching files: <disabled> Label: NTSysLog Icon: Group: <blank> Auto-reset to OK Timer:0 Alarm if no data Timer:0 The file name to be monitored c:\ntsys.log is the same file as the one that the monitoring command generates. The file is generated every poll cycle therefore the selection made for the Each poll cycle, perform search on menu is irrelevant and ignored. The execution of the monitoring command above would result in the file c:\ntsys.log being created containing the NT/2000 event log with any records added to the event log since the last poll cycle for the given instance. Note that the eldump application must be in your path in order to be executed as specified in the command above, otherwise you need to specify the full path of the command, for example: %PATROL_HOME%\lib\BIT\eldump -a %FROM={%Y%m%d%H%M%S} > c:\ntsys.log In the Search String Attributes Window: (for the definition of “warning” string) - String is Active: checked - String is Case Sensitive: <disabled> - Invert Search: <disabled> - Type of comparison: PSL Reg. Exp. - String to match: warning - State Change produced if string matches: Alarm - Consecutive matches to state: 1 - Matches must occur within(mins): 1 - Annotate in Graph?: checked - When to PopUp?: Always LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 32 In the Embedded Recovery Actions Window: - Alarm Embedded Recovery Action: <blank> - When to Execute: Always - Warn Embedded Recovery Action: <blank> - When to Execute: Always - Notify Embedded Recovery Action: <blank> - When to Execute: Always - OK Embedded Recovery Action: <blank> - When to Execute: Always - No Match Embedded Recovery Action: <blank> - When to Execute: Always In the Schedule Window: - From( Monday through Sunday): 00:00:00 - To( Monday through Sunday): 23:59:59 - Invert (all days of the week): <unchecked> - Poll Interval: 00:10:00 Note: Windows NT/2000 shell command must be enclosed in quotes if there are spaces in the command name. Example Monitoring command with spaces Keep the following in mind when using a monitoring command: the monitoring command cannot be used in combination with wildcards in the file name and/or pattern match files. the monitoring command is run as the user that is running the WatcherLogColl collector (this is, in most cases, patrol). If you want it to execute as some other user, you must configure an override for this collector. See "How To Monitor Protected Files" later in this document for more information. “D:\Program Files\BMC Software\Patrol3\lib\BIT\Eldump” > c:\data\NtEvent.log Macro Variables for the Monitoring Command Table 1 below lists the currently supported macro variables for the monitoring command. Macro Variable %FROM={<FORMAT>} Is replaced with The last date/time collection was performed for the instance. Where <FORMAT> is used to convert the timestamp to the format the command understands. The format uses the same specification as the asctime() PSL function, for details consult your PATROL Script Language Reference Manual. %TO={<FORMAT>} The current date/time. The format is the same as above. Table 1: LogWatch Monitoring Command Macro Variables LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 33 How to Monitor NT/2000 Event Logs Monitoring Windows NT/2000 Event Logs using the Event Viewer can become a repetitive task. If the task has to be performed on several hosts and each host can have different software that can generate several kinds of events, this can become a real challenge for the system administrator. If you want to automate the monitoring of Windows NT/2000 logs you can do it using LogWatch KM for PATROL. This section explains how to monitor NT/2000 Event Logs using the monitoring by command execution functionality of the LogWatch KM and the accompanying ELDump tool. The Windows distributions of the LOGWATCH KM creates the directory BIT under %PATROL_HOME%\lib and copies the eldump binary ELDump.exe and an accompanying README file into it. ELDump is a tool that dumps the contents of an NT/2000 event log as text. ELDump has been written by Jesper Lauritsen and exists in the freeware domain; Arackal Digital Solutions does not support this utility. The binary executable and documentation are free to the public and, can be downloaded from: http://www.ibt.ku.dk/jesper/ELDump/default.htm. Although LogWatch comes with ELDump, several other good NT/2000 Event log parsing tools are available; examples of two other tools are: DumpEvt from SomarSoft dumps the event log in a format suitable for importing into a database. Similar to the DUMPEL utility in the NT resource kit, but without some of the limitations. DumpEvt has been updated to now allow dumping the new Windows 2000 event logs (DNS, File Replication, and Directory Service). To learn more about the tool and download it, go to http://www.somarsoft.com/ DumpEL is the utility provided in the NT/2000 resource Kit. For more about DumpEL or the NT/2000 resource Kit, please search the Microsoft Corporation web pages at: www.microsoft.com Configuring LogWatch to monitor NT/2000 event logs is as simple as defining a new instance that monitors by command. The following example shows the file name and a command text used to define an instance that monitors the NT/2000 event log using the ELDump utility. Example Command to monitor NT event logs using the ELDump tool File Name: c:\ntsys.log Command to generate this file: %PATROL_HOME%\lib\BIT\eldump -a %FROM={%Y%m%d%H%M%S} –b %TO={%Y%m%d%H%M%S} > c:\ntsys.log The above settings would result in LogWatch executing the command every poll cycle, creating the c:\ntsys.log file, which contains any new event log lines that that have been detected since the last poll time. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 34 Simply dumping an event log ELDump.exe comes with an option that lets you dump the Application, Security, and System event. Specify the –l and any of the following choices: system, security or application. If –l is not specified the tool dumps the system event log, by default. Examples Command dump the system, security and application NT event log respectively eldump -l system dumps the system event log eldump -l security dumps the security event log eldump -l application dumps the application event log Organization of the log dump ELDump provides options that allow filtering on the data being dump, information like: Date, Time, Event ID, Source and Computer Columns. If you're familiar with Event Viewer, you can easily pick and understand the data, however you may want to look further into the options provided by the ELDump tool to dump data selectively. The example above dumps all the information for each event. Range of data to dump ELDump provides the –a and –b options to allow dumping messages for a specified range of time: -a time only dumps messages after or at the time specified as yyyymmddhhmmss -b time only dumps messages before the time specified as yyyymmddhhmmss The LogWatch KM provides the %FROM and %TO macros holding the latest time the collection was performed and the current collection time, these macros can be used in combination with the –a and –b provided by ELDump in order to dump only the new entries every poll cycle. The example below shows the format used to achieve this. -a %FROM={%Y%m%d%H%M%S} tells ELDump to dump only messages after or at the time specified by %FROM={%Y%m%d%H%M%S} Note that the macro needs a time pattern; this way the KM can translate the timestamp using asctime into a date/time that is understood by the command (in this case ELDump). The syntax is as follows: %FROM={<PATTERN>} where pattern is replaced by a format specification understood by the asctime psl function. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 35 In this example the combination %Y%m%d%H%M%S corresponds to yyyymmddhhmmss, for example Aug 1st,2000 at 9:30:10 am is represented as: 20000801093010 For more about the format specification of asctime, please consult your BMC PATROL Script Language Reference Manual. -a %TO={%Y%m%d%H%M%S} tells ELDump to dump only messages before the time specified by %TO={%Y%m%d%H%M%S} The format and pattern follow the same rules as the %FROM macro explained above. The above discussion outlines how to add the ELDump command to a LogWatch instance in order to have your NT/2000 Event logs monitored.. You can now proceed to add any search strings, recovery actions or scheduling attributes to this definition as you would with any normal LogWatch monitoring definition. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 36 Embedded Recovery Actions Reference Macro variables are discussed below Embedded recovery actions are specified in the Embedded Recovery Actions Window. They can be OS commands only and execute once per string match. These recovery actions are referred to as being "embedded" because they are actually stored in the LogWatch INI file on the agent host, and are explicitly executed by the KM code. This differs from the normal BMC Patrol recovery actions in several ways: PATROL recovery actions require modifications to the actual KM (requiring redeployment of the KM to effected servers, KM change management, and PATROL Agent restarts). PATROL recovery actions fire only on state changes (i.e. OK to ALARM) LogWatch makes use of macro variables which may be used to easily provide more detailed information pertaining to the string match As stated previously, embedded recovery actions can be of type OS only. That means that basically any command that you would normally type on the command line (on the Patrol Agent host) can be used. Example A useful embedded recovery action An example of an embedded recovery action would be: echo “An error has been detected in Backup log %FILENAME monitored by %LABEL. The error detected was: %MATCHED_LINE. This requires immediate attention!” | mailx –s “Backup error detected!” [email protected] This would result in a mail message being sent to user Joe whenever a string match occurred on the monitored backup log. A complete list of macros is given below. Keep the following in mind when using Embedded recovery actions: Warning! Reference See the “How to Monitor Protected Files” section for details concerning changing the monitoring user. Example OS recovery action commands with spaces there can only be a single embedded recovery action specified per match level per monitored instance. the OS command is run as the user that is running the WatcherLogColl collector (this is, in most cases, patrol). If you want it to execute as some other user, you must configure an override for this collector. See "How To Monitor Protected Files" later in this document for more information. any Windows NT/2000 programs that are not console applications (i.e. they generate a window) will not run correctly, as no graphical display will be available for them to attach to. For example, if you want to start Netscape whenever you notice there is an ALARM string match, simply typing "netscape.exe" into the recovery action window will not work – the netscape process will start, but no window will be displayed. X-Windows programs do not suffer from this problem, provided that an X-emulator is running on the Patrol Agent machine and that the DISPLAY environment variable is set appropriately. Windows NT/2000 shell commands must be enclosed in quotes if there are spaces in the command name as in: “c:\Program Files\software\my_prog.exe” LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 37 Macro Variables for Embedded Recovery Actions Table 2 below lists the currently supported macro variables and what each of them expands to. These variables may be used in embedded recovery actions; the KM expands the variables and places their contents in place of the macro variable specified in the recovery action. In the example given in the Embedded Recovery Action section above, %FILENAME is expanded to the full name of the Log File being monitored, %LABEL is replaced by the label you have given to the Log File monitor, and %MATCHED_LINE is expanded to the entire line that was found to match your criteria. The entire command is then passed to the operating system and executed. Macro Variable %FILENAME %INST_NAME Is replaced with The full name (including path) of the file being monitored The name of the instance that is monitoring this Log File. This is not the same as the label name and would be used for interfacing with the PATROL namespace. %LABEL The label given to the Log File monitor. This is usually a name that you can easily relate to the Log File so you don’t need to remember its entire path. The PEM events sent by the PATROL Agent during state changes will use this label in the origin field. %GROUP This is the group associated with the Log File monitor. By default this will be LOGWATCH. %SIZE The current size of the Log File %GROWTH_RATE The current growth rate of the Log File %MATCHED_LINE The entire line that was found to match the currently configured search criteria. In case of the No Match Embedded Recovery Action the macro contains the line that did not match the search criteria. %HOST The name of the host that the monitor is running on %SEARCH_STRING The search string that was used for the current match %PREVIOUS_LINE The entire previous line to the matched line. Table 2: LogWatch Embedded Recovery Action Macro Variables If you configure a master logfile monitor to use one or more Embedded Recovery Actions, all child monitors that are created by the master employ those same embedded recovery actions. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 38 Generated PEM Events PEM events are generated automatically by the KM in 8 different situations: 1. Whenever a string match occurs in one of the four notification levels, a PEM event is generated outlining the match details. The PEM event Type field is set according to which notification level triggered the event, as outlined in the following table. Notification Level PEM Event Type ALARM ALARM WARN WARNING NOTIFY INFORMATION OK INFORMATION Table 3: LogWatch Notification Level to PEM Event Type Mappings 2. When the instance goes back into the OK state on its own because the Auto-reset to OK Timer configuration option is set. 3. When the instance goes back into the OK state because a Patrol Console user uses the Reset Log File Status menu item for the instance. 4. If the monitor is part of a group and a string match occurs in one of the four notification levels. 5. If the monitor goes into the ALARM state because the Alarm if no data Timer configuration option is set. 6. If the log file corresponding to the monitor is found. 7. If the log file corresponding to the monitor is not found. 8. If Mutual Exclusion is enabled and a match occurs because a line of data in the log file matches none of the strings of the Mutual Exclusion list. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 39 Case 1: String Match In the first case, the resulting PEM event is of class "StringMatchFound" and has the following contents: <logfilename> [Inst=<instance name>]: <state> match found. Matching contents:[<matching logfile data>]. Pattern: [<string condition matched>]. where: <logfilename> is the full path name of the Log File generating the event <instance name> is the name of the LogWatch instance associated with the Log File monitor (see Patrol Agent Users Guide for more information on PATROL instances) <state> is one of ALARM, WARN, NOTIFY and OK <matching Log File data> is the entire line from the Log File that matched the search criteria. <string condition matched> is the entire search criteria starting with the type of comparison made and including the string or value compared. If the comparison type is ‘PSL. Regular expression’, the result starts with “string =~”. For ‘equals to’ the result starts with “string ==” and similar patterns are used for the following numeric comparisons: “<”, “>”, “<=”, “>=”. Case 2: Timeout In the second case, the PEM event is of class "OkStateReset" and has the following format: <label> [Inst=<instance name>]: Logfile status reset to OK due to Auto-Reset Timer. where: <label> is the label that was given to the Log File monitor <instance name> is the name of the LogWatch instance associated with the Log File monitor (see Patrol Developer’s Course Notes for more information on PATROL instances) LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 40 Case 3: Menu Command If a Patrol Console user clears the alarm via the menu command, the resulting PEM event is of class "ManualAlarmReset" and has the following contents: <label> [Inst=<instance name>]: state has been reset to OK due to user invocation of the 'Reset Logfile Status' menu command. where: <label> is the label that was given to the Log File monitor <instance name> is the name of the LogWatch instance associated with the Log File monitor (see Patrol Developers Course Notes for more information on PATROL instances) Case 4: Group Member If a log file monitor is a member of a Group and a string match is detected, an event of class "GroupAlarm" is sent and has the following contents: Monitor '<label>' from LOGWATCH has changed to the <state> state, causing the group '<group name>' to also go into this state. where: <label> is the label that was given to the Log File monitor <state> is the state that the group has changed to <group name> is the name of the group that is sending the event Case 5: No data added If no new data is added to a monitor in the time specified by the Alarm if no data Timer field, the resulting PEM event is of class " AlarmStateAuto" and has the following contents: <label> [Inst=<instance name>]: Logfile status set to ALARM due to no data Timer. where: <label> is the label that was given to the Log File monitor <instance name> is the name of the LogWatch instance associated with the Log File monitor (see Patrol Developers Course Notes for more information on PATROL instances) LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 41 Case 6: Log File Found In the sixth case, the PEM event is of class "LogFileFound" and has the following format: <label> [Inst=<instance name>]: logfile for given instance has been found. where: <label> is the label that was given to the Log File monitor <instance name> is the name of the LogWatch instance associated with the Log File monitor (see Patrol Developers Course Notes for more information on PATROL instances) Case 7: Log File Not Found In the seventh case, the PEM event is of class "LogFileNotFound" and has the following format: <label> [Inst=<instance name>]: logfile for given instance does not exist or not found where: <label> is the label that was given to the Log File monitor <instance name> is the name of the LogWatch instance associated with the Log File monitor (see Patrol Developers Course Notes for more information on PATROL instances) Case 8: Mutual Exclusion List Match In the eighth case, the PEM event is of class "MUXListMatch" and has the following format: <logfilename> [Inst=<instance name>]: Mutual Exclusion List match found. Line not matched:[<non-matching line>]. where: <logfilename> is the full path name of the Log File generating the event <instance name> is the name of the LogWatch instance associated with the Log File monitor (see Patrol Agent Users Guide for more information on PATROL instances) < non-matching line > is the entire line from the Log File that matched none of the strings of the Mutual Exclusion List. For more details see Matching using Mutual Exclusion List Option on page 47. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 42 With exception of case 4, the cases above send events that have the origin field set to: LOGWATCH.<instance_name>.LogFileStatus Where <instance_name> is the name that the KM has assigned to the monitor. The Group Member events have the origin field set to: BITWATCH_GROUP By parsing this from within your help desk application or event correlation engine, you can write advanced rules that effect all LogWatch events, or just those pertaining to the instance <instance_name>. Your rule writing capability will depend greatly on the sophistication level of your event management system. • If you don't want the LogWatch KM to generate the custom ALARM, WARN or OK events, you can add the variable /LOGWATCH/disableEvents to your PATROL Agent's configuration (using pconfig, wpconfig, xpconfig) and set its value to something other than "". • You can use these customized PEM events to performed advanced filtering and correlation of Log File alerts using your external integrated helpdesk/Event Management System. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 43 Using the ‘‘All Matching Files’’ Monitoring Option In some cases, it is useful to be able to monitor all of the files located in a given directory matching a particular filename expression. This type of monitoring is enabled by selecting the “Monitor all matching files” checkbox in the configuration GUI wizard for a given monitor and results in a master instance being created that is “in charge of” one or more child monitors. An instance becomes a master if the File Name to be monitored contains the wildcard character “*” and the Monitor all matching instances checkbox is enabled. Given a master definition, the LogWatch KM performs actions to periodically get a list of the matching files and creates or updates a list of child logfile monitor instances accordingly. An instance is a child of a master if it has been created by a master. LogWatch deals with master and child instances differently than with normal “standalone” logfile monitor instances: The child instance inherits properties like search strings, recovery actions and monitoring schedule from the master. Every time the master is updated, the monitor information gets propagated from the master to the children when the master’s collector runs. If a master becomes inactive, all of its child instances are destroyed. If a master is deleted, all of its children are deleted as well. Only the master instance in saved to the external configuration file and displayed in the list of current instances for deletion or modification in the GUI; a child instance, therefore, can be changed only by changing its’ corresponding master. The LogWatch KM checks for files matching the configured filename pattern and updates, removes and creates instances accordingly: - If a file monitored by a child instance is removed, the instance that was monitoring the file will be deleted. - If a new file is found to match the master’s filename pattern, a child instance is automatically created to monitor the new file and parsing of the new file will start from byte 0. Using master instances may seem confusing at first, but LogWatch automatically does the majority of the hard work for you. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 44 Reading the file from the beginning at a specified date/time For very specific log files it may be necessary to re-read or process the entire file on a regular basis. The default behavior of the LogWatch KM is to process only new lines of data. The Each poll cycle, perform search on menu in the configuration GUI wizard allows changing this selection. It has three choices: 1. Entire File – parses the entire file every poll cycle. 2. New data only – parses only new data appended to the file every poll cycle. 3. New data only, but entire file at date/time – parses new data every poll cycle and parses the entire file at approximately date and time. This section provides the steps for setting the date and time the file is to be processed completely if you select the third choice (“New data only, but entire file at date/time”). It assumes you followed the steps to create a log file definition and selected the Next button on the Definition Details Window. Figure 12: Read Data Settings Window The Read Data Settings Window is displayed only if the Each poll cycle, perform search on Selection is “New data only, but entire file at date/time” in the Definition Details Window. Here you can set the approximate date/time for the log file to be re-read. 1. The Specify approximate time for the file to be read fields allows telling the monitor specifically when it is to re-read the associated log file. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 45 2. The Select the choice that best meet your needs menu selection indicates whether the KM is to re-read the file on specific days of the weeks or on a given day of the month. 3. Specify day(s) of the week has checkboxes available for every day of the week. If your choice was “Selected Days of the week”; the KM will re-read the log file on the selected days. 4. Specify Nth day of the month allows selecting or typing of a number from 1 to 31, indicating the day of the month you want the log file to be re-read. This value is only taken in consideration and used by the KM if your choice was “Specified day(Nth) of the month”. 5. If, for example, you wish to re-read this log file Sundays about 9:30 AM, the Read Data Settings Window Schedule Window would be setup to look like the one below. Once you verify the date and time settings are correct, press the OK button. Example Selecting date/time that KM will reparse entire logfile monitor. Figure 13: Read Data Settings Window Example LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 46 Matching using Mutual Exclusion List Option In some cases, it is useful to be able to monitor the log files for new lines of data that do not match a specific set of strings. This set of strings is called Mutual Exclusion list and selecting the Consider these strings as being mutually exclusive checkbox in the configuration GUI wizard enables it. A Mutual Exclusion list is created internally by taking each of the search string definitions that are Active and which Type of comparison is PSL Reg. Exp. The settings for State Change produced if string matches, Consecutive matches to state, Matches must occur within(mins), Annotate in graph? and When to PopUp? are not used if Mutual Exclusion is enabled. Example Making a list of mutually exclusive strings. In order to clarify how the Mutual Exclusion List feature works, let’s assume we have an application that creates a specific log and we want to monitor that log file to match any line that does not contain the expressions: “record entry saved” or “db update completed” To do this, we set the check the Consider these strings as being mutually exclusive field in the Search Strings Definition Window add the following string definitions: In the Search String Attributes Window: (for the definition of “record entry saved”) - String is Active: checked - String is Case Sensitive: <disabled> - Invert Search: <disabled> - Type of comparison: PSL Reg. Exp. - Search String: record entry saved In the Search String Attributes Window: (for the definition of “db update completed”) - String is Active: checked - String is Case Sensitive: <disabled> - Invert Search: <disabled> - Type of comparison: PSL Reg. Exp. - Search String: db update completed The above definition creates the following Mutual Exclusion list: record entry saved db update completed The KM generates events of type MUXListMatch for every line of data that matches none of the strings in the Mutual Exclusion list. For ex: The line: “RQST000005167 record entry saved” will not generate a MUXListMatch, but the line “database rollback” will generate a MUXListMatch event. The MUXListMatch events are notification events; the KM generates a MUXListMatch and a PopUp window for every line of data that matches none of the strings in the list. The KM will also execute the Notify Embedded Recovery Action if one has been specified. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 47 How to Monitor Protected Files A protected file is one that is only readable by the owner. Since, in most cases, the LogWatch collector is executed by the patrol user, no content monitoring will be possible on any log file that the patrol user does not have read access to. The following error will be displayed on your Patrol Console if the KM cannot monitor a protected file whenever the monitored file is updated: ERROR (LOGWATCH:WatcherLogColl)==>Couldn't open file [/var/adm/sulog] for read (no string matching will be performed)! ERROR (LOGWATCH:WatcherLogColl)==>Please check that the user <defaultAccount> has read permissions for the file. The size and growth rate parameters will still collect and show accurate information, even if the file is not readable; you will not be able to search the file for content though. In the following example, we outline the steps that should be used to monitor the contents of the sulog file on a UNIX machine. This file is normally readable only by root. Define a log file monitor for the sulog, as outlined in the Adding a Log File Monitor in page 10. At this point the KM will be monitoring the log’s size and growth-rate and will be generating the above error on your console whenever the file is changed. Right-mouse on the new sulog monitor and run the Monitor this instance as… menu command located under the Admin submenu. You will be presented with a username/password dialog; enter the name of the user allowed to configure the PATROL Agent (this can be any user found having “C” access in the Agent’s accessControlList pconfig variable). By default everyone can configure the agent, so just typing in the patrol username and password will probably work. This will enable you to write to the PATROL Agent’s configuration database. Now, a new username/password dialog will be presented to you. Enter the name and password of the user that you wish to have monitor the sulog; we want to enter the information for the "root” user for this example. Pressing OK in this dialog writes the information into the Agent’s configuration database. Alternatively, for mass distribution: Using one of the Agent configuration utilities wpconfig (WinNT), xpconfig, or pconfig, connect to the Patrol Agent that is to monitor the sulog file. Create a new variable called: /AgentSetup/LOGWATCH.<instname>.OSdefaultAccount where <instname> is the name of the instance as it appears in your Patrol console. Proceed to set the value of this new variable; there will be fields in the dialog where you can enter in an appropriate username and password (root is required for monitoring the sulog). As soon as you apply the configuration changes to the agent, LogWatch will be able to monitor the sulog. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 48 Warning!: it is extremely important that no users other than the PATROL administrators be given write access to the LogWatch INI file after performing the above steps, due to the fact that the sulog log file monitor is now monitoring the file as root. Since the same collector is responsible for executing the embedded recovery action, care must be taken to ensure that any access to this recovery action (and thus the Configuration Wizard GUI) is carefully guarded. Displaying the Contents of a Protected File If you need to view the contents of a protected file, select either the Show Last N LogFile Lines or Show First N LogFile Lines menu commands from the log file instance that you want to view. The KM will ask you to enter a username and password and will attempt to open the file using this authentication information. If successful, the contents of the file will be displayed in a popup report window on your PATROL Console. LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 49 The LogWatch INI Files Note LogWatch has been updated to automatically read any external changes made to the INI files including icon, label, and group! All configuration information is stored in an external Microsoft Windows INI-style configuration files. The KM created one ini file per logfile definition. The choice to use the INI file format was based on several factors: the INI file can then be viewed or modified using any text editor can be easily copied or backed up, or put under change control can be easily write protected by the PATROL administrator is not limited by field sizes, as PATROL pconfig variables are There is one configuration file for each PATROL Agent running on the machine. By default these files are located in the $PATROL_HOME/config directory (UNIX) or %PATROL_HOME%/config (Windows), but this location can be easily changed (as outlined below). The name of each INI file is tied to the port number that each agent is running on and the instance name. Example The name and default location of the LogWatch INI files for an Agent on port 3181 If there is a PATROL Agent running on port 3181, the associated LogWatch INI file for an instance name log1 would be %PATROL_HOME%/config/lw3181-log1.ini by default If you want someone other than the patrol account to own the file, you will have to manually change the ownership using the appropriate method for your operating system. When you use the KM GUIs to make configuration changes to LogWatch, all reading and writing of the configuration file is performed automatically by the KM code. If you use scripts, an editor, or some other utility to modify the INI files directly, LogWatch will pickup the modified lines and automatically make the changes to the appropriate Log File monitors. You could use this feature to implement monitoring of files with rotating monitoring schedules or filenames, for example. Format Change: the KM will backup any pre-existing ini if you have any version 2.x; however we recommend you back up the file prior to loading LogWatch v3 if you had a previous release install. If you experience any issues with the auto conversion performed by the KM, please contact our support department at: [email protected]. Warning: The KM uses the label to create an unique instance name and keeps a list of instance names in the variable: /LOGWATCH/instanceList If you change the instance list, you need to change the ini file section that has the name as well as rename it. We do not recommend changing names manually, using the GUI instead. Changing the Location of the INI Files Directory If you wish to change where the INI files are located, simply set the Agent configuration variable /LOGWATCH/configDir to point to the full path using pconfig, xpconfig, or wpconfig (see the Patrol Agent Reference Guide for more information on using these external utilities). LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 50 Example Changing the location of the INI file If you want the INI files to be in /home/bob, set the "/LOGWATCH/configDir" agent setup variable to "/home/bob " and then use the "Re-read INI file" menu command to force the KM to use the new directory. Where to go from here… To… Descriptions of the menus provided and related functions … Detailed explanation for each of the InfoBox fields … Attributes and meaning of the KM Parameters … Description of the fields for each wizard window including default values and format expected … LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved Proceed to… Pg. 52 Chapter 5. Menu Summary Pg. 545 Chapter 6. InfoBox Item Summary Pg. 577 Chapter 7. Parameter Summary Pg. 622 Chapter 8. Fields Summary 51 ote all can only be only from a oper Console and ared files (such se used by other atch products) will deleted. The ort#>nce_name.ini files eserved. Chapter 5. Menu Summary This chapter summarizes the various menu commands available in the LogWatch KM. To access these commands, perform one of the following: Using the Windows NT/2000 Console, right-click your mouse on the Log File instance icon you are interested in and go to the KM Commands menu pick Using UNIX, right-click your mouse on the Log File instance icon you are interested in. LogWatch Menu Commands: Name Admin Logfile Maintenance Re-read INI file Show INI file Add to Preloaded KMs Reset… Reset Logfile Status Clear Last Matched Info Reset GUI Lock Uninstall LogWatch LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved Description All submenus of this menu pick are related to maintaining Log File monitors. This displays the Log File Maintenance dialog box, enabling you to add, deleted, or edit LogWatch monitoring attributes. The username/password combination entered upon selecting this menu should correspond to someone who has proper OS level privileges for writing the LogWatch INI file (see "Using the LogWatch KM" for more detailed information). This item forces the KM to re-read the LogWatch INI file and completely reinitialize it. Note: all monitoring of Log Files is stopped during the reset operation. This item copies the contents of the existing LogWatch INI file to the Patrol Console. This item adds the LOGWATCH_SETUP.kml to the /AgentSetup/preloadedKMs configuration variable, forcing the PATROL Agent to load the LogWatch KM upon startup. Use the choices under this menu to manually reset several Instance specific variables. If an instance is currently in the ALARM state, running this menu command will reset its state to OK. A PEM event is sent to any listeners when this command is run. This should be used to clear the Last Matched Info items located in the instance's InfoBox. When a Patrol Console user is editing the LogWatch INI file, a temporary lock is created in the PATROL Agent’s name space so that 2 users don’t inadvertently cancel each other’s changes. There are certain rare circumstances where it may be necessary to manually remove the lock file. This menu command is used for this. This command will uninstall the LogWatch KM from the given agent. 52 ference “How to Monitor ed Files” section for concerning changing itoring user. a large number of ay cause the Patrol run out of memory. Monitor this instance as… Enables anyone having configuration access to the PATROL Agent to change which user the WatcherLogColl parameter executes as. Ordinarily, running this collector as the default patrol user is sufficient, but if you want to monitor protected files or wish to have your Embedded Recovery Action execute as a different user (root for example), the instance must be monitored by that user. Use the choices under this menu pick to enable or disable Debug debugging of the LogWatch KM for the associated instance. Enable This enables debugging for the associated instance. Most of this information will not be entirely useful to you, but will help BIT support if the need arises. Note: debugging information will be dumped to all Patrol consoles that have the LogWatch KM loaded. Disable This turns off debugging. Enable RA Output Logging This enables redirection of embedded recovery action output to a text file, making debugging of the Embedded Recovery actions easier. The file generated is located in the $PATROL_HOME/log (UNIX) or %PATROL_HOME%/log (Windows) directory. The name of the output file is: logwatch_db_out.txt Disable RA Output Logging This turns off RA Output Logging. Use this command to display a specific number of lines Show First N Logfile Lines from the top of associated Log File instance with the output going to the Patrol Console. Use this command to display a specific number of lines Show Last N Logfile Lines from the bottom of associated Log File instance with the output going to the Patrol Console. This forces the collector for the associated Log File Refresh Parameters monitor to execute immediately. This is useful when you want to see updated information, but don’t want to wait for the scheduled poll-time to come around. Displays if only new data or entire file is to be parsed every Settings for Data to Read poll. It also contains the date/time details in case file is to be read on a regular basis. Displays a report containing the schedule for the Show Monitoring Times corresponding Log File monitor. Displays a report containing the currently Mutual Show Search Strings Exclusion list or the configured search strings for each notification level, as well as the status of each (Active or Inactive), and the current match N times with M minutes settings. Displays contact and company information. About LogWatch… Table 4: Summary of Menu Commands LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 53 Where to go from here… To… Detailed explanation for each of the InfoBox fields … Attributes and meaning of the KM Parameters … Description of the fields for each wizard window including default values and format expected … LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved Proceed to… Pg. 555 Chapter 6. InfoBox Item Summary Pg. 577 Chapter 7. Parameter Summary Pg. 622 Chapter 8. Fields Summary 54 Chapter 6. InfoBox Item Summary This chapter summarizes the various InfoBox items available in the LogWatch KM. To access these commands, perform one of the following: Using the Windows NT/2000 Console, right-click your mouse on the appropriate Log File instance icon and go to the InfoBox… menu pick. Using UNIX, click and hold your middle mouse button on the appropriate Log File instance icon. LogWatch InfoBox Items: Name Currently Active? Full Name Monitoring Command Custom Polling Interval (secs) Being Monitor By Monitoring Times Current NOTIFY Match Criteria Current OK Match Criteria Current WARN Match Criteria Current ALARM Match Criteria Row and Column Filter Auto-Reset Timers OK, Alarm LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved Description Indicates the monitoring status for this instance. If "No", no monitoring is currently being performed. Displays the full path name of the monitored Log File. If the monitor is a master instance, this item shows “<filename>[MASTER]”. If the filename was specified as a pattern match, the name of the currently matching file will be provided, also shown surrounded by “[]”. Displays the command that is to be executed to generate the log file prior to parsing. Number of seconds between successive checks for new Log File text. User the instance is being monitored by. Day and time ranges that this instance will actively monitor the associated Log File. Outside of these time ranges, the monitor’s Active InfoBox item will still show “Yes”, but no history will be stored and no state changes will occur. A quote (" ") separated list of strings that, if found in new Log File text, will trigger a NOTIFY level match A quote (" ") separated list of strings that, if found in new Log File text, will trigger a OK level match A quote (" ") separated list of strings that, if found in new Log File text, will trigger a WARN level match A quote (" ") separated list of strings that, if found in new Log File text, will trigger a ALARM level match Displays the Row, Column and Separator information Displays two pieces of information: <OK AutoReset>, <Alarm Auto-Reset>. OK-Auto Reset: Indicates the number of minutes the KM will wait before automatically setting to OK state. If this has not been configured for the instance, 55 it will read "<disabled>". Alarm-Auto Reset: Indicates the number of minutes the KM will go into ALARM if no new data is added to the Log File. If this has not been configured for the instance, it will read "<disabled>". Last Matching for N, OK, W, A Last Polled Size, Touched KM Licensee, Expires For support, contact KM Version Shows the last matched line that caused the respective log file instance to notify or change state to Ok/Warn/Alarm. Displays two pieces of information: <Last Polled File Size>, < Last Touched Time >. Last Polled File Size: Indicates the last noted Log File size (in bytes) Last Touched Time: Shows the last time the associated Log File was changed. Displays two pieces of information: <LogWatch Licensed To>, <LogWatch License Exp Date> LogWatch Licensed To: Company name the product is licensed to. LogWatch License Exp Date: Displays the expiry date of your currently installed license. This will display "<never>" if you have purchased a permanent license. Shows the email address you should use if you require support for this KM. Displays the KM Version, as set by the vendor. Table 5: LogWatch InfoBox Items Where to go from here… To… Attributes and meaning of the KM Parameters … Description of the fields for each wizard window including default values and format expected … LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved Proceed to… Pg. 577 Chapter 7. Parameter Summary Pg. 622 Chapter 8. Fields Summary 56 Chapter 7. Parameter Summary This chapter outlines the parameters provided by the LogWatch KM including their types, poll intervals, security and recovery actions. To access the non-collector parameters, double click on a monitored Log File instance; if you have not yet added any instances to be monitored, you will have only the LOGWATCH_SETUP currently displayed (this instance has no parameters associated with it as it is only used for configuration). Proceed to the Chapter 3. Setting up a Log File Monitors for details concerning adding a new Log File monitor. LogWatch Parameters: Parameter Name AlarmStringsMatched Number of alarm string conditions matched per poll cycle. Consumer Type Graph View Type Not applicable Default Polling Interval Not applicable Default Security None Default Recovery Actions Table 6: AlarmStringsMatched Parameter Description Parameter Name CurrentSize This parameter holds the size of the associated Log File at any given time. Consumer Type Graph View Type Not applicable Default Polling Interval Not applicable Default Security None Default Recovery Actions In the cases where the size of the Log File is Recommended Recovery important, a possible recovery action could be to Actions email/page a user, archive it and then truncate it down to a reasonable size. Table 7: CurrentSize Parameter Description Parameter Name GrowthRate Description This parameter holds the current growth rate exhibited by this Log File, over the last polling interval; the polling interval is dictated by the WatcherLogColl parameter. Consumer Graph Not applicable Type View Type Default Polling Interval LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 57 Not applicable None Log File growth rate can often be important. Using this parameter, you could easily set a maximum allowed growth-rate and force an email, or page to go out to a technician indicating the problem. It would also be possible to make a "smarter" recovery action that first checks the CurrentSize parameter and performs a sort of correlation before sending off a message (i.e. having a small Log File that has an erratic growth-rate is not really a problem in most cases). Table 8: GrowthRate Parameter Default Security Default Recovery Actions Recommended Recovery Actions Parameter Name LinesNotMatched Number of lines that do not match any of the defined conditions every poll cycle. Consumer Type Graph View Type Not applicable Default Polling Interval Not applicable Default Security None Default Recovery Actions Table 9: LinesNotMatched Parameter Description Parameter Name LogFileStatus This parameter indicates in which of the 3 possible states the associated Log File is currently in (i.e. OK, WARN, or ALARM). This consumer's value is set by the collector whenever a string match occurs on new Log File data. Consumer Type Stoplight View Type Not applicable Default Polling Interval Not applicable Default Security There are no default recovery actions Default Recovery Actions The thresholds associated with each of the states are preset to: OK = 0 WARN = 1 to 5 ALARM = 5 to 10 These threshold ranges should not be changed as the collector relies on them to change the state of the LogFileStatus parameter as appropriate. Since this is a stoplight type parameter, each of the 3 Recommended Recovery states can execute a PATROL recovery action. Actions Note that PEM events are automatically sent by the KM whenever an OK, WARN, or ALARM level string match occurs. Table 10: LogFileStatus Parameter Description LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 58 Parameter Name NotifyStringsMatched Number of notify string conditions matched per poll cycle. Consumer Type Graph View Type Not applicable Default Polling Interval Not applicable Default Security None Default Recovery Actions Table 11: NotifyStringsMatched Parameter Description Warning! The: NOTIFYMatchedLines parameter has been removed and superceded by NotifyStringsMatched and the others <State>StringsMatched Parameter Name OKStringsMatched Number of OK string conditions matched per poll cycle. Consumer Type Graph View Type Not applicable Default Polling Interval Not applicable Default Security None Default Recovery Actions Table 12: OKStringsMatched Parameter Description Parameter Name SizeOfLinesMatched Total size(in bytes) of the lines that match string conditions every poll cycle. Consumer Type Graph View Type Not applicable Default Polling Interval Not applicable Default Security None Default Recovery Actions Table 13: SizeOfLinesMatched Parameter Description Parameter Name TimeBetweenUpdates Number of seconds between file updates. 0 means no updates have been performed. Consumer Type Consumer View Type Graph Default Polling Interval Not applicable Default Security Not applicable Default Recovery Actions Table 14: TimeBetweenUpdates Parameter Description LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 59 Parameter Name TotalStringsMatched Number of string conditions matched per poll cycle. Description Consumer Type Consumer View Type Consumer Default Polling Interval Graph Default Security Not applicable Default Recovery Actions Table 15: TotalStringsMatched Parameter Parameter Name WarnStringsMatched Number of warns string conditions matched per poll cycle. Consumer Type Consumer View Type Consumer Default Polling Interval Graph Default Security Not applicable Default Recovery Actions Table 16: WarnStringsMatched Parameter Description Parameter Name WatcherLogColl Description This parameter polls the associated Log File, looking for new text data. If any is found, the new data is checked against the user-defined ALARM, WARN, NOTIFY and OK criteria and appropriate actions taken. A single collector sets each of this instance’s consumer parameters. Standard None 30 seconds Inherited None None Type View Type Default Polling Interval Default Security Default Recovery Actions Recommended Recovery Actions Table 17: WatcherLogColl Parameter Parameter Name ExtraFilesList Indicates which files should be sent to other PATROL Agents when the LogWatch KM is committed. This parameter should never be active. Standard Type None View Type Not applicable Default Polling Interval Not applicable Default Security Not applicable Default Recovery Actions Table 18: ExtraFilesList Parameter Description LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 60 Where to go from here… To… Description of the fields for each wizard window including default values and format expected … LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved Proceed to… Pg. 622 Chapter 8. Fields Summary 61 Chapter 8. Fields Summary Find below all the fields shown in the wizard windows that define a Log File, the meaning of each of the fields, and other relevant information such as default value. Name Active File Name Command to generate this file Monitor all matching files Label Icon Group Name Auto-reset to OK Timer Alarm if no data Timer Each poll cycle, perform search on Default Value On Purpose Specifies whether or not the KM should monitor this log file. File name including full path for the Log File. Monitoring command to be used if the Log File is generated by the execution of a command. If set, the KM will execute the command to generate the file every poll cycle. Specifies whether or not the KM should monitor all the Off files that match the given filename pattern. A wildcard (“*”) character must be specified somewhere in the filename to enable this feature. Label for the instance (optional). If not set the filename is used. Icon for the instance (optional), if not set it uses log. Log Specifies the name of a container for instance to be logically created under (optional). If not specified the monitor is created under LOGWATCH_SETUP. 0 minutes Specifies the number of minutes the monitor will wait (disabled) before automatically resetting the LogFileStatus parameter’s state from WARN or ALARM back to OK. The KM treats an auto-timeout as an OK string match, so the OK Embedded recovery action will also be fired, if it is configured. This time is approximate as time checks are rounded to the nearest 30 seconds. 0 minutes Specifies the number of minutes the monitor will wait (disabled) for new data to be added to the log file before automatically setting the LogFileStatus parameter into the ALARM state. This time is approximate as time checks are rounded to the nearest 30 seconds. New data only Specifies whether or not the KM should read the logfile associated to the monitor from the beginning every poll cycle, processing only the new data added or reading the new data every poll cycle and the entire file at approximate date and time. Table 19: Fields for Definition Details Window LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 62 Name Specify approximate time for the file to be read Select the choice that best meet your needs Specify day(s) of the week Specify Nth day of the month Default Value 00:00:00 Purpose Tells the monitor specifically when it is to re-read the associated log file. Selected Days of the week This menu selection indicates whether the KM is to reread the file on specific days of the weeks or on a given day of the month. Sun, Mon, Tue, We, Thu, Fri, Sat (All days are selected) 1 (means first day of the month) These checkboxes represent every day of the week. The KM will re-read the log file on the selected days. Indicates the day of the month the log file is to be reread. This value is only taken in consideration and used by the KM if your choice was “Specified day(Nth) of the month”. Table 20: Fields for Read Data Settings Window Name Column Default Value Column Delimiter Row Consider these strings as being mutually exclusive Off (disabled) Purpose Specifies whether or not the KM should match only specific columns of the log file. Used to define the word separator instead of white spaces. Used only if Column and/or Row have been set. Specifies whether or not the KM should match only specific rows of the log file. Indicates whether or not the KM should treat all the strings as part of a Mutual Exclusion List. Table 21: Fields for Search Strings Window Name String is Active Default Value On String is Case Sensitive Off – indicating case will not be considered. Off – indicating search will not be inverted. Invert Search LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved Purpose Specifies whether or not the monitor should actively look for the associated string within any new Log File contents. Specifies whether or not case matters in the string search. Specifies whether or not to invert the result of the search for the specified string. 63 Type of Comparison PSL. Reg. Exp by default – indicating the string will be compared as a regular expression using PSL grep String to match State Change produced if string matches Consecutive matches to state Matches must occur within When to PopUp? Annotate in Graph? Alarm Specifies the type of comparison to be performed when checking if the search string matches a given condition. Types available are: • PSL. Reg. Exp • < • > • == • <= • >= Specifies the string to use for searching any newly found Log File data. All regular expressions supported by the grep PSL function are allowed in this field (see pg 4136 of the Patrol Script Language Reference Manual – July 15, 1998 for more details on supported regular expressions). Note that new type of comparisons have been added starting with LogWatch v3. Search strings need to define based on the comparison type; for example: 7 would be a “good” choice if the comparison type is Equal To. Usually comparison types other than RegExp are used when the Columns and/or Rows fields have been set. Indicates the change of state that will be generated by KM if the match condition is met. Possible values are: Alarm, Warn, Notify or OK. 1 Indicates how many string matches must be found before the Log File monitor changes to the associated state. Note: the Matches must occur within field further effects the state change. 1 minute Indicates the time window in which the string matches must occur. If the above-specified number of strings is not received within the time window, no state change occurs. Note: if the Consecutive matches to state field is 1, the value in this field is ignored. Dictates when a dialog box should be displayed on the Never PATROL Console whenever a string match occurs for the given notification level. Values available are: • Never • Only is state changes • Always Specifies whether or not to annotate string matches to On the PATROL agent's annotation database. Note: the annotation database can fill quickly if a large number of string matches occur Table 22: Fields for Search String Definition Window LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 64 Name Alarm Embedded Recovery Action When to Execute? Warn Embedded Recovery Action When to Execute? Notify Embedded Recovery Action When to Execute? OK Embedded Recovery Action When to Execute? No Match Embedded Recovery Action When to Execute? Default Value Purpose Set to the OS command that you would like to be executed by the PATROL Agent whenever a string match is found for the Alarm state. Always Dictates when the Alarm Embedded Recovery Action must be executed. Values available are: • Always • Only if state changes • Never Set to the OS command that you would like to be executed by the PATROL Agent whenever a string match is found for the Warn state. Always Dictates when the Warn Embedded Recovery Action must be executed. Values available are: • Always • Only if state changes • Never Set to the OS command that you would like to be executed by the PATROL Agent whenever a string match is found for the Notify state. Always Dictates when the Notify Embedded Recovery Action must be executed. Values available are: • Always • Only if state changes • Never Set to the OS command that you would like to be executed by the PATROL Agent whenever a string match is found for the OK state. Always Dictates when the OK Embedded Recovery Action must be executed. Values available are: • Always • Never Set to the OS command that you would like to be executed by the PATROL Agent whenever a new log file line is found that does not match any of the configured search strings. Always Dictates when the No Match Embedded Recovery Action must be executed. Values available are: • Always • Never Table 23: Fields for Embedded Recovery Actions Window LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 65 Name From Default Value 00:00:00 To 23:59:59 Invert Poll Interval Purpose Specifies From time (for each day of the week) when the Log File monitor will start to operate. Note: This time is in 24-hour format. (i.e. 00:00-23:59) Specifies To time (for each day of the week) when the Log File monitor will stop monitoring. Note: This time is in 24-hour format. (i.e. 00:00-23:59) If From and To are both the same value(for the same day of the week), the Log File monitor will assume it is to be inactive for the entire 24 hour period. Unchecked Inverts the monitoring. If the monitor is inverted, it will NOT monitor during the specified time range. 10 minutes Determines the time between successive checks for new characters in a given Log File Table 24: Fields for Log File Schedule Window LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 66 Appendix A: INI Configuration File and Explanation ; INI configuration file - Auto-generated by INIMANIP.LIB [v3.0] ; Copyright(c) 2004, Arackal Digital Solutions. All rights reserved. ; Last Written: Thu Jul 25 07:24:56 2004 ; Format: ; [<section name>] ; <name>=[<value>] [INI-GENERAL] version=3.1 [test3] active=1 fullname=c:\temp\test3.log fullCommand= patternMatchAll=0 label=test3 icon=log groupName= okAutoResetTimeout=0 alarmAutoResetTimeout=0 readFromBeg=2 readAtTime=0 weekOrMonthSel=1 weekDaysList=1 2 3 4 5 6 7 nMonthDay=1 stateList=2,1 compTypeList=1,1 activeList=1,1 popupList=1,3 annotateList=1,1 stringList=string2\;string1 caseList=0,0 invertList=0,0 matchNtimesList=1,1 matchWithinList=1,1 columns= colDelim= rows= mutualExclusion=0 alarmRA= warnRA= notifyRA=dir>>c:\temp\rares.log okRA= nomatchRA= raExecTypeList=3,3,3,3,3 scheduleList=00:00:00-23:59:59-0,00:00:00-23:59:59-0,00:00:00-23:59:59-0,00:00:00-23:59:59-0,00:00:00-23:59:590,00:00:00-23:59:59-0,00:00:00-23:59:59-0 pollinterval=60 LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 67 Name/Value Pair Formats [InstanceName] (an unique name for this instance) active=Boolean ( 1 if the monitor is Active 0 if not ) fullname=String ( entire path and filename for the log file ) fullCommand=String ( monitoring command that generates the above file, when applicable ) patternMatchAll=Boolean ( 1 if match all files for pattern specified in fullname, 0 if not ) label=String ( label for the Instance ) icon=String ( name of the icon with no extension ) groupName=String ( name of the Group ) okAutoResetTimeout=Integer ( representing number of minutes to auto-reset to OK ) alarmAutoResetTimeout=Integer ( representing number of minutes before ALARM if no new data ) readFromBeg=Integer (1 if the KM has to re-read the file from the beginning every poll, 2 if the KM reads only the new data added to the file, 3 if the KM behaves like option 2 + it has to re-read the file at a specified date/time) readAtTime= Integer ( containing the time to re-read the monitor from the beginning in seconds ) weekOrMonthSel= Integer (1 if the KM has to re-read the file on selected days of the week, 2 if the KM re-reads the file on Nth day of the month) weekDaysList=List of integers separated by “ ” ( values can go from 0 to 7. 0 means none of the days of the week are selected. 1 2 3 4 5 6 7 means all days of the weeks. 1 being Sunday, 2 Tuesday and so on.) nMonthDay= Integer ( containing the Nth day of the month the KM has to re-read the file ) stateList=List of States separated by “,” ( States can be 1,2,3,4 for A,W,N,O – one item per string ) compTypeList=List the kind of string comparison separated by “,” Values can be 1,2,3,4 for RegExp, Less Than, Greater Than, Equal to respectively) activeList=List of Active flags separated by “,” ( 1 if true or 0 if false – one item per string ) popupList= List of PopUp flags separated by “,” ( 1,2,3 for Never, Only if State Changes or Always respectively) annotateList= List of Annotate Flags separated by “,” ( 1 if true or 0 if false – one item per string ) stringList= List of Strings separated by “\;” caseList= List of Case flags separated by “,” ( 1 if true or 0 if false – one item per string ) invertList= List of Invert Search flags separated by “,” ( 1 if true or 0 if false – one item per string ) matchNtimesList= List of Integers separated by “,” ( representing number of matches) matchWithinList= List of Integer separated by , ( representing minutes within matches occur ) columns=String (column(s) to be matched, optional; ex: 1-3, 1) colDelim=String (character that delimits columns, optional; by default KM uses blank spaces) rows=String (row(s) to be matched, optional; ex: 3, 5-) mutualExclusion= Integer ( 0 if Mutual Exclusion is OFF, 1 otherwise. ) alarmRA=string ( containing an OS command ) warnRA=string ( containing an OS command ) notifyRA=string ( containing an OS command ) okRA=string ( containing an OS command ) nomatchRA=string ( containing an OS command ) raExecTypeList= List of Integers defining when to execute the corresponding RA separated by “,” (1,2,3 for Never, Only if State Changes and Always; respectively) Values have the correspond to the following RA in order: alarmRA, warnRA, notifyRA, okRA and nomatchRA scheduleList=List with seven elements of “<FROM>-<TO>-<INVERT>” separated by “,”. The first of the seven elements represents Sunday with the last item representing Saturday. pollinterval=Integer ( containing the monitor’s polling Interval in seconds ) LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 68 Index % I %FILENAME ...........................................................37 %GROUP ..................................................................37 %GROWTH_RATE..................................................37 %HOST .....................................................................37 %INST_NAME ...................................................32, 37 %LABEL...................................................................37 %MATCHED_LINE.................................................37 %SEARCH_STRING................................................37 %SIZE .......................................................................37 Icon field................................................................... 61 Installation on UNIX .............................................. 6, 7 Installation on Windows ......................................... 6, 7 Invert field .................................................... 21, 44, 65 Invert Search field .................................................... 62 A Active Field ...............................................................61 Add a new Log File definition ...................................11 Add to Preloaded KMs ............................................6, 8 Adding a Log File Monitor........................................10 Admin ..................................................................10, 51 Alarm Embedded Recovery Action field ............21, 64 Alarm if no data Timer field......................................61 Annotate in Graph? field ...........................................63 Attributes for Embedded Recovery Actions..............64 Attributes for Log File Definition .............................61 Attributes for Search String Definition................62, 63 Auto-reset to OK Timer field ....................................61 L Label field ........................................................... 13, 61 List of Macros........................................................... 21 Logfile Maintenance ........................................... 10, 51 LogFileStatus Parameter........................................... 57 LogWatch Features..................................................... 3 LogWatch INI file .......................10, 25, 27, 36, 48, 51 M Macro Variables ................................................. 32, 37 Matches must occur within field................... 18, 46, 63 Modify an existing Log file definition ....................... 23 Modify Log File Window.......................................... 23 Modifying a Log File Monitor................................. 23 Monitor all matching files field ................................ 61 N Consecutive matches to state field.................18, 21, 63 CurrentSize Parameter...............................................56 No Match Embedded Recovery Action field............ 64 Notification Levels ................................................... 16 Notify Embedded Recovery Action field ........... 21, 64 NOTIFYMatchedLines Parameter............................ 58 D O Defining a Simple Log File Monitor .........................12 Defining the Embedded Recovery Actions ...............20 Defining the Log File Monitoring Schedule..............21 Defining the Search Strings to monitor for................15 Definition Details Window....11, 12, 13, 20, 21, 24, 44 Deleting Log Files from the Monitored List..............26 OK Embedded Recovery Action field ................ 21, 64 E Sample INI File ........................................................ 66 Schedule Window ..............................21, 22, 25, 44, 45 Search String Attributes Window ....................... 16, 18 Search String field ........................................ 16, 62, 63 Search Strings Definition Window ....14, 15, 18, 19, 46 Show PopUp? field ....................................... 18, 46, 63 State Change produced if string matches field.... 16, 63 String is Active field................................................. 62 String is Case Sensitive field .............................. 16, 62 C Embedded Recovery Actions Window.....19, 20, 21, 44 ExtraFilesList Parameter ...........................................59 F File Name field..............................................12, 13, 61 From field ......................................................21, 44, 65 G Group Name field ................................................14, 61 GrowthRate Parameter ..............................................57 LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved P Poll Interval field ................................................ 22, 65 S T The LogWatch INI File ............................................ 49 69 To field ..........................................................21, 44, 65 W Warn Embedded Recovery Action field ............. 21, 64 WatcherLogColl Parameter .................... 56, 57, 58, 59 Working with Groups ............................................... 29 Warn Embedded Recovery Action ............................21 LogWatch KM Installation and User Guide Copyright ©2002-2004, Arackal Digital Solutions All rights reserved 70