How to Maintain Caching in eDirectory 8 and eDirectory 8.5 Feature Article

How to Maintain Caching in
eDirectory 8 and eDirectory 8.5
Feature Article
NOVELL APPNOTES
Novell Technical Services
Worldwide Support Services
Novell, Inc.
[email protected]
Compiled by Scott Ivie and Shawn Pierson; technical review by Daniel Sanders,
Tom Doman, Steve McLain, and Duane Buss
As Novell eDirectory implementations grow larger, it is important not to sacrifice
performance to gain new functionality. Caching is a critical feature of eDirectory
that should be tuned and maintained for optimal system performance. This
AppNote discusses the caching implementation and offers maintenance tips for
eDirectory 8 and eDirectory 8.5.
Contents:
4
•
Understanding Caching in eDirectory 8 and eDirectory 8.5
•
Monitoring and Configuring Cache for eDirectory 8
•
Monitoring and Configuring Cache for eDirectory 8.5
•
Using iMonitor to Monitor and Configure Cache Settings
•
Conclusion
Topics
caching, directory services, network management, performance tuning and optimization
Products
eDirectory 8, eDirectory 8.5
Audience
network administrators, consultants, integrators
Level
intermediate
Prerequisite Skills
familiarity with eDirectory
Operating System
n/a
Tools
console commands, iMonitor
Sample Code
no
www.novell.com/appnotes
Understanding Caching in eDirectory 8 and eDirectory 8.5
With eDirectory 8.5, two types of caching have been implemented:
•
Block (or page) Caching
•
Entry Caching (sometimes referred to as record cache)
Note: DS build versions previous to eDirectory 8 build version 8.73 only support
block caching.
Physical block or block caching is typically implemented by database systems.
eDirectory 8 has employed this type of caching from its introduction. Before
reading a block (or page) from the database, eDirectory will search its block cache
for the needed information.
Entry caching was introduced in eDirectory 8 with DS build version 8.73 and
above, and with the initial release of eDirectory 8.5. The entry cache contains
logical entries in the Directory, as opposed to physical blocks. Caching of entries
reduces the processing time required to instantiate entries in memory from the
block cache. Before accessing block cache or the disk to retrieve a particular
entry, eDirectory will first search its entry cache.
Block and entry caches have a certain amount of redundancy, but each is designed
to boost performance for different operations. Block cache is typically used for
update operations, while entry cache is most useful for operations that tend to
browse and traverse the tree by reading through entries. Both cache types work to
provide optimal performance for information retrieval and updates. For example,
block cache is helpful when it comes to searching indexes, whereas the entry
cache helps when it comes to retrieving entries referenced from an index.
How Much Memory Does eDirectory Need?
By default, eDirectory will split the allocated “NDS Cache” equally across the
block and entry caches.
In light of the added support for entry caching, cache settings for DS build
versions previous to 8.73 may not be sufficient with new eDirectory 8.5 versions.
For example, if DS build version 8.73 performed correctly with only 50 MB of
cache prior to an upgrade to eDirectory 8.5, you may find that performance has
decreased after the upgrade. This is because only 25 MB is now allowed for block
caching, as opposed to the 50 MB before upgrading. Likewise, the entry caching
may be insufficient with only 25 MB allocated to it.
A p r i l
2 0 0 1
5
To ensure optimal performance, you should configure your system based on cache
fault indicators, database size, and memory available on the server. Don’t expect
to see zero cache faults and don’t assume that it has to be at zero to have optimal
performance in eDirectory. Nevertheless, try to reduce cache faults when the
number of faults per unit of time is high
Note: This is a very broad, general rule of thumb. It could vary significantly
from eDirectory implementation to implementation.
Based on this information, you could use the following formula to calculate the
amount of memory needed:
((SizeOfDIBSet + (SizeOfDIBSet * 4)) = MemoryForNDSDIB
where SizeOfDIBSet equals the bytes for all files that start with NDS in the DIB
directory (for NetWare, this is the SYS:_NETWARE directory).
Once you have figured an amount of memory the database may need, you can
check to see if it exceeds the 60% limit by dividing MemoryForNDSDIB by the
Total Server Memory and multiplying that amount by 100.
If you do exceed the limit, you may want to adjust the multiplier of 4 down to 2
(do not go below 2 on this multiplier). If you still exceed the limit, either install
more memory or prepare for potential performance degradation.
Monitoring and Configuring Cache for eDirectory 8
With eDirectory 8 (not to be confused with eDirectory 8.5), the administrator can
regulate the amount of memory NDS will consume for cache by specifying a
cache limit. If no limit is explicitly set, eDirectory will use a default of 8 MB. To
find out what the current cache settings are, an administrator can turn on
DSTRACE at the console and set DSTRACE to *P.
For example, to view the cache settings on a NetWare server, you would type
following commands at the console:
SET DSTRACE=ON
SET DSTRACE=P*
Then toggle over the Directory Services screen and you should see a display
similar to the following:
6
www.novell.com/appnotes
TUNABLE PARAMETER VALUES
ServerStateUpThreshold = 30 minutes
External Reference Life Span = 193 hours
JanitorInterval = 2 minutes
FlatCleaningInterval = 720 minutes
BacklinkInterval = 780 minutes
Heartbeat Data = 60 minutes
Heartbeat Schema = 240 minutes
Schema synchronization enabled = 1
SMI Max Cache = 60000000 (Allocated = 5283880, Blks In Use = 0)
SMI Entries Cached Per Thread = 50 (Cached: 3, ~= 6912 bytes)
SMI Attr Recs Cached Per Thread = 20 (Cached: 0, ~= bytes)
SMI Partitions Cached Per Thread = 20 (Cached: 5)
SMI Force Checkpoint Interval = 180 (cannot change)
SMI Maximum Read Transaction Seconds = 2400 (cannot change)
SMI Maximum Read Transaction Inactive Seconds = 30 (cannot change)
The “SMI Max Cache” line (bolded above) represents the maximum amount of
the cache allocated to eDirectory. In this example, it is currently set to 60,000,000
bytes or about 60 MB.
If you want to change the cache hard limit for eDirectory 8 (again, do not confuse
this with eDirectory 8.5), you can type the following command to set a hard
maximum cache amount on a NetWare server:
SET DSTRACE=!MB<amount_of_RAM_to_use_in_bytes>
For example, to set a hard limit of 8 MB, you would type the following command
at the NetWare console:
SET DSTRACE=!MB8388608
The number shown is in decimal. The command will set a persistent setting in a
file called _NDSDB.INI found in the SYS:\_NETWARE directory on a NetWare
server.
The following represents the cache parameter that would be set in the
_NDSDB.INI file when using the SET command above for 8 MB:
cache=8388608
A p r i l
2 0 0 1
7
Monitoring and Configuring Cache for eDirectory 8.5
eDirectory 8.5 provides tools for monitoring how well cache is being utilized. The
iMonitor utility (released with eDirectory 8.5) allows an administrator to monitor
cache hits and cache faults for both the block cache and entry cache. A cache hit
occurs whenever eDirectory finds an item in its cache. A cache fault occurs
whenever an item is not found in cache and must be fetched from disk. If there are
lots of cache faults, increasing the amount of cache memory may improve
performance. The ideal system would be one in which there were no cache faults;
however, this is possible only if eDirectory is able to cache the entire database.
With the introduction of eDirectory 8.5, two types of mechanisms for controlling
cache memory consumption are provided. Each is mutually exclusive of the other.
These mechanisms are:
•
Dynamically Adjusting Limit
•
Hard Memory Limit
The administrator can use either one at any time, but the last one used always
replaces any prior settings.
Dynamically Adjusting Limit
The first mechanism that is provided for regulating memory consumption is a
dynamically adjusting limit. A dynamically adjusting limit causes eDirectory to
periodically adjust its memory consumption in response to the needed memory for
caching and flow of memory consumption by other processes. The user specifies
the limit as a percentage of available physical memory. Using this percentage,
eDirectory recalculates a new memory limit at fixed intervals. The new limit is
simply the percentage of physical memory that is available at that point in time.
This limit is used to control cache memory consumption within eDirectory until
the next calculation occurs. Thus, eDirectory’s responsiveness in adjusting its
memory consumption depends on the length of the recalculation interval. The
shorter the interval, the more responsive to change. The longer the interval, the
less responsive to change. Shorter intervals are not necessarily better, because
they will likely cause more memory allocation and freeing to occur.
In connection with the percentage, the user can also set minimum and maximum
thresholds.The minimum is the lowest number of bytes that eDirectory will adjust
down to. If not set, it defaults to 16 MB. The maximum threshold determines the
highest number of bytes that eDirectory will adjust up to. It can be set to either a
number of bytes or number of bytes that eDirectory will leave available. If the
maximum threshold has not been set using one of these two mechanisms, a default
maximum of 4 GB is used.
8
www.novell.com/appnotes
Note: If a minimum threshold is set and a maximum threshold is set as a
number of bytes to leave, it is possible that the calculated maximum will
fall below the minimum. In this case, the minimum takes priority over
the maximum. For example, suppose you have following settings:
Minimum threshold: 8 MB
Percentage: 75
Maximum threshold: leave 10 MB
Suppose that there is 16 MB of available physical memory when
eDirectory decides to adjust its cache limit. eDirectory would calculate a
new limit of 12 MB. It would then check to see if this falls outside the
range of the minimum and maximum thresholds.
In this case, the maximum threshold is a calculated value: 16 MB minus
10 MB yields a maximum threshold of 6 MB. The calculated limit of 12 MB
is greater than this, so eDirectory would set its limit to 6 MB. eDirectory
would then check this limit against the minimum threshold. Since 6 MB is
less than 8 MB, eDirectory would set its final limit to 8 MB. Thus, the
minimum gets priority over the maximum whenever there is a conflict.
Hard Memory Limit
The second mechanism that is provided for regulating memory consumption is the
hard memory limit. This is basically the same mechanism that exists in prior
versions of eDirectory. Once a limit like this is set, it is not changed until the user
either sets a different hard memory limit or sets a dynamically adjusting limit.
Remember that the hard memory limit and dynamically adjusting limits are
mutually exclusive. The last setting is what will apply.
A p r i l
2 0 0 1
9
An administrator is allowed to specify a hard memory limit in one of three ways:
•
A fixed number of bytes
•
Percentage of total physical memory
•
Percentage of available physical memory
When a hard memory limit is specified by percentages, it is always translated to a
fixed number of bytes.
Default Mechanism for Regulating Memory Consumption
If the server contains a replica, eDirectory will use a dynamically adjusting limit
of 51% of available memory, with a minimum of 8 MB and a minimum to leave
24 MB. Otherwise, eDirectory will use a hard memory limit setting of 16 MB.
Settings in the _NDSDB.INI File
At startup, eDirectory looks for the database options file, _NDSDB.INI, in the
directory where DIB files stored. On NetWare servers it is SYS:\_NETWARE.
On Windows NT/2000 servers, this is normally \NOVELL\NDS\DIBFILES. This
file is a simple text file that can be created or modified with any text editor. If you
are using DSTRACE to define your cache settings, they will automatically be
populated in this file.
Note: Although a user may alter the _NDSDB.INI file at any time, the changes
will not take effect until the eDirectory module has been restarted. If
the change is done via the DSTRACE command, no eDirectory module
restart is needed.
Below are the command syntaxes for controlling cache memory consumption
within eDirectory. First, here is the “old-style” command for setting a hard
memory limit:
cache=<cacheBytes>
The new syntax for setting either a hard memory limit or a dynamically adjusting
limit is as follows:
cache=<cache options>
The cache options are listed in the table below. Multiple options may be specified,
in any order, separated by commas. All are optional.
10
www.novell.com/appnotes
Option
Description
DYN or HARD
Dynamically adjusting limit or hard limit
%: <percentage>
Percentage of available or physical memory to use
AVAIL or TOTAL
Percentage is for available memory or total physical memory (Note: This is ignored for a dynamically adjusting limit, because dynamically
adjusting limits are always calculated based on
the available physical memory.)
MIN: <bytes>
Minimum number of bytes
MAX: <bytes>
Maximum number of bytes
LEAVE: <bytes>
Minimum number of bytes to leave
Here are some example settings. All settings must be in lowercase!
To set a dynamically adjusting limit of 60% of available memory, with a
minimum of 16 MB:
To use the “old-style” method to set a hard memory limit of 8 MB:
cache=8000000
cache=DYN,%:60,MIN:16000000
To set a hard memory limit of 60% of total physical memory, with a minimum of
16 MB:
cache=HARD,%:75,MIN:16000000
In addition to the cache setting for regulating memory consumption, eDirectory
also provides settings for controlling the dynamic adjust interval and the interval
for cleaning up older version of entries and blocks. These are as follows:
cacheadjustinterval=<seconds>
cachecleanupinterval=<seconds>
The default for both of these commands is 15 seconds, if not set otherwise in the
_NDSDB.INI file.
A p r i l
2 0 0 1
11
Finally, the following setting allows an administrator to control the percentage
split between entry and block cache:
blockcachepercent=<percent>
The range for this setting is between 0 and 100 (inclusive). A value of 70 means
that 70 percent of cache memory will used for block cache and 30 percent for
entry cache. The default is 50 percent if not set otherwise in the _NDSDB.INI file.
Note: Setting the block cache percent to zero is never a good idea.
DSTRACE Options for Regulating Cache
From the console screen in NetWare, a number of DSTRACE commands can be
used to regulate cache memory consumption within eDirectory.
The following is the syntax for setting a simple hard memory limit:
SET DSTRACE=!MB<bytes>
Here is an example of how to a simple hard memory limit:
SET DSTRACE=!MB8388608
The following is the syntax for setting a calculated hard memory limit:
SET DSTRACE=!MHARD[,AVAIL|TOTAL][,%:<percent>]
[,MIN:<minimum_bytes>][,MAX:<maximum_bytes>]
[,LEAVE:<min_bytes_to_leave>][,NOSAVE
Here is an example of how to set a hard memory limit of 75% of total physical
memory, with a minimum of 16 MB, and specify not to save these options to the
_NDSDB.INI file:
SET DSTRACE=!MHARD,%75,MIN:16777216,NOSAVE
The following is the syntax for setting a dynamically adjusting limit:
SET DSTRACE=!MDYN,%:<percent>[,MIN:<minimum_bytes>]
[,MAX:<maximum_bytes>][,LEAVE:<min_bytes_
to_leave>][,NOSAVE]
Here is an example of how to set a dynamic limit of 75% of available memory,
with a minimum of 8 MB:
SET DSTRACE=!MDYN,%:75,MIN:8388608
Note: Configuring cache via DSTRACE does not require a reset of DS or the
server. Changes go into effect immediately.
12
www.novell.com/appnotes
Using iMonitor to Monitor and Configure Cache Settings
The best way to monitor and configure cache settings is to use iMonitor, which
was released with eDirectory 8.5. Using iMonitor allows you to intuitively view
information about the current cache settings, look at the statistics to determine
possible adjustments, and make the appropriate settings. Best of all, this is done
through an Internet browser for any operating system on the back end (NetWare,
Windows NT/2000, or UNIX/Linux).
Your browser should open up iMonitor if the NDSIMON.NLM is loaded and
running on the eDirectory 8.5 server. To start iMonitor, go to the ConsoleOne
utility and right-click on the server you want to work with. From the resulting
menu, select “Launch iMonitor,” as shown in Figure 1.
Figure 1: Launching the iMonitor utility from ConsoleOne.
Click the Login button at the top left of your browser window, as shown in Figure
2. This will allow you to access the cache settings.
A p r i l
2 0 0 1
13
Figure 2: Logging in allows you to access the cache settings.
Figure 3 displays the Agent Configuration button. Clicking on this button will
show you the Agent Configuration screen.
Figure 3: The Agent Configuration button.
In the Settings options in the left frame, you can now see the Database Cache
selection as illustrated in Figure 4. Click on this selection to go the screen where
you can monitor and make changes to the cache settings for this server.
Figure 4: The Database Cache selection.
14
www.novell.com/appnotes
The screen capture in Figure 5 shows the “Database Information” and “Database
Cache” statistics that are found in the Agent Configuration option of iMonitor.
Figure 5: The Database Cache Configuration screen.
This page allows you to configure and monitor the cache segments being used by
the NDS database subsystem. This covers another set of the old “!” (bang)
commands created for this purpose. (iMonitor’s online context-sensitive help
contains a list of which “!” commands correspond to which iMonitor options.)
The information provided on this page will help you determine whether you have
an adequate cache size and, if not, help give you an idea how much you should
have. For most applications of the Directory, the default behavior and cache
settings will be adequate.
The Database Cache page is divided into three sections: Database Information,
Database Cache Information, and Database Cache Configuration. We’ll first
describe each item in each section, followed by a discussion on how to use that
information.
DIB Size (KB). The size in KB of the NDS database and associated files on disk.
Stream files are part of the associated files but are never cached. These should be
subtracted from the DIB size to compute the size of the NDS database.
DIB Block Size (KB). The size in KB of one database block on disk.
Maximum Size (KB). The maximum size in KB to which the specified cache is
allowed to grow.
Current Size (KB). The current size in KB of the specified cache.
Items Cached. The number of items in the specified cache.
A p r i l
2 0 0 1
15
Old Versions Cached. The number of old versions in the specified cache. Without
getting too technical, old versions of cache items are kept to maintain the
consistency of read transactions in the database. In other words, if one thread is in
a read transaction and another is
in a write transaction, old versions of blocks modified by the writer are
maintained on behalf of the reader. This is done so the reader’s results are
guaranteed to produce a consistent view during the life
of its transaction even though modifications are taking place during that time.
Old Versions Size (KB). The size in KB of the old version items cached.
Hits. The number of times an item was successfully accessed from the specified
cache.
Hit Looks. The number of items looked at in the cache before an item was
successfully accessed from the specified cache. The hit look to hit ratio is a
measure of cache lookup efficiency. Normally, the ratio should be quite close to
1:1.
Faults. The number of times an item was not found in the specified cache and had
to be obtained from a lower level cache or from
the disk.
Fault Looks. The number of items looked at in the cache before it was determined
that the desired item was not in the specified cache. The fault look to fault ratio is
a measure of cache lookup efficiency. Normally, the ratio should be quite close to
1:1.
The Database Cache Configuration Screen
The “Database Cache Configuration” screen found in iMonitor allows you to
easily configure the eDirectory cache for a server. In fact, this is the preferred
method to configure your cache. It allows you to adjust the cache for whatever
server platform you are running, and it prevents you from making mistakes related
to not being able to remember all the commands needed for adjusting the cache
(for example, the DSTRACE functions under NetWare).
16
www.novell.com/appnotes
The Database Cache Configuration screen is shown in Figure 6.
Figure 6: The Database Cache Configuration screen.
As you can see, this screen has two radio buttons for the two methods of
specifying how much cache to allow the NDS database to use: Dynamic Adjust
and Hard Limit. They are mutually exclusive settings.
Following is an explanation of each of the settings on the Database Cache
Configuration screen.
Dynamic Adjust. This setting allows the NDS database to dynamically adjust the
amount of system memory to be used for the cache, based on the amount it thinks
it needs and the parameters specified below.
Cache Adjust Percentage. This specifies the percentage of available memory
allowed to be used for the record and block caches combined.
Cache Size Constraints. While dynamically adjusting, NDS will follow the
constraints specified here: namely, use no less than the specified amount of
memory for the cache, and no more than the total amount of available memory
minus the specified amount.
Hard Limit. This setting allows the NDS administrator to specify the exact
amount of system memory to be use for the cache.
Cache Maximum Size. This is the size in KB of the record and block caches
combined.
Block Cache Percentage. This is the percentage of the system memory available
for caching that should be allocated to the block cache. The remaining percentage
will be allocated to the record cache.
A p r i l
2 0 0 1
17
Cache Adjust Interval. This interval only applies when Dynamic Adjust is set. It
controls how often the cache size is adjusted based on the specified percentage
and constraints.
Cache Cleanup Interval. This setting controls how often unused old versions are
removed from the cache.
Cache Settings Permanent. When checked, any changes submitted via iMonitor
will be made permanent overwriting any previously saved settings or system
defaults.
If no cache settings have been previously specified as permanent, when the
eDirectory 8.5 agent starts up, the cache is defaulted to a hard limit of 16 MB for
the first 10 minutes. Since the eDirectory agent usually loads when a server is
rebooted, this default behavior allows other applications the opportunity to load
and request system resources first. After 10 minutes, the default behavior is to
switch to Dynamic Adjust mode based on the amount of available memory.
The default values for Dynamic Adjust mode are as follows:
Cache Adjust Percentage = 51%
CacheSizeConstraints = > 16MB < (Total Available Memory - 24
MB)
For both Dynamic Adjust and Hard Limit, the default Block Cache Percentage =
50% and the default Cache Adjust and Cache Cleanup Intervals are 15 seconds.
Conclusion
This AppNote has explained how caching works in eDirectory 8 and eDirectory
8.5. It has also presented some maintenance and tuning tips for optimizing the
performance of eDirectory for various situations.
Copyright © 2001 by Novell, Inc. All rights reserved.
No part of this document may be reproduced or transmitted
in any form or by any means, electronic or mechanical,
including photocopying and recording, for any purpose
without the express written permission of Novell.
All product names mentioned are trademarks of
their respective companies or distributors.
18
www.novell.com/appnotes