PDF - IBM Redbooks

Transcription

PDF - IBM Redbooks

Front cover
IBM BladeCenter JS21
The POWER of Blade Innovation
High-performance blade server ideal
for extremely dense HPC clusters
First IBM blade server with
native virtualization for
server consolidation
Exceptional price-performance
per watt for WebSphere with
AIX or Linux OS
Stephen Hochstetler
Benjamin Ebrahimi
Tom Junkin
Bernhard Zeller
ibm.com/redbooks
International Technical Support Organization
IBM BladeCenter JS21: The POWER of Blade
Innovation
October 2006
SG24-7273-00
Note: Before using this information and the product it supports, read the information in
“Notices” on page ix.
First Edition (October 2006)
This edition applies to the IBM BladeCenter JS21, Linux, and IBM AIX 5L Version 5.3, product
number 5765-G03.
© Copyright International Business Machines Corporation 2006. All rights reserved.
Note to U.S. Government Users Restricted Rights -- Use, duplication or disclosure restricted by GSA ADP
Schedule Contract with IBM Corp.
Contents
Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Trademarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
The team that wrote this redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Become a published author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Comments welcome. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Chapter 1. Introduction to BladeCenter and JS21 technology . . . . . . . . . . 1
1.1 BladeCenter terminology. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Benefits of the BladeCenter family . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 BladeCenter versus an IBM rack server . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Introduction to IBM BladeCenter JS21 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Chapter 2. BladeCenter JS21: Hardware components . . . . . . . . . . . . . . . . 7
2.1 BladeCenter JS21 system specifications . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2 Physical specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
2.3 Physical packaging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2.4 BladeCenter JS21 models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.5 Minimum and optional features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.6 Processor features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.7 Memory features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.8 Internal Serial-Attached SCSI disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.9 BladeCenter chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.9.1 BladeCenter power modules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.9.2 BladeCenter management modules . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.9.3 Media features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
Chapter 3. BladeCenter JS21: Software environment . . . . . . . . . . . . . . . . 25
3.1 Operating system support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.1.1 AIX 5L . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.1.2 Red Hat Enterprise Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
3.1.3 SUSE Linux Enterprise Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.2 Virtual input/output Server. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
3.3 System management tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3.1 BladeCenter Web interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3.2 IBM Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3.3 IBM Cluster Systems Management. . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3.4 IBM middleware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
© Copyright IBM Corp. 2006. All rights reserved.
iii
Chapter 4. BladeCenter JS21: Virtualization . . . . . . . . . . . . . . . . . . . . . . . 35
4.1 Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.1.1 Advanced POWER Virtualization . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
4.1.2 Dynamic logical partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
Chapter 5. Planning considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
5.1 Documentation and planning resources . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5.2 Network planning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
5.2.1 Minimal network requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
5.2.2 High-performance, low-latency network requirements . . . . . . . . . . . 51
5.2.3 Multiple BladeCenter environments . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.3 Operating system support and installation . . . . . . . . . . . . . . . . . . . . . . . . 53
5.4 Systems management. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.4.1 BladeCenter Web interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.4.2 IBM Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.4.3 IBM Cluster Systems Management. . . . . . . . . . . . . . . . . . . . . . . . . . 59
Chapter 6. Hardware setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
6.1 BladeCenter chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
6.2 BladeCenter internal network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
6.3 Management module configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
6.3.1 Connecting to the management module . . . . . . . . . . . . . . . . . . . . . . 72
6.3.2 Configuring the management module . . . . . . . . . . . . . . . . . . . . . . . . 75
6.3.3 Advanced management module: Management interface changes . . 79
6.3.4 Management module: Hardware vital product data . . . . . . . . . . . . . 81
6.3.5 Management module redundancy. . . . . . . . . . . . . . . . . . . . . . . . . . . 82
6.3.6 Management module: TCP/IP ports . . . . . . . . . . . . . . . . . . . . . . . . . 83
6.3.7 Resetting the management module . . . . . . . . . . . . . . . . . . . . . . . . . 84
6.4 I/O module configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.4.1 Setting the IP address of an I/O module . . . . . . . . . . . . . . . . . . . . . . 85
6.4.2 Setting external management over all ports . . . . . . . . . . . . . . . . . . . 87
6.4.3 Enabling the external I/O module ports . . . . . . . . . . . . . . . . . . . . . . . 88
6.5 Blade server configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
6.5.1 Assigning names to blade servers . . . . . . . . . . . . . . . . . . . . . . . . . . 90
6.5.2 Setting the boot sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.5.3 Assigning the media tray. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.6 Firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
6.6.1 Getting the latest firmware, tools, and support . . . . . . . . . . . . . . . . . 95
6.6.2 Identifying and updating the latest firmware in general. . . . . . . . . . . 96
6.6.3 Updating BladeCenter JS21 firmware using AIX . . . . . . . . . . . . . . . 99
6.6.4 Updating BladeCenter JS21 firmware (BIOS) using Linux . . . . . . . 101
6.6.5 Updating BladeCenter JS21 Ethernet adapter using Linux. . . . . . . 102
6.6.6 Updating the management module and advanced management module
iv
IBM BladeCenter JS21: The POWER of Blade Innovation
firmware using the Web interface . . . . . . . . . . . . . . . . . . . . . . . . . . 104
6.6.7 Updating I/O module firmware using a Web interface. . . . . . . . . . . 106
6.6.8 Updating IBM BladeCenter 2-port Fibre Channel switch module
firmware using FTP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
6.6.9 Updating the firmware using the stand-alone diagnostics CD . . . . 111
6.7 Providing a console for the BladeCenter JS21 . . . . . . . . . . . . . . . . . . . . 121
6.7.1 Configuring Serial over LAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
6.7.2 Using Serial over LAN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6.8 System Management Services interface . . . . . . . . . . . . . . . . . . . . . . . . . 133
6.8.1 Activating the System Management Services interface . . . . . . . . . 133
6.8.2 Setting up remote initial program load . . . . . . . . . . . . . . . . . . . . . . 135
6.8.3 Selecting boot options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
6.9 Open Firmware interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
6.9.1 Activating the Open Firmware interface . . . . . . . . . . . . . . . . . . . . . 152
6.9.2 Specifying IP parameters to Open Firmware . . . . . . . . . . . . . . . . . 160
6.10 Blade server Ethernet controller enumeration. . . . . . . . . . . . . . . . . . . . 162
6.11 Blade server I/O module routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
6.12 SAS hardware RAID configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
6.12.1 RAID configuration methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
6.12.2 Configuring RAID using the AIX stand-alone diagnostics . . . . . . . 171
6.12.3 Configuring RAID using iprconfig on Linux . . . . . . . . . . . . . . . . . . 183
Chapter 7. Installing and managing the Virtual I/O Server . . . . . . . . . . . 193
7.1 VIOS installation in a JS21 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
7.1.1 VIOS image installation from a DVD . . . . . . . . . . . . . . . . . . . . . . . . 194
7.1.2 VIOS image installation from a NIM server . . . . . . . . . . . . . . . . . . . 194
7.1.3 VIOS image installation from a Linux server . . . . . . . . . . . . . . . . . . 202
7.2 Initial VIOS configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
7.2.1 Virtualization setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209
7.2.2 Setting the date and time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
7.2.3 Initial network setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 210
7.3 VIOS management using IVM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
7.3.1 IVM graphical user interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
7.3.2 IVM command-line interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
7.4 Storage management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216
7.5 VIOS partition configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
7.5.1 Creating the logical volume . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
7.5.2 Creating logical partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
7.5.3 Updating the logical partition profile . . . . . . . . . . . . . . . . . . . . . . . . 231
7.5.4 Powering on a logical partition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
7.5.5 Opening a virtual terminal to a logical partition . . . . . . . . . . . . . . . . 231
7.5.6 Installing an operating system on a logical partition . . . . . . . . . . . . 232
7.5.7 Optical device sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
Contents
v
7.5.8 Logical partition configuration changes . . . . . . . . . . . . . . . . . . . . . . 235
7.6 Network management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
7.7 Advanced storage management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
7.7.1 Virtual storage assignment to a partition . . . . . . . . . . . . . . . . . . . . . 244
7.7.2 Virtual disk extension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
7.7.3 VIOS system disk mirroring. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
7.8 Maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
7.8.1 VIOS maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
7.8.2 Logical partition maintenance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
7.8.3 Command logs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 257
Chapter 8. Installing AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
8.1 General topics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
8.2 Preparing AIX installation from CD/DVD . . . . . . . . . . . . . . . . . . . . . . . . . 260
8.3 Preparing AIX network installation using NIM . . . . . . . . . . . . . . . . . . . . . 261
8.3.1 Configuring AIX environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
8.3.2 Installing the NIM filesets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
8.3.3 Configuring the NIM master . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
8.3.4 Defining the NIM client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
8.3.5 Allocating resources to a client . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
8.3.6 Activating the client installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
8.4 Installing AIX on the client . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
8.5 Changing client from rsh to nimsh . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
Chapter 9. Installing Linux . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281
9.1 Installing Linux from a CD/DVD. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
9.2 Basic preparations for a Linux network installation . . . . . . . . . . . . . . . . . 283
9.2.1 Installing Linux using the network: General remarks . . . . . . . . . . . 284
9.2.2 Configuring a BOOTP or DHCP service . . . . . . . . . . . . . . . . . . . . . 285
9.2.3 Configuring a Trivial File Transfer Protocol service . . . . . . . . . . . . 287
9.2.4 Configuring a File Transfer Protocol service . . . . . . . . . . . . . . . . . . 288
9.3 Installing SLES using the network. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
9.3.1 Overview of SLES network installation . . . . . . . . . . . . . . . . . . . . . . 291
9.3.2 Preparing the installation source directory and service. . . . . . . . . . 293
9.3.3 Basic attended SLES network installation. . . . . . . . . . . . . . . . . . . . 299
9.3.4 Configuring the boot image file with mkzimage_cmdline . . . . . . . . 306
9.3.5 Unattended installation with SLES . . . . . . . . . . . . . . . . . . . . . . . . . 311
9.4 Installing Red Hat Enterprise Linux AS 4 Update 3 using the network . . 321
9.4.1 Overview of Red Hat Enterprise Linux network installation . . . . . . 322
9.4.2 Preparing the installation source for Red Hat Enterprise Linux . . . 324
9.4.3 Basic attended Red Hat Enterprise Linux network installation . . . . 326
9.4.4 Unattended installation with Red Hat Enterprise Linux . . . . . . . . . . 336
Chapter 10. System management scenarios . . . . . . . . . . . . . . . . . . . . . . 345
vi
10.1 BladeCenter Web interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346
10.2 IBM Director . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 346
10.2.1 Installing the components for IBM Director . . . . . . . . . . . . . . . . . . 347
10.2.2 Using the IBM Director Console in a BladeCenter context . . . . . . 355
10.3 Cluster Systems Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
10.3.1 Installing and setting up the CSM management server . . . . . . . . 363
10.3.2 Creating a CSM installation server . . . . . . . . . . . . . . . . . . . . . . . . 373
10.3.3 Defining cluster nodes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
10.3.4 Adding AIX nodes to the cluster . . . . . . . . . . . . . . . . . . . . . . . . . . 377
10.3.5 Adding Linux nodes to the cluster . . . . . . . . . . . . . . . . . . . . . . . . . 379
10.3.6 Testing cluster installation and basic command . . . . . . . . . . . . . . 383
Chapter 11. Performance tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385
11.1 Performance tuning for AIX . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
11.2 Tuning options for Linux on POWER: Selected examples . . . . . . . . . . 386
11.2.1 Disabling unused processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 386
11.2.2 Disabling the graphical user interface . . . . . . . . . . . . . . . . . . . . . . 388
11.2.3 Tuning the kernel parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . 390
11.2.4 Disabling simultaneous multithreading . . . . . . . . . . . . . . . . . . . . . 394
11.2.5 Tuning memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394
11.2.6 Tuning the file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
11.2.7 Selecting the disk technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
11.2.8 Tuning in Transmission Control Protocol and
User Datagram Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
11.2.9 Performance tuning tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
11.2.10 Tuning for applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
Chapter 12. Storage area network installation and booting . . . . . . . . . . 409
12.1 Storage area network setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 410
12.1.1 Assigning a disk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411
12.1.2 SAN boot characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
Appendix A. SUSE Linux Enterprise Server 9 AutoYaST XML file . . . . . 419
Appendix B. Red Hat Enterprise Linux 4 Kickstart file . . . . . . . . . . . . . . 427
Appendix C. SUSE Linux Installation Server setup . . . . . . . . . . . . . . . . . 431
Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
Related publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 439
Other publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
Online resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 440
How to get IBM Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
Contents
vii
Help from IBM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445
viii
Notices
This information was developed for products and services offered in the U.S.A.
IBM may not offer the products, services, or features discussed in this document in other countries. Consult
your local IBM representative for information on the products and services currently available in your area.
Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM
product, program, or service may be used. Any functionally equivalent product, program, or service that
does not infringe any IBM intellectual property right may be used instead. However, it is the user's
responsibility to evaluate and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document.
The furnishing of this document does not give you any license to these patents. You can send license
inquiries, in writing, to:
IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.
The following paragraph does not apply to the United Kingdom or any other country where such provisions
are inconsistent with local law: INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES
THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED,
INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT,
MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do not allow disclaimer
of express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made
to the information herein; these changes will be incorporated in new editions of the publication. IBM may
make improvements and/or changes in the product(s) and/or the program(s) described in this publication at
any time without notice.
Any references in this information to non-IBM Web sites are provided for convenience only and do not in any
manner serve as an endorsement of those Web sites. The materials at those Web sites are not part of the
materials for this IBM product and use of those Web sites is at your own risk.
IBM may use or distribute any of the information you supply in any way it believes appropriate without
incurring any obligation to you.
Information concerning non-IBM products was obtained from the suppliers of those products, their published
announcements or other publicly available sources. IBM has not tested those products and cannot confirm
the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on
the capabilities of non-IBM products should be addressed to the suppliers of those products.
This information contains examples of data and reports used in daily business operations. To illustrate them
as completely as possible, the examples include the names of individuals, companies, brands, and products.
All of these names are fictitious and any similarity to the names and addresses used by an actual business
enterprise is entirely coincidental.
COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrates programming
techniques on various operating platforms. You may copy, modify, and distribute these sample programs in
any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the
sample programs are written. These examples have not been thoroughly tested under all conditions. IBM,
therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy,
modify, and distribute these sample programs in any form without payment to IBM for the purposes of
developing, using, marketing, or distributing application programs conforming to IBM's application
programming interfaces.
ix
Trademarks
The following terms are trademarks of the International Business Machines Corporation in the United States,
other countries, or both:
3090™
AIX 5L™
AIX®
BladeCenter®
Chipkill™
DB2®
eServer™
i5/OS®
IBM®
iSeries™
Micro-Partitioning™
Power Architecture™
POWER Hypervisor™
Power PC®
POWER5™
PowerPC Architecture™
PowerPC®
POWER™
pSeries®
Redbooks (logo)
™
Redbooks™
RETAIN®
RS/6000®
ServerProven®
System p5™
System p™
System z9™
System z™
System/360™
Tivoli®
Virtualization Engine™
WebSphere®
xSeries®
z9™
zSeries®
The following terms are trademarks of other companies:
Java, Power Management, Solaris, and all Java-based trademarks are trademarks of Sun Microsystems,
Inc. in the United States, other countries, or both.
Microsoft, Windows, and the Windows logo are trademarks of Microsoft Corporation in the United States,
other countries, or both.
i386, Intel, MMX, Intel logo, Intel Inside logo, and Intel Centrino logo are trademarks or registered
trademarks of Intel Corporation or its subsidiaries in the United States, other countries, or both.
UNIX is a registered trademark of The Open Group in the United States and other countries.
AltiVec is a trademark of Freescale Semiconductor, Inc.
Linux is a trademark of Linus Torvalds in the United States, other countries, or both.
Other company, product, or service names may be trademarks or service marks of others.
x
Preface
Blade servers have shown themselves to be an efficient, powerful technology.
They have captured the industry focus because of their modular design, which
can reduce cost with a more efficient use of valuable floor space. They offer
simplified management, which can help to speed up tasks such as installing,
provisioning, updating, and troubleshooting hundreds of blade servers. You can
do all of this remotely using one graphical console with IBM® Director systems
management tools or cluster management software such as Cluster Systems
Management (CSM).
In addition, blade servers provide improved performance by doubling current
rack density. By integrating resources and sharing key components, costs
decrease and availability increases. The IBM BladeCenter® boasts innovative
modular technology, leadership density, and availability. It is designed to help
solve a multitude of real-world problems.
This IBM Redbook takes an in-depth look at the IBM BladeCenter JS21. This is a
2-core or 4-core blade server for applications requiring 64-bit computing. It is
ideal for computer-intensive applications and transactional Internet servers. This
book helps you to install, tailor, and configure the IBM BladeCenter JS21 running
either IBM AIX® 5L™ or the Linux® operating system (OS).
This document expands the current set of IBM BladeCenter JS21 documentation
by providing a desktop reference that offers a detailed technical description of
the JS21 system. This publication does not replace the latest IBM BladeCenter
JS21 marketing materials and tools. It is intended as an additional source of
information that you can use, together with existing sources, to enhance your
knowledge of IBM blade solutions.
The team that wrote this redbook
This redbook was produced by a team of specialists from around the world
working at the International Technical Support Organization (ITSO), Austin,
Texas.
Stephen Hochstetler is a Senior I/T Specialist at the ITSO, Austin Center. He
writes extensively and teaches IBM classes worldwide on all areas of
IBM System p™ and Linux. Before joining the ITSO six years ago, Stephen
worked in IBM Tivoli® Services, U.S., as a network management solution
architect.
xi
Benjamin Ebrahimi is a Staff Engineer in the IBM Development Laboratory in
Boeblingen, Germany, for the Open Systems Design and Development Group
since 2006. He has more than 10 years of experience in planning, integration,
installation, and development for multiple operating systems and architectures.
He holds a degree in General Engineering Science and Electrical Engineering
from the Hamburg University of Technology. His areas of expertise include
Linux, Microsoft® Windows®, IBM AIX, 16-bit, 32-bit, and 64-bit computer
architectures, different networks, for example, wide area network (WAN), local
area network (LAN), wireless LAN (WLAN), storage area network (SAN), and
development of programs, drivers, firmware, protocols, and hardware. Recently
he has been involved in JS21/cell development supporting JS21/cell installations
and infrastructure during system test and early customer installations.
Tom Junkin is a Senior I/T Specialist in the IBM Americas Techline
Organization. He has been with IBM for 35 years, with 15 years in the field
variously as a Workstation Specialist, Systems Engineer, and Investment
Territory Representative. He has also been a Program Manager for IBM federal
contract management. He attended the University of South Carolina. His areas
of expertise include pre-sales support for IBM RS/6000®, System p, AIX, and
server consolidation in the federal industry.
Bernhard Zeller is an I/T Specialist for Technical Sales System p in the IBM
Systems & Technology Group (STG), IMT Germany, in Mannheim. He has
17 years of experience in the AIX 5L and IBM System p field. He holds a degree
in electronics engineering from Fachhochschule Aalen. He has worked in IBM for
17 years. His areas of expertise include networking, operating systems, server
management, and speech synthesis technology.
Thanks to the following people for their contributions to this project:
Betsy Thaggard
ITSO, Austin Center
Donn Bullock
IBM, Raleigh
Kerry C. Anders
Walter Butler
Catherine Kostetsky
Frank Petsche
IBM, Austin
xii
Michael Long
Sanjay Verma
IBM, Beaverton
Torsten Bloth
IBM Germany
Become a published author
Join us for a two- to six-week residency program! Help write an IBM Redbook
dealing with specific products or solutions, while getting hands-on experience
with leading-edge technologies. You'll team with IBM technical professionals,
Business Partners and/or customers.
Your efforts will help increase product acceptance and customer satisfaction. As
a bonus, you'll develop a network of contacts in IBM development labs, and
increase your productivity and marketability.
Find out more about the residency program, browse the residency index, and
apply online at:
ibm.com/redbooks/residencies.html
Comments welcome
Your comments are important to us!
We want our IBM Redbooks™ to be as helpful as possible. Send us your
comments about this or other Redbooks in one of the following ways:
Use the online Contact us review redbook form found at:
ibm.com/redbooks
Send your comments in an email to:
[email protected]
Mail your comments to:
IBM Corporation, International Technical Support Organization
Dept. HYTD Mail Station P099
2455 South Road
Poughkeepsie, NY 12601-5400
Preface
xiii
xiv
1
Chapter 1.
Introduction to BladeCenter
and JS21 technology
This chapter presents an overview of the IBM BladeCenter and blade server
technology. It describes the following topics:
1.1, “BladeCenter terminology” on page 2
1.2, “Benefits of the BladeCenter family” on page 2
1.3, “BladeCenter versus an IBM rack server” on page 3
1.4, “Introduction to IBM BladeCenter JS21” on page 4
1
1.1 BladeCenter terminology
The term blade chassis refers to a chassis that can hold a number of
hot-swappable devices called blades. Blades come in two varieties: blade
servers and option blades.
A blade server is an independent server that contains one or more processors
and associated memory, disk storage, and network controllers. It runs its own
operating system and applications. Each blade server within a system chassis
slides into a blade bay. It plugs into a midplane or backplane to share common
infrastructure components. These components can include power supplies, fans,
CD-ROM, and diskettes, Ethernet and Fibre Channel switches, and system
ports.
Option blades can be sharable by blade servers. They provide additional
features, such as controllers for external input/output (I/O) or disk arrays, power
supplies, and so on.
1.2 Benefits of the BladeCenter family
For organizations that seek server consolidation, BladeCenter centralizes
servers for increased flexibility, ease of maintenance, reduced cost, and
streamlined human resources. Companies that have to install new applications
for e-commerce and on-demand business can achieve speed while ensuring
flexibility, scalability, and availability. For enterprise requirements such as
file-and-print and collaboration, BladeCenter is designed to offer reliability,
flexibility for growth, and cost-effectiveness. Clients with compute-intensive
applications that require highly available clustering can use the BladeCenter to
help achieve high degrees of scalability and performance.
Lower cost in certain configurations: Due to efficiencies in power usage,
heat emissions, and data center floor space utilization, when a configuration
of other servers exceeds one rack, it is often less expensive over a multi-year
period to use a single rack of BladeCenter chassis and multiple BladeCenter
JS21.
Physical server consolidation: You can use a BladeCenter server as an
ideal replacement for many uniprocessor or two-way servers to save space.
You can replace 14U of rack space for 14 1U servers with one 7U
BladeCenter chassis. Additional rack space that is normally taken for
Ethernet, Fibre Channel, and other switches is eliminated by the integrated
switches in the BladeCenter chassis.
2
High availability: The BladeCenter chassis offers redundant and hot-swap
components that can prevent failure of the chassis or blade servers when one
of these components fails. The chassis has redundant, hot-swap cooling,
power, midplane features, management modules, and I/O switches. It also
has a hot-swap media tray, with CD-ROM and diskette, that can be removed
and serviced while all blade servers are operating.
Integrated switch technology: All service is performed from the front and
rear of the BladeCenter. Therefore, there is no need to slide the chassis out of
the rack and remove the top cover for service. Also, numerous cables are
eliminated, reducing both cabling cost and servicer or administrator time.
More integrated systems management features: The BladeCenter chassis
includes a management module. This module eliminates the need for
individual management adapters, such as the Remote Supervisor Adapter I
or Remote Supervisor Adapter II, in each blade server for remote control. It
also eliminates the need for RS485 interconnect cabling between the blade
servers.
SAN optimization: A Fibre Channel switch module installed in a BladeCenter
chassis provides storage area network (SAN) access to all blade servers in
the chassis without internal cabling.
1.3 BladeCenter versus an IBM rack server
BladeCenter is an ideal solution for certain environments. In other environments,
an IBM rack server might be a better fit. You can choose servers in a rack in the
following cases:
Need for a small number of servers: A BladeCenter chassis is required for
one blade. To be cost-effective compared to certain stand-alone servers, you
require multiple blades in a chassis. Therefore, a chassis must be full or
nearly full.
Need for existing Peripheral Component Interconnect (PCI) adapters:
BladeCenter does not include adapter slots as shipped. An optional I/O
expansion card feature offers one PCI slot per blade, limiting the blade to one
integrated development environment (IDE) drive. The expansion feature
supports PCI cards designed for BladeCenter. Therefore, a mix of
BladeCenter and traditional rack-optimized servers might be appropriate.
Need for large internal storage or (non-SAN) external Small Computer
System Interface (SCSI) storage: The BladeCenter JS21 supports a
maximum of 146 gigabytes (GB) of Serial-Attached SCSI (SAS) storage
internally. While there is no provision for external SCSI storage, there is an
external Fibre Channel SAN storage option. Using the optional internal SCSI
Chapter 1. Introduction to BladeCenter and JS21 technology
3
storage feature doubles the space requirement of the blade, reducing by half
the number of blade servers that can be installed in a chassis.
Need for re-installation or re-purposing at end of original: You can
distribute stand-alone uniprocessor and 4-core servers individually for use as
departmental file/print servers and other low-horsepower uses. Because you
cannot use blade servers without a chassis, the entire chassis-and-blades
combination has to stay together. The original chassis can be useful if older
blade servers are discarded and newer, faster ones take their place in the
chassis.
Power requirements: The data center is wired for 110 volts (V) - 120 V
power. BladeCenter requires 220 V -240 V power.
Learn more about IBM BladeCenter and its components in The Cutting Edge:
IBM eServer BladeCenter, REDP-3581.
1.4 Introduction to IBM BladeCenter JS21
The IBM BladeCenter JS21 Type 8844 blade server offers significant
improvements in performance and reliability over the previous generation JS20
blade server, delivering up to three times better performance by using the
following features:
The faster IBM PowerPC® 970MP dual-core processors
Faster, more reliable double data rate 2 (DDR2) memory options
SAS hard disk drives
The first IBM blade server optimized for BladeCenter H
With built-in support for Advanced Performance Optimization with Enhanced
RISC (IBM POWER™) Virtualization, the BladeCenter JS21 offers an ideal blade
server solution for high-performance computing (HPC), AIX 5L, and server
consolidation.
The BladeCenter JS21 supports a maximum of 16 GB 533 MHz error-checking
and correction (ECC) IBM Chipkill™ DDR2 memory. Model 31X, with 1 GB
memory standard, offers two single active core, 64-bit PowerPC 970MP
processors. The 51X, with 2 GB memory standard, offers two dual-core, 64-bit
PowerPC 970MP processors. Each processor core includes 32/64 KB L1
(data/instruction) and 1 MB (non-shared) L2 cache. Processor clock rates vary
based on the model of the BladeCenter chassis that the BladeCenter JS21 is
inserted into. For processor speed details, see Table 2-7 on page 19.
Designed with the demands of enterprise and scientific computing in mind, the
BladeCenter JS21 is a highly differentiated solution for high-performance Linux
clusters, seismic analysis for oil and gas, UNIX® applications for retail and
4
finance, Web serving such as with IBM WebSphere®, grid solutions, and any
other custom or commercial application able to exploit the performance
acceleration of AltiVec technology for data-intensive workloads. The
BladeCenter JS21 represents a convergence of leadership technologies and is
therefore a critical component of the IBM BladeCenter and System p server
portfolio. The BladeCenter JS21 allows customers the ability to better use the
value proposition of the BladeCenter design combined with the reliability of AIX
5L and enterprise Linux.
The BladeCenter JS21 delivers outstanding deployment flexibility. You can
install it in the BladeCenter or BladeCenter H chassis to optimize your current
and future investments. As a heterogeneous infrastructure consolidation
platform, all BladeCenter chassis also support running the BladeCenter JS21
alongside your Intel®-based HS blades and Advanced Micro Devices (AMD)
Opteron-based LS blades in the same chassis with independent monitoring,
security, power, and systems management.
The BladeCenter JS21 includes power management capabilities to allow the
maximum uptime and performance possible for your servers. Built for speed and
reliability, the BladeCenter JS21 supports a choice of operating systems for
running HPC Linux clusters as well as AltiVec-optimized applications on AIX 5L
or Linux. Virtualization support is standard on the BladeCenter JS21. Therefore,
server and workload consolidation of multiple independent applications on a
single blade can be provided by Advanced POWER Virtualization for AIX 5L and
Linux environments by ordering Virtual I/O Server (VIOS) V1.2.1.
Chapter 1. Introduction to BladeCenter and JS21 technology
5
6
2
Chapter 2.
BladeCenter JS21: Hardware
components
This chapter describes the major hardware components associated with the IBM
BladeCenter JS21. It begins by reviewing the core BladeCenter infrastructure. It
also provides a detailed description of the BladeCenter JS21 and the range of
options that you can order from IBM.
7
2.1 BladeCenter JS21 system specifications
The IBM BladeCenter JS21 Type 8844 blade server offers significant
improvements in performance and reliability over the previous generation JS20
blade server, delivering up to three times better performance by using the
following features:
The faster IBM PowerPC 970MP dual-core processors
Faster, more reliable double data rate 2 (DDR2) memory options
Serial-Attached SCSI (SAS) hard disk drives
The first IBM blade server optimized for BladeCenter H
The BladeCenter JS21 supports a maximum of 16 gigabytes (GB) 400 MHz
error-checking and correction (ECC) Chipkill DDR2 memory or 8 GB 533 MHz
ECC Chipkill DDR2 memory. Model 31X, with 1 GB memory standard, offers two
single active core, 64-bit PowerPC 970MP processors. The 51X, with 2 GB
memory standard, offers two dual-core, 64-bit PowerPC 970MP processors.
Each processor core includes 32/64 KB L1 (data/instruction) and 1 MB
(non-shared) L2 cache. Processor clock rates vary based on the model of
BladeCenter chassis that the BladeCenter JS21 is inserted into. For processor
speed details, see Table 2-7 on page 19.
The BladeCenter JS21 includes power management capabilities to allow the
maximum uptime and performance possible for your servers. Built for speed and
reliability, the BladeCenter JS21 supports a choice of operating systems for
running HPC Linux clusters as well as AltiVec-optimized applications on AIX 5L
or Linux. Virtualization support is standard on the BladeCenter JS21. Therefore,
server and workload consolidation of multiple independent applications on a
single blade can be provided by Advanced POWER Virtualization for AIX 5L and
Linux environments by ordering Virtual I/O Server (VIOS) V1.2.1.
8
Table 2-1 describes the general system specifications of the BladeCenter JS21.
Table 2-1 BladeCenter JS21specifications
Description
Range
Operating temperature
Relative humidity
8% to 80%
Maximum altitude
2,133 m (7,000 ft)
Operating voltage
Operating frequency
50 Hz or 60 Hz
Maximum power consumption
10 to 35 degrees C (50 to 95 degrees F)
10 to 32 degrees C (50 to 90 degrees F)
BladeCenter (8677) 200-240 V ac
BladeCenter H (8852) 12.2 V dc
31x: 283 watts
51x: 308 watts
2.2 Physical specifications
Table 2-2 describes the physical specifications of the BladeCenter JS21.
Table 2-2 BladeCenter JS21 physical specifications
8844-31X
8844-51X
Processor
2-socket single active core 64-bit
PowerPC 970MP
2-socket dual-core 64-bit
PowerPC 970MP
Number of cores
2
4
L2 cache
1 MB/processor core
1 MB/processor core
Memory (400/533 MHz
synchronous dynamic random
access memory (SDRAM))
Hard disk drives (HDD)
controller
Integrated SAS or Redundant
Array of Independent Disks
(RAID)
Integrated SAS or RAID
Internal capacity
36/73 GB - 146 GB maximum
36/73 GB - 146 GB maximum
Integrated Ethernet controller
Dual 1 gigabit (Gb)
Dual 1 Gb
1 GB/16 GB 400 MHz
1 GB/8 GB 533 MHz
(1 GB/16 GB 533 MHz)
2 GB/16 GB 400 MHz
2 GB/8 GB 533 MHz
(2 GB/16 GB 533 MHz)
The processor operating frequency depends on the BladeCenter chassis, as
shown in Table 2-7 on page 19.
Chapter 2. BladeCenter JS21: Hardware components
9
2.3 Physical packaging
Table 2-3 describes the major physical attributes of the BladeCenter JS21.
Table 2-3 BladeCenter JS21 physical packaging
Dimension
Height
245 mm (9.7 inches)
Width
29 mm (1.14 inches)
Depth
446 mm (17.6 inches)
Weight
Minimum configuration
5.4 kg (12 lb) depending on the configuration
Figure 2-1 shows the JS21 with the cover off. You can see the two processors
and the memory slots completely full. The two optional SAS disks are installed.
Figure 2-1 View of the BladeCenter JS21 with the cover off
10
2.4 BladeCenter JS21 models
There are two models of the BladeCenter JS21. Both models have similar base
features with the exception of the microprocessors used in the processor
subsystem and memory options available.
The JS21 Type 8844-31X blade server with the PowerPC 970MP
single-active core microprocessor
The JS21 Type 8844-51X blade server with the PowerPC 970MP dual-core
microprocessor
Table 2-4 is a summary of the new BladeCenter JS21 models.
Table 2-4 Summary of JS21 models
High-density blade
Blade description
Family
JS21
Family of 64-bit PowerPC technology based blade servers, running
64-bit AIX 5L or Linux operating systems
Models
8844-31X
2-core PowerPC 970 Blade
2 x single active core PowerPC 970MP
2 x 512 MB 400 MHz memory dual inline memory modules
(DIMMs)
Operating frequency depends on the chassis type
8844-51X
4-core PowerPC 970 Blade
2 x dual-core PowerPC 970MP
2 x 1 GB 400 MHz DIMMs
Operating frequency depends on the chassis type
2.5 Minimum and optional features
Table 2-5 lists the standard configurations that the BladeCenter JS21 supports.
Table 2-5 Standard BladeCenter JS21 standard configuration
Model
Processor
L2 cache
Memory
Ethernet
HDD
8844-31X
2-socket single active core
2.7 GHz/2.6 GHz
1 MB/core
1 GB
(2 x 512 MB)
Dual Gb
Open
8844-51X
2-socket dual-core
2.5 GHz/2.3 GHz
1 MB/core
2 GB
(2 x 1 GB)
Dual Gb
Open
11
Restriction: The processor frequency depends on the type of BladeCenter
chassis that the BladeCenter JS21 is installed in, as described in Table 2-7 on
page 19.
Figure 2-2 shows the memory, SAS disk, and expansion option connectors,
which are the same for the 31X and 51X models.
Figure 2-2 BladeCenter JS21 connectors
The following list provides some of the optional features available on the
BladeCenter JS21. For a complete list of the supported modules and adapters,
see the following IBM ServerProven® Web site:
http://www.ibm.com/servers/eserver/serverproven/compat/us/eserver.html
Up to 16 GB of system memory
Up to 2 internal hard disk drives for up to 146 GB of internal storage
Support for small-form-factor ((SFF), 2.5”) 36 GB/73 GB SAS
10,000 revolutions per minute HDD
Standard or SFF Gigabit Ethernet expansion cards
12
Topspin Host Channel Adapter expansion card
QLogic iSmall Computer System Interface (iSCSI) expansion card
QLogic 4Gb SFF Fibre Channel expansion card
Myrinet Cluster expansion card
2.6 Processor features
This section provides more detailed information about the IBM PowerPC 970MP
microprocessor, which is used for the main processors on the BladeCenter JS21.
The BladeCenter JS21 leverages the high-performance, low-power 64-bit
IBM PowerPC 970MP microprocessor in either single-active core or dual-core
configurations.
Two processor modules are supported per single-wide BladeCenter JS21.
They are directly mounted to the blade planar board providing
multi-processing capability.
Each processor core includes 32/64 KB L1 (data/instruction) and 1 MB
(non-shared) L2 cache.
The 4-core configuration comprises two dual-core PowerPC 970MP
processors, and the 2-core configuration comprises two single-active core
PowerPC 970MP processors.
The PowerPC 970MP is the higher frequency, dual-core capable, 90 nm
follow-on to the single-core capable PowerPC 970FX 2.2 GHz
microprocessor used in the previous generation BladeCenter JS20.
The processor operating frequency depends on the BladeCenter chassis, as
shown in Table 2-7 on page 19.
AltiVec
The AltiVec is an extension to the IBM PowerPC Architecture™. It defines
additional registers and instructions to support single-instruction multiple-data
(SIMD) operations that accelerate data-intensive tasks.
A short history of vector processing
The basic concept behind vector processing is to enhance the performance of
data-intensive applications by providing hardware support for operations that can
manipulate an entire vector (or array) of data in a single operation. The number
of data elements operated upon at a time is called the vector length.
13
Scalar processors perform operations that manipulate single data elements such
as fixed-point or floating-point numbers. For example, scalar processors usually
have an instruction that adds two integers to produce a single-integer result.
Vector processors perform operations on multiple data elements arranged in
groups called vectors (or arrays). For example, a vector add operation to add
two vectors performs a pairwise addition of each element of one source vector
with the corresponding element of the other source vector. It places the result in
the corresponding element of the destination vector. Typically a single vector
operation on vectors of length n is equivalent to performing n scalar operations.
Figure 2-3 illustrates the difference between scalar and vector operations.
Vector Add Operation
Scalar Add Operation
4
+
7
11
7
1
6
3
3
10
+
5
6
11
4
5
2
12
7
17
7
8
12
Figure 2-3 Scalar and vector operations
Processor designers are continually looking for ways to improve application
performance. The addition of vector operations to a processor architecture is one
method that a processor designer can use to make it easier to improve the peak
performance of a processor. However, the actual performance improvements
that can be obtained for a specific application depend on how well the application
can exploit vector operations.
The concept of vector processing has existed since the 1950s. Early
implementations of vector processing (known as array processing) were installed
in the 1960s. They used special purpose peripherals attached to general
purpose computers. An example is the IBM 2938 Array Processor, which could
be attached to some models of the IBM System/360™. This was followed by the
IBM 3838 Array Processor in later years.
By the mid-1970s, vector processing became an integral part of the main
processor in large supercomputers manufactured by companies such as Cray
Research. By the mid-1980s, vector processing became available as an optional
feature on large general-purpose computers such as the IBM 3090™.
In the 1990s, developers of microprocessors used in desktop computers adapted
the concept of vector processing to enhance the capability of their
microprocessors when running desktop multimedia applications. These
14
capabilities were usually referred to as SIMD extensions and operated on short
vectors. Examples of SIMD extensions in widespread use today include:
Intel Multimedia Extensions (MMX™)
Intel Streaming SIMD Extensions (SSE)
AMD 3DNow!
Motorola AltiVec
The SIMD extensions in microprocessors that are used in desktop computers
operate on short vectors of length 2, 4, 8, or 16. This is in contrast to the classic
vector supercomputers that can often exploit long vectors of length 64 or more.
AltiVec extensions to PowerPC Architecture
The AltiVec extensions to PowerPC Architecture add a vector processor (VXU)
to the PowerPC logical processing model. This is illustrated in Figure 2-4.
Instructions
from Storage
Fixed-Point
Instructions
Fixed-Point
Processing
(FXU)
Branch
Processing
(BPU)
Floating-Point
Instructions
FloatingPoint
Processing
(FPU)
Vector
Instructions
Vector
Processing
(VXU)
Data to/from
Storage
Storage
Figure 2-4 PowerPC with AltiVec logical processing model
The VXU operates on vectors that are a total of 128 bits long. These can be
interpreted by the VXU as either:
A vector of sixteen 8-bit bytes
A vector of eight 16-bit half words
A vector of four 32-bit words
15
The AltiVec extensions to PowerPC Architecture define 32 vector registers that
form the vector register file (VRF). The VRF is architecturally distinct from the
standard PowerPC floating-point registers (FPRs) and general purpose registers
(GPRs).
The AltiVec extensions to PowerPC also define two additional registers:
The AltiVec status and control register, which is used to control the operation
of the VXU and report the status of some AltiVec operations
The VRSAVE register, which can be used to assist the operating system save
state across context switches by providing a mechanism for software to
indicate what vector registers are in use
Figure 2-5 illustrates the additional registers provided by AltiVec.
VRSAVE Register
VRSAVE
0
31
VSCR
0
Vector Status and Control Register
31
Vector Register File (VRF)
VR0
VR1
...
VR31
0
127
Figure 2-5 AltiVec registers
The AltiVec extensions to PowerPC Architecture define new instructions that use
the VXU to manipulate vectors stored in the VRF. These instructions fall into the
following categories:
Vector integer arithmetic instructions (on 8-bit, 16-bit, or 32-bit integers)
Vector floating-point arithmetic instructions (32-bit only)
Vector load and store instructions
Vector permutation and formatting instructions
Processor control instructions used to read and write from the AltiVec status
and control register
Memory control instructions used to manage caches
16
For additional information about the topics presented in this chapter, we
recommend that you read the PowerPC Microprocessor Family: AltiVec
Technology Programming Environments Manual. You can find this manual on
the Web at:
http://www-306.ibm.com/chips/techlib/techlib.nsf/techdocs/FBFA164F82437
0F987256D6A006F424D
For technical details about AltiVec, visit one of the following Web sites:
Unrolling AltiVec, Part 1: Introducing the PowerPC SIMD unit
http://www.ibm.com/developerworks/power/library/pa-unrollav1/
High-Performance Processors: Altivec Technology
http://www.freescale.com/files/32bit/doc/fact_sheet/ALTIVECFACT.pdf
AltiVec - Wikipedia
http://en.wikipedia.org/wiki/AltiVec
2.7 Memory features
The BladeCenter JS21 supports four DIMM slots with two-way interleaving for
pairs of 400 MHz or 533 MHz DDR2 SDRAM DIMMs including ECC and Chipkill.
The available DIMM sizes range from 512 MB to 4 GB. The minimum memory
size on the BladeCenter JS21 is 1 GB and the maximum memory size is 16 GB.
With two-way interleaving, memory must be populated two DIMMs at a time, and
the minimum requirement is two DIMMS.
The 2-core BladeCenter JS21 ships with 1 GB main memory (two PC2-3200 512
MB DIMMs). The 4-core BladeCenter JS21 ships with 2 GB main memory (two
PC2-3200 1 GB DIMMs).
17
Memory DIMMs
Table 2-6 lists the supported memory on the BladeCenter JS21.
Table 2-6 JS21 support memory DIMM types
DIMM size
PC2-3200 (400 MHz)
PC2-4200 (533 MHz)
IBM PN
Option PN
FRU PN
IBM PN
Option PN
FRU PN
512 MB
38L5914
73P2865
73P2869
38L5919
41Y2707
41Y2708
1 GB
38L5915
39M5809
39M5808
38L5920
41Y2711
41Y2710
2 GB
38L5916
39M5812
39M5811
38L5921
41Y2715
41Y2714
4 GB
38L5918
41Y2703
41Y2702
38L5922
41Y2723
41Y2722
2.8 Internal Serial-Attached SCSI disk
The BladeCenter JS21 provides support for up to two internal hard disk drives.
Hard disk drives
The base BladeCenter JS21 does not ship with any hard disk drives. You can
order up to two of the 2.5-inch SFF SAS hard disk drive options. The
BladeCenter JS21 can have up to two 73 GB SAS hard disk drives for a
maximum of 146 GB of internal storage or up to two of the 36 GB SAS hard disk
drives. The BladeCenter JS21 also supports Redundant Array of Independent
Disks (RAID 0 or 10) mirroring standard.
You do not have to install a hard disk drive if you have installed the QLogic 4Gb
SFF Fibre Channel expansion card and configured the BladeCenter JS21 to boot
from storage area network (SAN).
Restriction: Certain input/output (I/O) expansion cards that are installed
might preclude the attachment of the second SAS hard disk drive unless the
I/O expansion card is an SFF card. In this case, both the SFF I/O expansion
card and the second SAS HDD can be installed at the same time.
18
2.9 BladeCenter chassis
The core component of the BladeCenter infrastructure is the BladeCenter
chassis. Each BladeCenter chassis occupies seven rack units. With an additional
two rack units to accommodate high-speed switches, the BladeCenter H chassis
occupies nine rack units. You can install up to six BladeCenter or up to four
BladeCenter H chassis in a single 42U rack. The blade server processor
operating frequency depends on the BladeCenter chassis on which it is installed,
as shown in Table 2-7.
Table 2-7 Processor operating frequency based on the BladeCenter
Chassis and machine type
JS21 8844-31X
JS21 8844-51X
BladeCenter (8677)
2.6 GHz
2.3 GHz
BladeCenter H (8852)
2.7 GHz
2.5 GHz
BladeCenter T (8720)
2.6 GHz
2.3 GHz
BladeCenter chassis (8677)
This BladeCenter is a 7U rack mountable chassis that contains bays for up to
14 blade servers, four power modules, two switch modules, and two
management modules. Figure 2-6 shows the front view of the BladeCenter.
Figure 2-6 BladeCenter front view
19
Figure 2-7 shows a rear view of the BladeCenter.
Figure 2-7 BladeCenter rear view
This BladeCenter supports the following power-module options:
IBM BladeCenter 1200 watt Power Supply Module (part number 48P7052)
IBM BladeCenter 1200 watt to 1400 watt Power Supply Upgrade Kit (part
number 90P0197)
IBM BladeCenter 1800 watt Power Supply Module (part number 13N0570)
IBM BladeCenter 2000 watt Power Supply Module (part number 26K4816)
If the existing power modules are replaced with the 2000 watt power modules,
you must upgrade the management module firmware. If two management
modules are installed in the BladeCenter chassis, upgrade both management
modules to the same level of firmware.
20
If a BladeCenter JS21 is being installed in the 8677-1xx/2xx BladeCenter, the
power module upgrade guidelines contain a table identifying the power load
factor of all possible blades. You can find the guidelines at the following Web
address:
http://www.ibm.com/pc/support/site.wss/document.do?sitestyle=ibm&lndoci
d=MIGR-53353
BladeCenter H (8852)
This BladeCenter H is a 9U rack mountable chassis that contains bays for up to
14 blade servers, four power modules, four switch modules, four high-speed
switch modules, four high-speed bridge modules, and two management
modules. The BladeCenter H requires 2900 watt hot-swap redundant power
supply modules. Figure 2-8 shows the front view of this chassis.
Figure 2-8 Front view of BladeCenter H
21
Figure 2-9 shows the rear view of this chassis.
Figure 2-9 Rear view of BladeCenter H
22
Figure 2-10 shows a rear view drawing of BladeCenter H.
Figure 2-10 Rear view drawing of BladeCenter H
BladeCenter chassis T (8720)
This BladeCenter chassis is an 8U rack mountable chassis that meets the
special requirements of the telecommunications industry and is tested for
Network Equipment Building Standards (NEBS) compliance. With its low-profile
handles, the BladeCenter JS21 is intended to be the first blade to be supported
in all three chassis in the BladeCenter family. The BladeCenter T chassis
requires 1300 watt hot-swap redundant power supply modules.
2.9.1 BladeCenter power modules
There are multiple versions of power supplies available. The BladeCenter JS21
supports the power/thermal management architecture of BladeCenter, power
oversubscription is an option. There is no strict dependency of BladeCenter JS21
on the size or capacity of a particular power supply.
23
The standard redundant power supplies are installed in power bays one and two
of the BladeCenter. They provide power to the first six blade server bays. To
install blade servers in the remaining bays, seven through 14, you must install an
additional pair of redundant power supply modules in power bays three and four.
The standard redundant power supplies are installed in power bays one and
three of the BladeCenter H. They provide power to the first seven blade server
bays. To install blade servers in the remaining bays, eight through 14, you must
install an additional pair of redundant power supply modules in power bays two
and four.
2.9.2 BladeCenter management modules
The BladeCenter chassis contains two (redundant) management modules that
provide the manageability interface for the BladeCenter chassis. The
management module communicates with the BladeCenter JS21 within the
BladeCenter using an RS-485 SAS HDD inter-management network. The
BladeCenter JS21 supports both the standard IBM BladeCenter management
module and the advanced management module.
2.9.3 Media features
The BladeCenter chassis contains a media bay that contains one compact-disc
read-only memory (CD-ROM) drive or digital video-disc read-only memory
(DVD-ROM), a diskette drive, and a Universal Serial Bus (USB) port that you can
dynamically assign to any single BladeCenter JS21 in the BladeCenter chassis.
If your BladeCenter chassis was shipped before June 2003, an update to the
interface card on the media tray might be required for proper CD-ROM operation
with the BladeCenter JS21. To determine the part number of your existing media
tray, from the management module Web interface, under the heading Monitors
in the left column, select Hardware vital product data (VPD), and then look at
the module name media tray. If the field-replaceable units (FRU) number of the
media tray is 59P6629, call your hardware support center and request a free
replacement media tray.
Restriction: Diskette drives are not supported as boot devices on the
BladeCenter JS21.
24
3
Chapter 3.
BladeCenter JS21: Software
environment
This chapter discusses the software environments that are available for the IBM
BladeCenter JS21. It covers the operating systems and Virtual Input/Output
Server (VIOS) supported on the BladeCenter JS21, as well as the system
management tools.
25
3.1 Operating system support
The BladeCenter JS21 is supported by the AIX 5L, Red Hat Enterprise Linux,
and SUSE Linux Enterprise Server (SLES) operating systems. Specific version
details are listed in the following sections.
Both AIX 5L and Linux operating systems work with a 64-bit kernel and
64-bit/32-bit programs. Some of the included programs were compiled for 64-bit
when the code fully exploited the 64-bit address space.
3.1.1 AIX 5L
AIX has evolved from its beginnings on the IBM RT to become the operating
system of choice for the largest UNIX servers of IBM. AIX is an enterprise
operating system that scales from workstations all the way up to massively
parallel supercomputers.
Certified as C2 security-compliant by the U.S. government, AIX also supports
industry-standard security features such as Pluggable Authentication Modules
(PAM), authentication by x.509 certificates, OpenSSH, Kerberos, and
Lightweight Directory Access Protocol (LDAP). AIX fully exploits the features of
the JS21 with its 64-bit kernel and support for over 16 terabytes (TB) of disk
space.
With AIX 5L installed, the JS21 can run thousands of software titles that were
written for the AIX 5L platform, taking full advantage of the rich capabilities of
AIX 5L. At the time of writing of this book, AIX 5L 5.2M and AIX 5L 5.3E support
the BladeCenter JS21.
3.1.2 Red Hat Enterprise Linux
Red Hat has made its distribution of Linux available since 1994. Originally, Red
Hat offered its distribution for free download, and sold support contracts. In 2002,
Red Hat began marketing Red Hat Enterprise Linux. Unlike the original Red Hat
distribution, Red Hat Enterprise Linux was available only with a maintenance
contract from Red Hat. Red Hat Enterprise Linux runs on x86_64, i386™, ia64,
ppc/64, s390, and s390x platforms.
By early 2003, Red Hat stopped developing its non-enterprise distribution and
focused its efforts on Red Hat Enterprise Linux. Red Hat elected to turn the
development of its free software over to the community. This free
community-supported distribution became known as Fedora. Red Hat Enterprise
Linux AS 4 - Update 3 and later is supported on the JS21.
26
3.1.3 SUSE Linux Enterprise Server
SUSE has been a Linux distributor since 1992 and was acquired by Novell in
early 2004. Originally, SUSE provided a free downloadable distribution, for which
support contracts could be purchased as required. Eventually, their product line
was split. Novell now markets SUSE Linux Desktop, as well as SUSE Linux
Enterprise Server (SLES).
SUSE Linux Desktop is free to download, but SLES is available only with the
purchase of a maintenance contract. The desktop product is also available only
for x86_64 and IA32, but SLES supports IA32, ia64, ppc/64, s390, s390x, and
x86_64.
SLES Version 9 Update 2 and Update 3 are supported on the BladeCenter JS21.
SLES9, the latest version at the time of writing, ships with the 2.6 Linux kernel.
This brings significant gains to performance and scalability. SLES is AltiVec
savvy and has versions of GNU C Compiler (gcc) that allow manual exploitation
of the vector engine.
3.2 Virtual input/output Server
The Integrated Virtualization Manager (IVM) is a component included with the
VIOS Version 1.2, which is part of the Advanced Performance Optimization with
Enhanced RISC (POWER) Virtualization hardware feature. It enables companies
to consolidate multiple partitions onto a single server in a cost-effective way. With
its intuitive, browser-based interface, the IVM is easy to use. It significantly
reduces the time and effort required to manage virtual devices and partitions.
Restriction: The BladeCenter JS21 requires version 1.2.1 or later of the VIOS
for logical partitions (LPARs).
Integrated Virtualization Manager
To ease virtualization technology adoption in a server environment, IBM has
developed IVM, a simplified hardware management solution that inherits most of
the Hardware Management Console (HMC) features. It is limited to manage a
single server, avoiding the need for an independent personal computer. It is
designed to provide a solution that enables the administrator to reduce system
setup time and to make hardware management even easier, at a lower cost.
IVM provides a simple management model for a single system such as a JS21.
Although it does not provide all of the HMC capabilities, it enables the
exploitation of the IBM Virtualization Engine™ technology.
Chapter 3. BladeCenter JS21: Software environment
27
IVM is an enhancement of the VIOS, the product that enables I/O virtualization in
IBM POWER5™ processor-based systems. It enables management of VIOS
functions and uses a Web-based graphical interface that enables the
administrator to remotely manage the server with a browser. The Hypertext
Transfer Protocol Secure (HTTPS) protocol and server login with password
authentication provide the security required in an enterprise.
Because one of the goals of IVM is simplification of management, some implicit
rules apply to the configuration and setup:
When a system is designated to be managed by IVM, do not partition it.
The first operating system that you install must be the VIOS.
The VIOS is automatically configured to own all of the I/O resources and it can
be configured to provide service to other LPARs through its virtualization
capabilities. Therefore, all other LPARs cannot own any physical adapters. They
must access disk, network, and optical devices only through the VIOS as virtual
devices. Otherwise, the LPARs operate as they have previously with respect to
processor and memory resources.
Figure 3-1 shows a sample configuration using IVM. The VIOS owns all the
physical adapters, and the other two partitions are configured to use only virtual
devices. The administrator can use a browser to connect to IVM to set up the
system configuration.
LPAR #2
Virtual adapters
LPAR #1
Administrator’s
browser
VIOS + IVM
Physical adapters
Corporate LAN
Figure 3-1 Integrated Virtualization Manager configuration
28
A specific device called the Virtual Management Channel (VMC) has been
developed on the VIOS to enable a direct Hypervisor configuration without
requiring additional network connections. This device is activated, by default,
when the VIOS is installed as the first partition.
The VMC allows IVM to provide basic logical partitioning functions:
Logical partitioning configuration
Boot, start, and stop actions for individual partitions
Display of partition status
Management of virtual Ethernet
Management of virtual storage
Basic system management
Because IVM runs on an LPAR, it has limited service-based functions, and the
BladeCenter management module must be used. For example, a system power
on must be performed by physically pushing the system power on button or
remotely accessing the BladeCenter management module, because IVM does
not run while the power is off. The BladeCenter management module and IVM
together provide a simple but effective solution for a single partitioned server.
LPAR management using IVM is through a common Web interface developed for
basic administration tasks. Being integrated within the VIOS code, IVM also
handles all virtualization tasks that normally require VIOS commands to be run.
A major difference in IVM LPAR handling compared to HMC is dynamic
partitioning. The IVM does not use the RMC protocol that enables hardware
management to interact with the operating system’s resource allocation.
Although IVM is capable of performing a dynamic LPAR operation on itself (the
VIOS), changes on other LPARs are not applied until the LPAR is restarted.
Integrated Virtualization Manager design
All IBM System p5™ and IBM eServer™ p5 systems have the capability of being
partitioned because they are all pre-loaded with all the necessary firmware
support for a partitioned environment. On some systems,
IBM Micro-Partitioning™ is a priced option, but on high-end systems, such as the
IBM System p5 590 and p5-595, it is a standard element. It is also a standard
element on the IBM BladeCenter JS21.
29
Because the partitioning schema is designed by the client, every system is set up
by manufacturing in the same manufacturing default configuration that can be
changed or reset to when required. When configured using the manufacturing
default configuration, the system has the following setup from a partitioning point
of view:
There is a single predefined partition.
All hardware resources are assigned to the single partition.
The partition has system service authority, therefore, it can update the
firmware.
The partition is auto-started at power on.
Standard operating system installation methods apply for the partition
(network based or media based).
The system’s physical control panel is mapped to the partition, displaying its
operating system messages and error codes.
Base platform management functions, such as power control, are provided
through integrated system control functions (for example, BladeCenter
management module and control panel).
The manufacturing default configuration allows the system to be used
immediately as a stand-alone server with all resources allocated to a single
LPAR. If the VIOS is the first LPAR installed, the system configuration can be
changed to make the Hypervisor manage multiple LPARs.
Integrated Virtualization Manager architecture
The IVM has been developed to provide a simple environment where a single
control program has the ownership of the physical hardware and other logical
partitions use it to access resources. The VIOS has most of the required features
because it can provide virtual SCSI and virtual networking capability. Starting
with Version 1.2, the VIOS has been enhanced to provide management features
using the IVM.
To set up logical partitions, the IVM requires management access to the
Hypervisor. It has no service processor connection used by the HMC and it relies
on a new virtual I/O device type called the VMC. This device is only activated
when the VIOS installation detects that the environment has to be managed by
IVM.
30
On a VIOS partition with IVM activated, a new ibmvmc0 virtual device is present
and a management Web server is started listening to Hypertext Transfer
Protocol (HTTP) port 80 and to HTTPS port 443. The presence of the virtual
device can be detected using the lsdev -virtual command, as shown in
Example 3-1.
Example 3-1 Virtual Management Channel device
$ lsdev -virtual | grep ibmvmc0
ibmvmc0
Available Virtual Management Channel
Because IVM relies on VMC to set up logical partitioning, it can manage only the
system on which it is installed. For each IVM managed system, the administrator
must open an independent Web browser session.
Figure 3-2 provides the schema of the IVM architecture. The primary user
interface is a Web browser that connects to port 80 of the VIOS. The Web server
provides a simple graphical user interface (GUI) and runs commands using the
same command-line interface (CLI) that can be used to log on to the VIOS. One
set of commands provides LPAR management through the VMC, and a second
set controls VIOS virtualization capabilities.
Terminal emulator
Web browser
VIOS
Web server
LPAR 2
LPAR 1
Command line
CGI/HTML
IVM
LPAR CLI
Virtual I/O CLI
I/O subsystem
VMC
Hypervisor
Service processor
Figure 3-2 IVM high-level design
31
LPARs in an IVM managed system are isolated exactly as before and cannot
interact except using the virtual devices. Only the IVM has been enabled to
perform limited actions on the other LPARs such as:
Activate and deactivate
Send a power off (EPOW) signal to the operating system
Create and delete
View and change configuration
3.3 System management tools
IBM has devised a strategy for systems management called the Universal
Manageability Initiative. There are several technologies that have been
incorporated under this umbrella. They all work toward the same goal: simple,
effective solutions for managing heterogeneous environments.
One of the main building blocks in this strategy is the use of simple yet powerful
software to oversee the management of many different systems. This section
investigates some of the possible options that are available for managing such
systems. It reviews the BladeCenter Web interfaces, IBM Director, and Cluster
Systems Management (CSM).
3.3.1 BladeCenter Web interfaces
The BladeCenter Web interface allows system administrators to easily and
effectively manage up to 14 blades from an integrated interface. From trivial
tasks such as powering blades on or off, to more complex tasks such as firmware
management, the Web interface allows powerful control over all blades and I/O
modules that are attached to the BladeCenter chassis.
Management of other BladeCenter resources, such as I/O modules, is also
possible from here, as well as the retrieval of system health information.
BladeCenter-specific features are also configured from here such as the Serial
over LAN (SoL).
For more information, see IBM eServer BladeCenter Systems Management,
REDP-3582.
32
3.3.2 IBM Director
IBM Director assists with the remote management of many IBM and non-IBM
machines including the BladeCenter JS21. The IBM Director console allows
system administrators to manage multiple BladeCenter chassis from a common
interface. It is the ideal solution for administering BladeCenter chassis in
heterogeneous environments or environments where a Director infrastructure
exists.
From the IBM Director console, system administrators can monitor resource
utilization. You can use this key feature for performance and capacity planning,
and to alert support staff of critical errors such as hardware failure.
IBM Director also allows the remote management of software. You can remotely
reconcile and install software from the console interface. This can be useful in
large environments for resource-intensive tasks such as patch management. By
using the Software Distribution Premium Edition, you can remotely install
patches to several servers at the same time with the click of a button.
You can learn more about IBM Director in 10.2, “IBM Director” on page 346.
3.3.3 IBM Cluster Systems Management
IBM CSM provides many useful functions to manage a cluster from a single point
of control. These include resource monitoring, automated monitoring and
operation, remote hardware control, remote command execution, security,
configuration file management, parallel network installation, and diagnostics.
By consolidating these capabilities, CSM helps to increase efficiency of
administrators’ time and reduce their expenses. It helps administrators install
their clusters rapidly by automating many configuration tasks and by leveraging
existing open source products.
CSM also provides efficient monitoring of cluster resources without
overwhelming network bandwidth. The automated error detection that CSM
provides helps catch problems before they impact the environment, and assists
with rapid resolution and recovery of problems that occur. CSM’s architecture
and modular construction maximizes flexibility so that cluster solutions can
evolve and grow as requirements change.
CSM is a collection of components that have been integrated to provide the basis
to construct and manage a cluster. Each of these components provides specific
capabilities related to the management of the cluster. This component-based
architecture provides flexibility for future expansion of the capabilities provided
by CSM. Each of the CSM components can be easily personalized to help meet
specific needs. For example, a cluster administrator can set up monitoring of
33
application processes and take action if these processes end or stop abnormally.
In short, CSM has been designed for large-scale cluster environments that
require simple control over multiple homonymous servers.
We provide more details about CSM in 10.3, “Cluster Systems Management” on
page 362.
3.3.4 IBM middleware
The IBM middleware portfolio is enabled to run on the JS21 with either AIX 5L or
Linux as the operating system. For more information, refer to the following Web
site:
http://www.ibm.com/software/os/linux/software
34
4
Chapter 4.
BladeCenter JS21:
Virtualization
Logical partitions (LPARs) and virtualization increase the utilization of system
resources. This chapter provides details and configuration specifications about
this topic. The virtualization discussion includes virtualization enabling
technologies that are standard on the system, such as the Performance
Optimization with Enhanced RISC (POWER) Hypervisor, and optional ones,
such as the Virtual Input/Output Server (VIOS) feature.
The BladeCenter JS21 has native virtualization features that are enabled by
ordering and installing the no-charge VIOS. Without the VIOS installed, the
BladeCenter JS21 is a 2-core or 4-core symmetric multiprocessor (SMP) server.
35
4.1 Virtualization
With the introduction of more advanced POWER processors, partitioning
technology moved from a dedicated resource allocation model to a virtualized
shared resource model. This partitioning technology called Advanced POWER
Virtualization, the same virtualization capability supported on the System p5
servers, is built into the PowerPC 970MP processor, making the JS21 the first
IBM blade server with native virtualization. As an added incentive, a client can
obtain a license for VIOS to activate all the virtualization features of the JS21 at
no additional cost more than the cost of the blade (mandatory software
maintenance agreement not included). This section briefly discusses the key
components of virtualization on BladeCenter JS21 servers.
For more information about virtualization, see the following Web site:
http://www.ibm.com/servers/eserver/about/virtualization/
POWER Hypervisor
Combined with features designed into the PowerPC 970MP, the IBM POWER
Hypervisor™ delivers functions that enable other system technologies, including
Micro-Partitioning, virtualized processors Institute of Electrical and Electronics
Engineers (IEEE) virtual local area network (VLAN), compatible virtual switch,
virtual Small Computer System Interface (SCSI) adapters, and virtual consoles.
The POWER Hypervisor is a component of system firmware that is always
active, regardless of the system configuration. Therefore, it requires no separate
license apart from the VIOS for setup and usage.
The POWER Hypervisor provides the following functions:
Provides an abstraction layer between the physical hardware resources and
the logical partitions using them
Enforces partition integrity by providing a security layer between logical
partitions
Controls the dispatch of virtual processors to physical processors
Saves and restores all processor state information during logical processor
context switch
Controls hardware I/O interrupt management facilities for logical partitions
Provides VLAN channels between logical partitions, which removes the need
for physical Ethernet adapters for inter-partition communication
36
The POWER Hypervisor supports the following three types of virtual I/O
adapters.
Virtual Small Computer System Interface
The BladeCenter JS21 blade server uses SCSI as the mechanism for virtual
storage devices. This is accomplished using two paired adapters: a virtual SCSI
server adapter and a virtual SCSI client adapter. Virtual SCSI is available with the
Advanced POWER Virtualization and optional VIOS feature, which we describe
in 4.1.1, “Advanced POWER Virtualization” on page 38.
Virtual Ethernet
The POWER Hypervisor provides a virtual Ethernet switch function that allows
partitions on the same server as a means for fast and secure communication.
Virtual Ethernet working on LAN technology allows a transmission speed in the
range of 1 GBps to 3 GBps depending on the maximum transmission unit (MTU)
size. Virtual Ethernet requires a BladeCenter JS21 running either AIX 5L
Version 5.3 or the level of Linux supporting virtual Ethernet devices. Virtual
Ethernet is part of the base system configuration.
Virtual Ethernet has the following features:
A partition supports two virtual Ethernet connections. Each virtual Ethernet
connection can be connected to one of four VLANs. The VLAN provides
connectivity to other virtual Ethernet connections on client LPARs or the
VIOS.
Each partition operating system detects the VLAN switch as an Ethernet
adapter, without the physical link properties and asynchronous data transmit
operations. Layer-2 bridging to a physical Ethernet adapter is also included in
the virtual Ethernet features on the VIOS LPAR.
Note: Virtual Ethernet is based on the IEEE 802.1Q VLAN standard. No
physical I/O adapter is required when creating a VLAN connection between
partitions, and no access to an outside network is required. The Integrated
Virtualization Manager (IVM) simplifies the virtual Ethernet configuration by
controlling the VLAN numbers. Simplifying this for the user results in
restrictions where you do not have access to other IEEE 802.1Q capabilities
such as assigning one virtual Ethernet interface to multiple VLANs.
Chapter 4. BladeCenter JS21: Virtualization
37
Virtual teletypewriter console
Each partition requires access to a system console. Tasks such as operating
system installation, network setup, and some problem analysis activities require
a dedicated system console. The POWER Hypervisor provides the virtual
console using a virtual teletypewriter (TTY) or serial adapter and a set of
Hypervisor calls to operate on them. The operating system console is provided
by a console session with the IVM.
Note: The POWER Hypervisor is active when the server is running in partition
and non-partition mode. Consider the Hypervisor memory requirements when
planning the amount of system memory required. In AIX 5L V5.3, use the
lshwres command to view the memory usage.
lshwres -r mem --level sys -F sys_firmware_mem
You can also determine this using the console of the IVM: View/Modify
Partitions → System Overview → Reserved Firmware Memory.
4.1.1 Advanced POWER Virtualization
The VIOS Version 1.2.1 feature is an optional feature available at no additional
cost over the price of the base blade. Although included in the base price, the
client must still obtain a VIOS license for each JS21. This feature enables the
implementation of logical partitions on BladeCenter JS21 servers.
The virtual feature includes installation image for the VIOS software that
supports the following features:
Ethernet adapter sharing
Virtual SCSI Server
VIO software ships on a DVD
Software support of:
– AIX 5L V5.3
– SUSE Linux Enterprise Server 9 (SLES9) for POWER (Service Pack 3 or
later)
– Red Hat Enterprise Linux AS 4 for POWER
Partition management using IVM (VIOS V1.2.1 or later)
For details about Advanced POWER Virtualization and virtualization in general,
visit the following Web site:
http://www.ibm.com/servers/eserver/pseries/ondemand/ve/resources.html
38
Micro-Partitioning technology
The concept of Micro-Partitioning allows the resource definition of a partition to
allocate fractions of processors to the partition. Micro-Partitioning is only
available with POWER5 systems and the BladeCenter JS21. From an operating
system perspective, a virtual processor is indistinguishable from a physical
processor unless the operating system is enhanced to be aware of the
difference. The firmware on the BladeCenter JS21 (host firmware) virtualizes the
physical central processing units (CPUs). The host firmware layer presents
logical CPU numbers (0 - 3) to the operating systems.
A partition can be defined with a processor capacity as small as 10 processor
units. This represents one-tenth of a physical processor. The shared processor
partitions are dispatched and time-sliced on the physical processors under
control of the POWER Hypervisor. The shared processor partitions are created
and managed by the Integrated Virtualization Management (included with VIOS
Version software 1.2.1 or later). Dedicated and micro-partitioned processors can
coexist on the same BladeCenter JS21 blade server if they are available.
Table 4-1 lists processor partitioning information related to the BladeCenter
JS21.
Table 4-1 Processor partitioning overview of the BladeCenter JS21
Partitioning implementation
JS21
Cores (maximum configuration)
4
Dedicated processor partitions (maximum configuration)
4
Shared processor partitions (maximum configuration)
40
The maximum values stated in Table 4-1 are supported by the hardware.
However, the practical limits based on production workload demands and
application utilization might be significantly lower.
Virtual I/O Server
The VIOS is a special purpose partition that provides virtual I/O resources to
client partitions. The VIOS owns the real resources that are shared with the other
LPARs. The virtual I/O technology allows one or more partitions to share a
physical adapter assigned to a partition. This enables you to minimize the
number of physical adapters. The VIOS eliminates the requirement that every
partition must own a dedicated network adapter, disk adapter, and disk drive.
39
Figure 4-1 shows an organization view of Micro-Partitioning, including the VIOS.
It also includes virtual SCSI and Ethernet connections and mixed operating
system partitions.
Figure 4-1 Virtual partition: Organization view
Because the VIOS is an AIX 5L V5.3 operating system-based appliance, you can
provide redundancy for physical devices attached to the VIOS by using
capabilities such as Multipath I/O and IEEE 802.3ad Link Aggregation. Install the
VIOS partition from a special bootable DVD that is provided when you order the
VIOS. This dedicated software is only for the VIOS operations, therefore, the
VIOS software is only supported in VIOS partitions.
You can install the VIOS in the following ways:
Media (assigning the DVD-ROM drive to the partition and booting from the
media)
Using the Network Installation Manager (NIM)
40
Note: The VIOS supports logical mirroring and Redundant Array of
Independent Disks (RAID) configurations. Logical volumes created on RAID
or Just a Bunch of Disks (JBOD) configurations are bootable. Additionally, the
number of logical volumes is limited to the amount of storage available and the
architectural limits of the Logical Volume Manager (LVM).
Two major functions are provided with the VIOS: a shared Ethernet adapter and
Virtual SCSI. The following sections discuss these functions.
Shared Ethernet Adapter
A Shared Ethernet Adapter is a new service that acts as a layer 2 network switch
to route network traffic from a virtual Ethernet to a real network adapter. The
shared Ethernet adapter must be assigned to the VIOS partition.
Virtual Small Computer System Interface
Access to real storage devices is implemented through the Virtual SCSI services,
a part of the VIOS partition. This is accomplished using a pair of virtual adapters:
a virtual SCSI server adapter and a virtual SCSI client adapter. The virtual SCSI
server (target) adapter is responsible for issuing any SCSI commands it
receives. It is owned by the VIOS partition. The virtual SCSI client adapter allows
the client partition to access standard SCSI devices and logical unit numbers
(LUNs) that are assigned to the client partition.
All current storage device types, such as storage area network (SAN), SCSI, and
RAID are supported. iSCSI and Serial Storage Architecture (SSA) are not
supported. For more information about the specific storage devices that are
supported, visit the following Web site:
http://techsupport.services.ibm.com/server/vios/home.html
Integrated Virtualization Manager
To ease the virtualization technology adoption in the BladeCenter JS21
environment, IBM developed IVM, a simplified hardware management solution
that inherits most Hardware Management Console (HMC) features, but is limited
to managing a single server to avoid the need for a dedicated control
workstation. The goal is to provide a simple solution that enables the
administrator to reduce system setup time and to make hardware management
even simpler.
The IVM provides a simple management model for a single system such as a
BladeCenter JS21. Although it does not provide the full flexibility of an HMC, it
enables the exploitation of the IBM Virtualization Engine technology. The
BladeCenter JS21 is ideally suited for management using the IVM. The HMC,
41
which is designed to manage IBM System p servers with POWER5 processors,
does not support the BladeCenter JS21 with the PowerPC 970MP processor.
IVM is an enhancement of VIOS offered as part of VIOS V1.2.1, which is the
product that enables I/O virtualization in certain IBM Power Architecture™
systems. It provides the same VIOS features plus a Web-based graphical
interface that enables the administrator to remotely manage the BladeCenter
JS21 server with an Internet browser.
The IVM provides the following basic logical partitioning functions:
Logical partitioning configuration
Boot, start, and stop actions for individual partitions
Display of partition status
Management of virtual Ethernet
Management of virtual storage
Basic system management
The requirements for an IVM managed server are as follows:
Install the IVM (with VIOS) as the first operating system.
An IVM partition requires a minimum of one virtual processor and 512 MB of
random access memory (RAM).
The major differences in IVM in comparison to an HMC-managed system are as
follows:
All adapters are owned by IVM, and LPARs use virtual devices only.
No dynamic resource changes are allowed.
There are four virtual Ethernet networks available inside the system.
Each LPAR can have a maximum of one virtual SCSI adapter assigned.
It is not possible to have redundant VIOSs because the IVM manages all I/O.
Despite these limitations, IVM provides advanced virtualization functionality
without the need for an extra-cost workstation. For more information about IVM
functionality and best practices, see Virtual I/O Server Integrated Virtualization
Manager, REDP-4061.
42
4.1.2 Dynamic logical partitioning
The BladeCenter JS21 does not support dynamic logical partitioning (DLPAR) on
the client partitions. The BladeCenter JS21 uses the IVM of the VIOS to manage
the LPAR configuration. IVM does not support dynamic movement of system
resources across client partitions that are operational.
You can reconfigure resources on client LPARs without recycling the whole
server. You can use IVM to reconfigure resources across client LPARs if the
LPARs are stopped. This dynamic movement of resources does not affect other
LPARs that are running.
43
44
5
Chapter 5.
Planning considerations
This chapter provides the planning elements that are important for a successful
implementation of the BladeCenter JS21 in various environments. At the time of
publication, there are new technology levels for AIX Version 5.2 and Version 5.3
and new Linux distributions that support the JS21 BladeCenter. In addition, there
are new updates for the IBM Director and Cluster Systems Management (CSM)
that support the BladeCenter JS21. The details about versions and releases are
provided in this chapter and in the following chapters.
45
5.1 Documentation and planning resources
For the IBM Account team: If the IBM BladeCenter is new to your client, you
might want to review the IBM BladeCenter Solution Assurance Planning Review
Guide, Document SA749, before the ordering and delivery of the client’s system.
This document is not specific to the JS21 BladeCenter, but does provide
valuable guidance on what you have to consider for the BladeCenter chassis,
blade server installation, or both these installations, software installation,
networking, storage and site planning.
For clients: The BladeCenter JS21 ships with a documentation CD that provides
multiple installation, user and setup guides that are integral to your efficient
implementation of the BladeCenter. Current updates to these documents are
available online for your review and planning considerations. Visit the following
Web site:
http://www.ibm.com/pc/support/site.wss
From this site, select Servers as your product followed by BladeCenter or
BladeCenter H as the family.
5.2 Network planning
For a successful installation of the BladeCenter JS21, you must have a clear
plan on how you plan to use the various networking capabilities of the
BladeCenter infrastructure. This plan must address the following questions:
What network connectivity is required for the blade servers to support the
applications installed on them?
What network connectivity is necessary to manage the BladeCenter,
input/output (I/O) modules, and blade servers?
What virtual local area networks (VLANs) are required for each local area
network (LAN) switch I/O module to provide the network connectivity
established previously?
What Internet Protocol (IP) subnet is used for each VLAN and how are IP
addresses allocated to each device on the VLAN?
Are the IP addresses assigned statically or dynamically using Dynamic Host
Configuration Protocol (DHCP)?
What host names are used for each network interface?
46
How are host names resolved to IP addresses?
Are there requirements for a high-performance, low-latency interconnection
network?
Where multiple BladeCenter chassis are installed, how are they be
interconnected?
The following sections explore common network planning requirements that we
expect to arise in the installation of BladeCenter JS21s.
5.2.1 Minimal network requirements
At a minimum, most BladeCenter JS21 environments have the following network
requirements:
A dedicated hardware management subnet: It is used to access both the
management module and management interfaces of I/O modules that are
installed in each BladeCenter chassis.
A Serial over LAN (SoL) subnet internal to each BladeCenter chassis that
supports the SoL remote text console function: This is always implemented
using a LAN switch I/O module installed in I/O module bay 1.
A subnet connected to each BladeCenter JS21: It is used to install and
manage the operating system on the blade server. Where you run AIX 5L on
some blade servers, and Linux on other blade servers, you might have to use
multiple subnets for this purpose. It is possible to configure the installation
servers on the same subnet if you use directed Bootstrap Protocol (BOOTP)
with the Network Installation Manager (NIM) server.
One or more subnets connected to each BladeCenter JS21: They are used
by applications installed on the blade server to communicate with other
systems.
Chapter 5. Planning considerations
47
Figure 5-1 illustrates how these requirements can be provided in a logical
network view.
Hardware
Management
Subnet
I/O
I/O
Modules
I/O
Module
I/O
Module
Module
Management
Module
SOL
Subnet
Console
Console
Console
Console
Console
Console
JS20
Blade
Server
JS20
Blade
Server
JS20
Blade
Server
JS20
Blade
Server
JS20
Blade
Server
JS20
Blade
Server
eth0 eth1
eth0 eth1
eth0 eth1
eth0 eth1
eth0 eth1
eth0 eth1
Application
Subnet
OS
Management
Subnet
Figure 5-1 Network logical view
The following sections describe each of the logical networks illustrated in
Figure 5-1.
48
Hardware management subnet
We recommend that you establish a dedicated hardware management subnet.
The 10/100BaseT Ethernet interface on the management modules installed in
each BladeCenter chassis provides the gateway to this subnet for each
BladeCenter chassis. You have to install external LAN switches or hubs to
interconnect the 10/100BaseT Ethernet interface on the management modules
of each BladeCenter chassis with external management systems.
You can use this subnet to access the management module Web interface and
command-line interface (CLI). You can also use this subnet to access the Web
interface and CLI of I/O modules. System management applications such as the
IBM Director or CSM also use this subnet to communicate with the hardware
management functions of the BladeCenter infrastructure.
Restrict access to this subnet to those management systems, system
administrators, and operators who have responsibility for managing the
BladeCenter infrastructure.
You have to allocate multiple IP addresses to each BladeCenter chassis on the
hardware management subnet, including:
One IP address for the external interface of the management module in each
BladeCenter chassis
One IP address for the internal interface of the management module in each
BladeCenter chassis
One IP address for the management interface of each I/O module in each
BladeCenter chassis
Note: Although the logical network view (illustrated in Figure 5-1 on page 48)
shows the I/O management module interfaces connecting directly to the
hardware management subnet, they are physically connected using the
management module, which acts as gateway to these interfaces. The
management module performs a proxy Address Resolution Protocol (ARP)
function to make it appear as though the I/O module management interfaces
are attached to the hardware management subnet.
It is possible to use a different subnet for the management module internal
network interface and I/O module management interfaces. However, we do
not recommend this configuration.
Learn how to configure these IP addresses in 6.2, “BladeCenter internal network”
on page 66.
49
Serial over LAN subnet
The SoL remote text console function requires a subnet and underlying VLAN
that is implemented by a LAN switch I/O module installed in I/O module bay 1 of
the BladeCenter chassis. See Figure 2-10 on page 23. This subnet and VLAN
are entirely internal to each BladeCenter chassis and must not be externally
accessible.
If you use the 4-Port Gigabit Ethernet switch module or the Nortel Networks
Layer 2-7 Gigabit Ethernet switch module, the VLAN uses VLAN ID 4095. If you
use the Cisco Systems Intelligent Gigabit Ethernet switch module, you must
configure the VLAN ID to be 4095.
You have to assign a unique range of IP addresses to this subnet for use by the
SoL remote text console function.You have to specify only the starting IP
address within the range of IP addresses that you assign to the management
module. The management module then automatically assigns consecutive IP
addresses from the starting address that you provide to each blade server that
you have installed.
Important: One IP address is required for each blade.
You can learn how to configure the SoL subnet and VLAN in 6.7.1, “Configuring
Serial over LAN” on page 124
Operating system management subnet
We expect most environments that use the BladeCenter JS21 to rely on the
network installation procedure to install the operating systems. We discuss this
further in Chapter 7, “Installing and managing the Virtual I/O Server” on
page 193; Chapter 8, “Installing AIX” on page 259; and Chapter 9, “Installing
Linux” on page 281.
The operating system management subnet is used to support both the initial
installation and subsequent management of the operating systems installed on
BladeCenter JS21s. This subnet is implemented using a VLAN provided by the
Ethernet Switch I/O modules installed in I/O module bay 2 of each BladeCenter
chassis.
You might want to install both AIX and Linux operating systems on different
BladeCenter JS21s in the same environment. In this case, you might have to set
up multiple operating system management subnets and underlying VLANs, one
for blade servers running AIX and the other for blade servers running Linux if you
are performing network installations that use broadcast packets.
50
Application subnet
The primary reason you install a BladeCenter JS21 is to support applications.
Many applications have requirements to communicate with other systems. Use
one or more application subnets for this purpose.
The primary application subnet is implemented using a VLAN provided by the
LAN switch I/O modules installed in I/O module bay 2 of each BladeCenter
chassis. The LAN switch I/O module installed in I/O module bay 1 is used to
support the SoL VLAN.
Where different BladeCenter JS21s participate in different applications and there
is a requirement to separate application traffic, you might have to define multiple
application subnets and VLANs. Each BladeCenter JS21 is connected to the
appropriate application subnet based on the applications that are installed on the
blade server.
If application communication requirements with other systems are complex, you
can install an additional pair of Gigabit Ethernet interfaces on each BladeCenter
JS21. Do this using the Gigabit Ethernet expansion card in conjunction with
compatible I/O modules installed in I/O module bays 3 and 4.
5.2.2 High-performance, low-latency network requirements
Distributed memory parallel applications might require the installation of a
high-performance, low-latency interconnection network between BladeCenter
JS21s. This requirement can be supported through the use of an optional Myrinet
or Infiniband network.
To install a Myrinet network, you must install a Myrinet expansion card on each
BladeCenter JS21 that requires connectivity to the high-performance,
low-latency interconnection network. You must also install an optical pass-thru
I/O module in I/O module bay 4 of each BladeCenter chassis that contains blade
servers equipped with Myrinet expansion cards. Then connect the optical
pass-thru I/O module to external Myrinet switches to complete the Myrinet
network infrastructure.
51
Figure 5-2 illustrates this procedure.
Blade
Server
Blade
Server
Blade
Server
Myrinet
Expansion
Card
Myrinet
Expansion
Card
Myrinet
Expansion
Card
...
Blade
Server
Blade
Server
Blade
Server
Myrinet
Expansion
Card
Myrinet
Expansion
Card
Myrinet
Expansion
Card
Optical Pass-Thru Module
I/O Module Bay 4
Break-out
Cables
External
Myrinet
Switch
Figure 5-2 Myrinet network infrastructure
You can use the Myrinet network infrastructure to support application
programming interfaces such as the message passing interface (MPI) that are
often used by distributed memory parallel applications.
You can define IP addresses for each Myrinet network interface. You can also
use the Myrinet network infrastructure to support any application communication
based on IP protocols. For example, you can use this capability to support a
clustered file system such as IBM General Parallel File System (GPFS).
You can define a dedicated IP subnet for use with the Myrinet network
infrastructure that is distinct from the other IP subnets that are identified in 5.2.1,
“Minimal network requirements” on page 47.
52
5.2.3 Multiple BladeCenter environments
Large configurations of BladeCenter JS21s require the installation of multiple
BladeCenter chassis. To support the network requirements identified previously,
you have to install external LAN switches to interconnect the BladeCenter
chassis with each other.
5.3 Operating system support and installation
At the time of publication, the operating systems discussed in the following
sections support installation on the JS21.
AIX 5L
The following AIX 5L technology levels support installation on the JS21.
AIX 5L for Advanced Performance Optimization with Enhanced RISC V5.2
(POWER V5.2) with the 5200-08 technology level (APAR IY77270), plus
APAR IY80493 or a later technology level
AIX 5L for POWER V5.3 with the 5300-04 technology level (APAR IY77273),
plus APAR IY80499 or a later technology level
Advanced POWER Virtualization for AIX 5L V5.3 and Linux environments
require Virtual I/O Server (VIOS) V1.2.1 (5765-G34)
Linux
The following Linux distributions support installation on the JS21.
SUSE Linux Enterprise Server 9 for POWER Service Pack 3 (SP3), or later
Red Hat Enterprise Linux AS 4 for POWER update 3, or later
For AltiVec-optimized application development
Note: The BladeCenter JS21 is not shipped with an operating system
pre-installed. System administrators must acquaint themselves with the
installation instructions for their respective operating system.
There are several methods to install the JS21 in a BladeCenter. The first is an
attended method, using CD-ROM media. The second is a remote method
through a network installation. You can accomplish network installation using
NIM, Linux installation server, IBM Director, or CSM depending on the operating
systems involved. NIM and Linux installation servers are discussed in Chapter 8,
“Installing AIX” on page 259 and Chapter 9, “Installing Linux” on page 281.
53
Using the CD installation media is similar to other System p systems. You assign
the CD-ROM to a blade, follow the installation steps answering any prompts, and
then repeat the process on the next blade.
With network installation, you can perform several installations at the same time.
The method is designed to reduce the installation time required when a large
number of blades require operating system installation. We choose this method
as the focus for the following sections.
Network installation planning
You can use two approaches to set up a network installation environment for the
BladeCenter JS21:
Establish one or more network installation servers and manually initiate
network installation tasks.
Use a systems management tool, such as the IBM CSM or xCAT, which you
can use to automate much of the setup of network installation servers and
initiation of network installation tasks.
The remainder of this section focuses on the planning considerations for the first
approach. You can find planning considerations for the second approach in
Chapter 7 of Cluster Systems Management for AIX and Linux V1.5 Planning and
Installation Guide, SA23-1344. You can find this guide at:
http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?topic=
/com.ibm.cluster.csm.doc/clusterbooks.html
Required network services
Network installation depends on several different network services:
A BOOTP server or DHCP server
A Trivial File Transfer Protocol (TFTP) server
A Network File System (NFS) server
You can use one physical server to provide all three of these network services. If
you want to install AIX on BladeCenter JS21s, this server must run AIX. If you
want to install Linux on the BladeCenter JS21s, we recommend that you use the
same Linux distribution in the network installation server that you are planning to
install on the BladeCenter JS21s.
In some situations, you might want to install AIX on some BladeCenter JS21s
and Linux on other BladeCenter JS21s. Although it is possible to use a single
AIX server to do this, we recommend that you establish two network installation
servers, one for installing AIX and the other for installing your chosen Linux
distribution.
54
Important: A potential problem might arise when you have multiple servers
acting as BOOTP or DHCP servers. Be careful when planning your network
so that they do not interfere with one another.
One approach is to place the BladeCenter JS21s running AIX and the AIX
network installation server on a different VLAN from the BladeCenter JS21s
running Linux and the Linux installation server. If you use this approach, you
must also disable the relay of BOOTP or DHCP requests in your network
routers.
To learn how to set up AIX network installation servers, see 8.3.3, “Configuring
the NIM master” on page 263. For installation instructions for specific Linux
distributions, see 9.2.1, “Installing Linux using the network: General remarks” on
page 284. For instructions about setting up fundamental network installation
parameters for Linux, see 9.2, “Basic preparations for a Linux network
installation” on page 283.
Preparing for network installation
The BladeCenter JS21 firmware has the capability to boot an operating system
over a network using the BOOTP. Use this capability to initiate network
installation of the BladeCenter JS21.
The BOOTP protocol is a client-server protocol. When initiating a network
installation on a BladeCenter JS21, it behaves as a BOOTP client. Therefore,
you have to set up a BOOTP server to support initiating a network installation.
When you instruct the BladeCenter JS21 firmware to boot over a network using
BOOTP, it sends a request to the BOOTP server. The BOOTP server must
generate a response that contains the following information:
The IP address, subnet mask, and default gateway that the BladeCenter
JS21 must use for the network interface that sent the BOOTP request
The IP address of a TFTP server that the BladeCenter JS21 must contact,
and the name of the file on that server which the BladeCenter JS21 must
request, to get the operating system installation boot image
The BOOTP protocol has been around since the mid1980s. In many
environments, BOOTP has now been replaced by the newer DHCP protocol.
DHCP was designed to interoperate with BOOTP, and most DHCP servers can
serve BOOTP clients.
When you perform network installations of AIX, use the AIX BOOTP server.
When you perform network installations of Linux, use a DHCP server that is
configured to support BOOTP clients.
55
When the BladeCenter JS21 firmware has received a response from a BOOTP
server, it contacts the TFTP server specified in the BOOTP response to load the
operating system’s installation boot image. The operating system’s installation
boot image then starts running on the BladeCenter JS21. Eventually it contacts
the NFS server to obtain the files that it requires to perform the operating system
installation.
Selecting network interface
The BladeCenter JS21 has two standard Gigabit Ethernet interfaces. The first is
connected to I/O module bay 1, and the second is connected to I/O module
bay 2. See Figure 2-10 on page 23.
Restriction: The BladeCenter JS21 does not support the keyboard, video,
and mouse (KVM) console supported by other blade servers at this time.
Instead the BladeCenter JS21 supports SoL remote text consoles through the
BladeCenter management module.
The SoL remote text console function uses the first Gigabit Ethernet interface on
each BladeCenter JS21 for communication between the management module
and the service processor found in each BladeCenter JS21. This interface is
known as eth0 under Linux and en0 under AIX. Under normal conditions, using
the Gigabit Ethernet interface by the SoL remote text console function is entirely
transparent. It does not impact usage of the Ethernet interface by the operating
system running on the BladeCenter JS21.
Unfortunately, the initiation of network installation using the BOOTP protocol
from the BladeCenter JS21 firmware temporarily disrupts the operation of the
SoL remote text console when it occurs on the same physical Gigabit Ethernet
interface. This disruption makes it difficult to diagnose problems with the network
installation process. Therefore, we recommend that you use the second Gigabit
Ethernet interface on each BladeCenter JS21 during network installation of the
operating system. The second interface is known as eth1 under Linux and en1
under AIX.
To use the second interface, you must have a LAN switch I/O module, or
pass-thru I/O module connected to an external LAN switch, installed in I/O
module bay 2. You can learn more about why you must use the second Gigabit
Ethernet interface for network installation in the IBM RETAIN® tip H181655 on
the Web at:
http://www.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR-55282
56
5.4 Systems management
The BladeCenter JS21 supports a rich set of systems management capabilities.
This section discusses planning considerations associated with exploiting some
of these systems management capabilities.
5.4.1 BladeCenter Web interface
The BladeCenter Web interface allows the following functions:
System administrators can easily and effectively manage up to 14 blade
servers from an integrated interface.
You can power the BladeCenter JS21 on or off.
You can perform firmware management.
You can have powerful control over all blade servers and I/O modules that
are attached to the BladeCenter chassis even with a mixed environment.
System Management Services (SMS) menus are supported on the
BladeCenter JS21.
IBM only supports the Linux systems of clients with a SupportLine contract
covering Linux. Otherwise, contact the Linux distributor for support.
You can manage other BladeCenter resources such as I/O modules and
retrieval of system health information.
You can configure SoL for the JS21 blade server.
5.4.2 IBM Director
IBM Director is a systems management tool. It was originally developed to
provide a comprehensive management solution for servers and workstations
based on Intel microprocessors. Recently its capabilities have been expanded to
support management of other platforms such as the BladeCenter JS21.
IBM Director enables you to remotely manage many IBM and non-IBM servers
including the BladeCenter JS21. The IBM Director console allows system
administrators to manage multiple BladeCenter chassis in a heterogeneous
environment or environments where a Director infrastructure exists. IBM Director
V5.1 supports the following functions on the BladeCenter JS21:
Events
Resource monitoring
Inventory (limited)
Remote session
Software distribution
57
File transfer
Process management
Management Processor Assistant (MPA)
Visit the following Web site to download IBM Director 5.1:
http://www.ibm.com/servers/eserver/xseries/systems_management/ibm_direc
tor/
This section reviews the major planning considerations associated with the
installation of IBM Director. For a comprehensive treatment of planning for IBM
Director, refer to Implementing IBM Director 5.10, SG24-6188.
IBM Director has several components. We address three of the primary
components in this section and in 10.2, “IBM Director” on page 346.
IBM Director Server: This is the core of the IBM Director system
management solution.
IBM Director Console: This provides the user interface that system
administrators and operators use to interact with IBM Director.
IBM Director Agents: These provide the mechanism by which the IBM
Director can monitor and control a specific server and operating system
environment.
IBM Director Server for the BladeCenter JS21
Support for IBM Director Server on the BladeCenter JS21 was announced in IBM
Director Version 5.10. For installation instructions for the IBM Director Server,
see 10.2.1, “Installing the components for IBM Director” on page 347.
IBM Director Agent for the BladeCenter JS21
Support for the BladeCenter JS21 was introduced in IBM Director Agent
Version 4.12 and is enhanced in Version 5.10. You can learn how to install the
IBM Director Agent in 10.2.1, “Installing the components for IBM Director” on
page 347
For maximum functionality, the IBM Director Server requires IP connectivity to
the management module in each BladeCenter chassis and to every blade server.
This connectivity can be provided by connecting the IBM Director Server to both
the hardware management subnet and operating system management subnets
described in 6.3.6, “Management module: TCP/IP ports” on page 83.
If your environment primarily comprises servers running Linux, consider
installation of the IBM Director Server under Linux.
58
5.4.3 IBM Cluster Systems Management
IBM CSM is a systems management tool. It simplifies the management of
clusters of servers running the AIX and Linux operating systems. This section
focuses on the planning considerations that are associated with using CSM to
manage BladeCenter JS21s.
For more planning information, refer to Cluster Systems Management for AIX
and Linux V1.5 Planning and Installation Guide, SA23-1344.
Supported CSM and operating system releases
Preliminary support for the BladeCenter JS21 was introduced in CSM for
AIX V1.5.1.1, and CSM for Linux V1.5.1.1. There is also a Red Hat Package
Manager (RPM) for CSM for Linux update V1.5.1.1 and a program temporary fix
(PTF) for CSM for AIX V1.5.1.1.
CSM requires specific support for each operating system version that is running
on the servers that it manages. Currently (at the time of publication) CSM
Version 1.5.1 supports the following operating systems on the BladeCenter
JS21:
AIX 5L Version 5.2 with APAR IY77440
AIX 5L Version 5.3 with APAR IY7740
The default base versions of SUSE Linux Enterprise Server 9 (SLES9) for
POWER systems
The default base versions of Red Hat Enterprise Linux (AS) 4 for POWER
Certain CSM for Linux on POWER functions require non-IBM software. The
following non-IBM software is required. You can obtain it from the listed sources.
AutoUpdate V4.3.4,or later levels
You must use this if you want to perform the software maintenance
installation and upgrade of non-CSM RPMs on your Linux-managed nodes
from the management server. Download the software from:
http://freshmeat.net/projects/autoupdate
sg3_utils-1.06-1.ppc64
http://people.redhat.com/pknirsch
59
For your convenience, the following required IBM and non-IBM software is
included on the CSM CD-ROM:
tftp-HPA 0.34
fping-2.4b2-5
conserver 8.1
IBMJava2-JRE 1.4.2
Planning elements for CSM for AIX and Linux is covered in detail in Chapter 3 of
IBM Cluster Systems Management for AIX V5L and Linux Planning and
Installation Guide V1.50, SA23-1344. To check for updates to this document,
refer to:
Important: Follow the instructions regarding software installation and
prerequisites closely. This is especially important if you are upgrading from
CSM Version 1.5.1.0 using PTFs.
Management node selection
Each CSM cluster requires a management node, which provides the central
point of administration and control for the entire cluster. CSM has complex rules
concerning the compatibility of management nodes, non-management nodes
(cluster nodes), and the operating system they run. We attempt to simplify these
rules by focusing exclusively on the requirements for a management node that
only manages BladeCenter JS21s. If your environment is more complex, refer to
the CSM planning documents.
You can use a CSM management node to both install and manage BladeCenter
JS21s, or you can use a management node to manage BladeCenter JS21s that
are installed using other mechanisms. Most environments use CSM to both
install and manage BladeCenter JS21s; therefore, we focus on this scenario.
If you use a management node to both install and manage BladeCenter JS21s
running Linux, you must use the same type of Linux distribution on both the
management node and the BladeCenter JS21s. However, the management
node does not require the same processor architecture that is used in the
BladeCenter JS21. For example, if you plan to install SLES on BladeCenter
JS21s, you can use the following types of management node and operating
system combinations to both install and manage the BladeCenter JS21s:
60
A supported IBM eServer xSeries® server running SLES
A supported IBM eServer pSeries® server running SLES
An IBM HS20 Blade Server running SLES
An IBM BladeCenter JS21 running SLES
We performed our testing of CSM using a BladeCenter JS20 as the management
server, another BladeCenter JS20 node with AIX, and an AIX LPAR on a JS21
and a Linux LPAR on a JS21 as two managed cluster nodes.
Network considerations and summary
IBM CSM is designed to work in the type of network environment that is outlined
in 5.2, “Network planning” on page 46, with multiple subnets that segregate
different types of network traffic. The CSM management node is normally
equipped with multiple Ethernet network interfaces that are connected as follows:
One interface connects to the hardware management subnet to support
CSM’s hardware management functions.
One interface connects to the operating system management subnet to
support CSM’s capabilities to install operating systems, maintain software
and configuration files, and provide ongoing monitoring of cluster nodes.
One interface connects to an external network to enable remote access to the
management node by system administrators and operators.
61
62
6
Chapter 6.
Hardware setup
This chapter explains the various activities that you have to perform before you
can install an operating system on the BladeCenter JS21. You have to
understand the components, their relationships, and how to configure them. This
chapter covers the following topics:
6.1, “BladeCenter chassis” on page 64
6.2, “BladeCenter internal network” on page 66
6.3, “Management module configuration” on page 71
6.4, “I/O module configuration” on page 85
6.5, “Blade server configuration” on page 90
6.6, “Firmware” on page 94
6.7, “Providing a console for the BladeCenter JS21” on page 121
6.8, “System Management Services interface” on page 133
6.9, “Open Firmware interface” on page 152
6.10, “Blade server Ethernet controller enumeration” on page 162
6.11, “Blade server I/O module routing” on page 169
6.12, “SAS hardware RAID configuration” on page 170
63
6.1 BladeCenter chassis
Each BladeCenter chassis can accept the same blades, but uses different
switches and power modules. Figure 2-7 on page 20 shows the rear view of
BladeCenter, Figure 2-10 on page 23 shows the rear view of BladeCenter H, and
Figure 6-2 on page 65 shows the rear view of the BladeCenter T as an
orientation for locating the management and input/output (I/O) modules. For
more information, see Installation and User's Guide - IBM BladeCenter (Type
8677), Installation and User's Guide - IBM BladeCenter H (Type 8852), or
Planning and Installation Guide - IBM BladeCenter (Type 8677).
BladeCenter product documentation
You can find these manuals on the following IBM Web site:
http://www-03.ibm.com/servers/eserver/support/bladecenter/allproducts/i
nstalling.html
For each product, select Hardware only → Product documentation.
Figure 6-1 shows the front view of the BladeCenter T.
Figure 6-1 Front view of the BladeCenter T
64
Figure 6-2 shows the rear view drawing of the BladeCenter T.
Figure 6-2 Rear view of the BladeCenter T
All BladeCenter chassis run with one or two management modules (MM) or
advanced management modules (AMM). BladeCenter-H supports the AMM only.
The first management module must be located in management module bay 1
and the second in management module bay 2. The Ethernet switch modules
must be located in I/O module bays 1 and 2. Additional modules can be added in
the remaining I/O module bays depending on the type of the chassis. Because
the interfaces are always paired and the pair associated to I/O module bays 1
and 2 is always Ethernet, there must be Ethernet switches in these bays.
However, the type of Ethernet switches might be different in bay 1 compared to
bay 2.
Chapter 6. Hardware setup
65
6.2 BladeCenter internal network
This section explains the BladeCenter’s internal network. Understanding the
internal network paths is essential for a correct setup and integration of a
BladeCenter into a network environment. It is also important for planning and
implementing the management network besides the production network end
optional using virtual LANs (VLANs). Figure 6-3 on page 67 shows a schematics
abstract of the BladeCenter’s internal network.
Note: This schematic does not exactly match the real electric design. It is a
simplified abstract to explain network paths for planning and configuring the
BladeCenter’s network setup. You can find a more detailed schematic in
BladeCenter chassis management:
http://www.research.ibm.com/journal/rd/496/brey.pdf
66
BladeCenter
Chassis
Management
Workstation
Management Module (MM)
(redundant MM not shown)
Proxy
ARP
INT (eth1)
EXT (eth0)
Management Interface
HTTP/HTTPS/Telnet/
SSH/DOT/SNMP/LDAP
MANAGEMENT MODULE BAY 1
Ethernet Switch Module
Management
Network
Management
Interface
MGT 1
EXT 1
MGT 2
EXT 2
INT 1
EXT 3
..
INT 14
EXT 4
I/O MODULE BAY 1
Management
Interface
MGT 1
EXT 1
MGT 2
EXT 2
INT 1
EXT 3
..
INT 14
EXT 4
I/O MODULE BAY 2
SoL Data
BSMP
Build in Ethernet
Controller
Port 2 (eth1)
Port 1 (eth0)
Port 2 (eth1)
Port 1 (eth0)
Production
Network
Network (LAN)
SoL Data
...
BSMP
Build in Ethernet
Controller
Optional Network
connection
Production VLANs
SoL VLAN
default ID = 4095
Management VLAN
(untagged)
BladeCenter JS21
BAY 1
...
BladeCenter JS21
BAY 14
External management
over all ports
(enable/disable)
The AMM has no internal interface eth1
Figure 6-3 BladeCenter internal network
67
Within the BladeCenter there are two types of physical networks: The internal
management network and the internal production network. The internal
management network connects the management module to the management
port of the I/O modules (for simplification, we do not show the optional redundant
management module in Figure 6-3 on page 67). The internal production network
is a separate (individual) connection from each blade to the I/O module bay. Each
blade, by default, has two Ethernet ports. Port 1 (eth0) is connected to I/O
module bay 1 and port 2 (eth1) to I/O module bay 2. If an expansion card
(daughter card) is installed in a blade, the two ports of the expansion card are
connected to I/O module bays 3 and 4.
Note: BladeCenter HS20-8678 blade servers map the switch modules in the
opposite way: The switch module in module bay 1 accesses the blade server’s
eth1 interface, and the switch module in module bay 2 accesses the blade
server's eth0 interface.
To use HS20-8678 blade servers in the same chassis with other HS20 blade
servers, you must upgrade the HS20-8678 blade server Basic I/O system
(BIOS) to level 1.05 or later. Then modify the BIOS settings for each
HS20-8678 blade server. When you turn on the server, press F1 to enter
BIOS setup. In the BIOS setup screen, select BIOS setup → Advanced
Setup → Core Chipset Control → Swap the numbering of onboard NICs
[Yes].
In Figure 6-3 on page 67, we show two Ethernet switch modules (ESM) in I/O
module bays 1 and 2. The ESMs have two internal management ports going to
MM1 and MM2 (MM2 is optional and not shown), 14 internal ports going to the
blades and four to six external (uplink) ports to connect to the external
production network. To connect the management module’s external interface
(eth0) to the external management network, you must use the management
module’s remote management and console RJ45 Ethernet connector.
Note: The management module provides the only access to the internal
management network. This means that you cannot access the internal
management network through the Ethernet switch modules from the
production network, from outside the BladeCenter (EXTx ports), or from a
blade server itself (INTx ports).
The management module performs a proxy Address Resolution Protocol (ARP)
function for the Internet Protocol (IP) traffic on the internal management network
and then passes it along to its external (eth0) interface. Therefore, you can
directly reach the Ethernet switch module’s management interface through the
management module from the external management network. The management
68
module only responds to ARP requests that are either addressed to the IP
address of the ESM, or originate from the IP address of the ESM.
Note: Due to the proxy ARP function of the management module, you see all
of the IP addresses of the internal management network, that is, the
management module’s internal port eth1 (the advanced management module
has no internal port eth1) and the management ports of the ESMs (if accessed
through the management module and not using the external uplink ports)
associated with the hardware Medium Access Control (MAC) address of the
management module’s eth0 interface in the outside network.
In the I/O module bays there are connections to both networks, the internal
management network and the internal production network.
Note: Although the internal management network and the internal production
network are connected through the ESM, there are methods implemented to
separate the management and production traffic. BladeCenter ESMs have
certain hard-coded filters that prevent any traffic that enters any of the
upstream ports from exiting out of the management module facing ports and
vice versa. This also prevents any unexpected spanning-tree loops.
The only way to control the management module is through the external RJ45
Ethernet connector of the management module. It is also not possible to pass
substantive data between switch modules across the midplane using the
MGTx ports. The ESM does not forward data between the MGTx ports and
any of the internal (INTx) or external (EXTx) uplink ports. If you want to pass
data from one switch module to another, then the modules must either be
cabled directly to each other or connected by way of an external switch or
router.
This is especially important if you have enabled “External management over all
ports” (see 6.4.2, “Setting external management over all ports” on page 87) for
the ESM (represented by a closed “External management over all ports” switch
in Figure 6-3 on page 67). In this case, you can reach the ESM’s management
interface (HTTP, Telnet, Secure Shell (SSH), Simple Network Management
Protocol (SNMP), and so on) through two paths. You can reach it from the
external management network through the management module that supports
pass-through function to the internal management port of the ESM on the one
side, and from the external production network through the ESM’s external ports
on the other side. Even in this scenario the ESM’s management interface does
not pass through data from one path to the other.
69
Note: The BladeCenter Ethernet switch modules are designed for secure
management. Accessing the management interfaces of the Ethernet switch
modules using the management network is enabled by the management
module interface performing a proxy ARP. This enables the secure
management subnet to access the management interfaces of the I/O modules
through a single physical Ethernet port on the management module.
If you do not have a separate subnet for management available to you, then
you can also attach the ESM external ports to this subnet. To keep duplicate
IP addresses from being reported, the default setting is off for “External
management over all ports” (see 6.4.2, “Setting external management over all
ports” on page 87). If you want to turn on the “External management over all
ports” setting, then keep the management module Ethernet interface in a
separate network because this is the proper configuration.
You must carefully consider when you connect the external management network
and the external production network (see the dotted Ethernet link in Figure 6-3
on page 67) or leave them separated. When you connect both, use different
VLANs to separate the networks.
Another method of separating the network paths is used for Serial over LAN
(SoL). SoL traffic goes from the management module’s management interface
through the ESM in I/O module bay 1 to the blade servers’s blade system
management processor (BSMP).
Note: The BSMP is also called integrated systems management processor
(ISMP) or baseboard management controller (BMC).
To do the separation, SoL uses the VLAN ID (VID) 4095. The default VID for the
production network is 1 (you can define additional VLANs with an ID less than (<)
4095 in the ESM for the production network, and the VID 4095 is blocked for this
purpose). Therefore, the SoL traffic does not interact with the internal production
network. However, it partly uses the same physical network path.
For more information, see the following IBM Redpapers:
Nortel Networks L2/3 Ethernet Switch Module for IBM eServer BladeCenter,
REDP-3586
Cisco Systems Intelligent Gigabit Ethernet Switch Module for the
IBM eServer BladeCenter, REDP-3869
IBM eServer BladeCenter Layer 2-7 Network Switching, REDP-3755
70
For more information, see the following documents:
BladeCenter chassis management
BladeCenter networking
http://www.research.ibm.com/journal/rd/496/hunter.pdf
BladeCenter processor blades, I/O expansion adapters, and units
http://www.research.ibm.com/journal/rd/496/hughes.pdf
6.3 Management module configuration
This section describes the steps to set up the BladeCenter management module
so that you can work with BladeCenter JS21s and other blade servers. You can
find further information about the configuration of the management module in
Management Module Installation Guide - IBM BladeCenter and Advanced
Management Module Installation Guide - IBM BladeCenter, BladeCenter H.
Refer to the procedure described in “BladeCenter product documentation” on
page 64.
Note: You can access the management module’s management interface
(HTTP, Telnet, SSH, and so on) only by using TCP/IP over the module’s
remote management and console RJ45 Ethernet connector or the serial port of
the advanced management module. You cannot use the keyboard, video,
mouse (KVM) connectors for this purpose.
The primary setup task is to assign IP addresses, which are necessary to
communicate with the management module using TCP/IP. To learn more about
selecting these IP addresses, see Chapter 5, “Planning considerations” on
page 45. The management module has two Ethernet interfaces, an external
interface (eth0) and an internal interface (eth1) as seen in Figure 6-3 on page 67.
71
6.3.1 Connecting to the management module
All management module types support connection through the remote
management and console RJ45 Ethernet connector. Figure 6-4 shows the
connectors of the management module.
Note: The advanced management module also supports command-line
Interface only (CLI-only) connection through the external serial port of the
advanced management module.
For information about accessing the management module’s CLI, see the
Management Module Command Line Interface Reference Guide - IBM
BladeCenter and BladeCenter T. Refer to the procedure described in
“BladeCenter product documentation” on page 64.
Figure 6-4 Management module: Output connectors
72
Figure 6-5 shows the connectors of the advanced management module.
Figure 6-5 Advanced management module: Output connectors
You can access the management interface through a network or through a
computer that is connected directly to the management module. To establish a
remote console to the management module, you require the following equipment
and information:
A computer with Internet browser capability. To facilitate connections at
multiple locations, you can use a notebook computer.
The management module’s MAC address (listed on the label on the
management module).
73
For a network connection to the management module, the following
equipment:
– A standard Ethernet cable
– A local Ethernet network port (facility connection)
An Ethernet crossover cable for direct connection of a computer to the
management module’s remote management and console Ethernet connector
Note: The advanced management module can use either a standard Ethernet
cable or an Ethernet crossover cable to make this connection.
The external interface (eth0) is accessible using the 10/100BaseT RJ45
connector on the management module. The internal interface (eth1) is connected
to the management interfaces of all the installed I/O modules that support such
interfaces (Figure 6-3 on page 67). This includes all switch I/O modules.
Tip: Note that:
The function of a new management module is to request an IP address for
the external interface using Dynamic Host Configuration Protocol (DHCP).
If the management module does not receive a valid response from a
DHCP server within two minutes of powering on, it uses the static IP
address 192.168.70.125 with the subnet mask 255.255.255.0.
The default host name is MMxxxxxxxxxxxx, where xxxxxxxxxxxx is the
burned-in MAC address.
The default IP addresses with a subnet mask of 255.255.255.0 are:
Management module:
– External port: 192.168.70.125
– Internal port: 192.168.70.126
Devices in I/O module bays:
–
–
–
–
Module bay 1: 192.168.70.127
Module bay 2: 192.168.70.128
Module bay 3: 192.168.70.129
Module bay 4: 192.168.70.130
You can reset the IP addresses of a management module that was previously
configured back to the factory defaults by using the IP reset button (shown in
Figure 6-4 on page 72) on the management module. You can find the procedure
for doing this in 6.3.7, “Resetting the management module” on page 84.
74
6.3.2 Configuring the management module
Perform the following procedure to set the IP addresses for the management
module’s external and internal interfaces when the management module is not
connected to a DHCP network.
1. Connect the management module to an isolated private Ethernet network.
Also connect a workstation that has a Web browser to the same isolated
private Ethernet network. You can also connect the workstation directly to the
management module using an appropriate cable (crossover for any
management module that is not an advanced management module).
2. Configure a static IP address for the workstation Ethernet interface that is in
the same subnet as the management module default IP addresses. For
example 192.168.70.100 with a subnet mask of 255.255.255.0 for the
workstation Ethernet interface. Do not use addresses in the range of
192.168.70.125 through 192.168.70.130. These addresses conflict with the
default addresses assigned by the management module.
3. Connect to the management module Web interface by pointing your Web
browser on the workstation to:
http://192.168.70.125
Note: Alternatively, you can connect the management module to a normal
network that has a DHCP server running. Either define the MAC address of
the management module in the DHCP server before connecting. Or check
which address has been given to the management module after
connecting, and use this address instead of 192.168.70.125. This keeps
you from having to change your computer’s IP address. This method is
only possible if DHCP is not disabled on the management module.
4. Enter a valid user ID and password. The factory default configuration of a
management module defines a user ID named USERID with a password of
PASSW0RD. The “0” in the password is a zero. In production environments,
consider changing these defaults.
5. From the management module Web interface, select MM Control →
Network Interfaces, as shown in Figure 6-6. Enter the external and internal
IP addresses that you want (standard management module only), subnet
masks, and default gateway for the management module. We also
recommend that you switch off DHCP by setting it to Disabled - Use static IP
configuration. Click Save to store the new IP addresses.
75
Figure 6-6 Management module: IP address configuration
76
Figure 6-7 shows the interface for the advanced management module.
Note: The management of the I/O modules by the management module is
not based on the IP interfaces and the internal management network at all.
It uses a different hardware link (I2C). Therefore, you can configure the I/O
modules by the management module even when the IP addresses are not
set up correctly.
Figure 6-7 Advanced management module: IP address configuration
Attention: The advanced management module has no internal Ethernet
interface eth1. For the management module, the IP address for eth1 must
be on the same subnet as eth0 on the management module as well as the
management interfaces of the I/O modules. Otherwise, the I/O module’s
management interfaces cannot be reached through the management
module. The data rate, duplex mode, and maximum transmission unit are
read-only fields for eth1.
77
6. Restart the management module. In the management module Web interface,
select MM Control → Restart MM. Click the Restart button on the displayed
page. You are prompted to confirm the restart before it occurs.
7. Remove the management module from the isolated private Ethernet network
and connect it to the network that you will use to manage the BladeCenter.
You can now connect to the management module Web and command line
interfaces using the IP address that you assigned to the management module
external network interface.
Now consider performing other management module setup tasks, such as:
Changing the factory set default user ID and password for security reasons
Setting the management module date and time so that log entries have useful
time stamps
Defining user IDs and passwords for system administrators and operators
who manage the BladeCenter and disable the factory set default user ID
Alternatively, you can configure the management module to use a Lightweight
Directory Access Protocol (LDAP) directory for this purpose.
Configuring the management module to send alerts to management systems
using SNMP, or system administrators using e-mail using Simple Mail
Transfer Protocol (SMTP)
Enabling the use of Secure Sockets Layer (SSL) to securely access the
management module Web interface
Enabling the use of SSH to securely access the management module CLI
Note: The management module’s remote disk feature is currently not
available for blade servers JS20 and JS21.
For additional information about how to perform these tasks, refer to the
Advanced Management Module and Management Module User's Guide - IBM
BladeCenter, BladeCenter T, BladeCenter H and the Management Module
Command Line Interface Reference Guide - IBM BladeCenter and BladeCenter
T. You can find these publications using the procedure described in “BladeCenter
product documentation” on page 64.
78
6.3.3 Advanced management module: Management interface
changes
The Web interface of the advanced management module is mostly identical with
the Web interface of the standard management module. The new items are
Serial Port, Configuration Mgmt (which was Restore Defaults), and the Service
Tools menu, as shown in Figure 6-8.
Figure 6-8 Advanced management module: Web interface system status
79
The configuration management menu has new functions to back up and restore
configuration, as shown in Figure 6-9.
Figure 6-9 Advanced management module: Configuration management
80
6.3.4 Management module: Hardware vital product data
To get an overview of the hardware that is installed in the BladeCenter, select
Monitors → Hardware VPD. Figure 6-10 shows an example of the displayed
data.
Tip: You can see the BladeCenter JS21 MAC addresses in the hardware vital
product data (VPD).
Figure 6-10 Management module: Hardware VPD
81
6.3.5 Management module redundancy
Each BladeCenter unit comes with at least one management module. Some
BladeCenter units support installation of a second, redundant management
module. Only one of the management modules in a BladeCenter unit can be
active, and this functions as the primary management module. If a redundant
management module is installed, it remains inactive until it is switched to act as
primary, either manually or automatically, if the primary management module
fails.
You can configure the failover behavior using the uplink command described in
the Management Module Command Line Interface Reference Guide - IBM
BladeCenter and BladeCenter T. To locate this document, use the procedure
described in “BladeCenter product documentation” on page 64.
Note: During failover, the ARP table is flushed and automatically rebuilt over a
time period of some minutes. When it is rebuilt, you will be able to re-establish
a network connection to the management module. This ARP table rebuilding
does not impact the network attached to the Ethernet switch modules.
To check which management module is currently active, in the management
module’s Web interface, select Monitors → System Status → Management
Module, as shown in Figure 6-11.
Figure 6-11 Displaying the active and backup management module
If two management modules are installed in a BladeCenter unit, both must
always have the same level of firmware and the same IP address. The firmware
must support redundant management module function to enable changeover of
control from the primary (active) management module to the redundant
management module. The configuration settings of the primary management
module are automatically transferred to the second management module. This
transfer can take up to 45 minutes.
82
Note: Configuration data and the firmware itself are automatically transferred
from the primary management module to the redundant management module.
Therefore, you have to apply firmware updates only on the primary (active)
management module.
Sometimes the firmware transfer does not work on the back-level firmware. (See
6.6, “Firmware” on page 94 for more information.) In this case, you have to
update both the management modules separately. To do this, you must activate
the backup management module by deactivating the primary management
module.
Note: To switch the active management module, from the Web interface,
select MM control → Restart MM → Switch Over.
6.3.6 Management module: TCP/IP ports
The BladeCenter management module uses a variety of TCP/IP ports for
communication. In this section, we show which ones the administrators can
change and which ones they cannot change. You have to ensure that your
network (firewalls and routers) allows communications through these ports for
the adapter and management module to function properly.
The ports in Table 6-1 are user configurable. The default port numbers used are
indicated in column 2.
Table 6-1 User configurable TCP/IP ports
Port name
Default port number
Description
HTTP
80
Web server HTTP connection - TCP
HTTPS
443
SSL connection - TCP
Telnet
23
Telnet command-line interface connection TCP
SSH
22
SSH command-line interface - TCP
SNMP agent
161
SNMP get/set commands - User Datagram
Protocol (UDP)
SNMP traps
162
SNMP traps - UDP
83
The TCP/IP ports in Table 6-2 are fixed and cannot be changed.
Table 6-2 Fixed TCP/IP ports
Port number
Description
25
E-mail alerts - TCP
53
Domain Name System (DNS) resolver - UDP
68
DHCP client connection - UD
427
Service Location Protocol (SLP) connection - UDP
1044
Remote disk function - TCP
1045
Persistent remote disk (disk on card) - TCP
5900
Remote console video redirect - TCP
6090
IBM Director commands - TCP
13991
IBM Director alerts - UDP
6.3.7 Resetting the management module
This section shows how to reset the BladeCenter management module in a
scenario where the passwords are lost or the IP addresses are unknown. The
management module reset button is shown in Figure 6-4 on page 72. Use a
straightened paper clip or a similar item to access it.
The reset button has two uses:
Resetting the network configuration. To do this, press and hold the reset
button for 3 seconds or less.
Resetting the entire management module (including user IDs and passwords)
to the factory defaults. This requires a sequence of keeping the button
pressed and releasing it:
a. Press and hold the reset button for 5 seconds.
b. Release the button and wait for 5 seconds.
c. Press and hold the button for another 10 seconds.
Note: This sequence must be as precise as possible to ensure success:
Keep the button pressed for 5 seconds, release it and wait for 5 seconds,
and again keep it pressed for 10 seconds. After you do a full factory
defaults reset in this way, the default user ID and password will be in effect:
USERID (all caps) and PASSW0RD (with a zero, not the letter O).
84
After you reset the network configuration, you can access the Web interface to
reconfigure it. To do this, you have to know the management module's IP
address as described in 6.3.2, “Configuring the management module” on
page 75.
Attention: Because it is possible to reset the passwords using the reset
button, you must ensure that your BladeCenter chassis is physically secure so
that only authorized personnel can physically access the reset button.
6.4 I/O module configuration
This section explains the basic setup of the I/O modules so that you can work
with BladeCenter JS21s. An advanced configuration of the I/O modules is
beyond the scope of this book.
The primary setup tasks for I/O modules are:
Assigning IP addresses to the I/O module management interfaces
Enabling the I/O module external ports
Optionally enabling or disabling “External management over all ports”
You can learn about the selection of IP addresses for the I/O module
management interfaces in “Hardware management subnet” on page 49.
When you first install a new I/O module, the management module assigns a
default IP address to the management interface of the I/O module. The default IP
address is chosen based on the I/O module bay where the I/O module is
installed. The I/O module installed in I/O module bay 1 is assigned the IP
address 192.168.70.127, the I/O module installed in I/O module bay 2 is
assigned 192.168.70.128, the I/O module installed in I/O module bay 3 is
assigned 192.168.70.129, and the I/O module installed in I/O module bay 4 is
assigned 192.168.70.130.
6.4.1 Setting the IP address of an I/O module
Set the IP address of each I/O module management interface and enable the
external ports.
1. Connect to the management module using the Web browser interface as
explained in 6.3.2, “Configuring the management module” on page 75.
2. From the management module Web interface, select I/O Module Tasks →
Configuration.
85
3. In the window shown in Figure 6-12, scroll down to the entry for the I/O
module that you want to configure. Enter the IP address, subnet mask, and
gateway that you want to assign to the I/O module management interface.
Click Save to activate the new IP address. You are prompted to confirm that
you want to change the IP address.
Figure 6-12 I/O module IP address
86
6.4.2 Setting external management over all ports
The setting shown in Figure 6-13 defines whether you can access the
management interface (HTTP, Telnet, SSH, SNMP, and so on) of the I/O module
over the internal management port only (see the “External management over all
ports” switch in Figure 6-3 on page 67) or using the INTx and EXTx ports too.
Figure 6-13 I/O module external management over all ports
87
Note: Although “External management over all ports” is an option of the I/O
module, you can only control it (switch it on and off) using the management
module. You can do this for each I/O module separately.
1. Select the Advanced Configuration link for the I/O module.
2. Scroll down to the Advanced Setup item of the displayed page, as shown in
Figure 6-13. In the “External management over all ports” field, select either
Enabled or Disabled from the drop-down list. Click the Save button.
Note: It depends on your network topology whether you want to enable or
disable the external management of the I/O module over all ports. Several
scenarios are described in Cisco Systems Intelligent Gigabit Ethernet
Switch Module for the IBM eServer BladeCenter, REDP-3869.
6.4.3 Enabling the external I/O module ports
To enable the external I/O module ports:
1. From the management module Web interface, select I/O Module Tasks →
Admin/Power/Restart.
2. In the I/O Module Advanced Setup section, from the Select a module menu,
select the required I/O module. Set the external ports to Enabled, as shown
in Figure 6-14.
3. Click the Save button.
Note: Depending on the firmware level of the management module, this task
can be listed under Configuration instead of Admin/Power/Restart.
88
Figure 6-14 Enabling or disabling the I/O module external ports
Attention: You have to select the I/O module that you want using the Select a
module menu, and not by using the check boxes in the table shown in
Figure 6-14.
At this point, the I/O module management interface has an IP address. Also, the
external ports on the I/O module are enabled so that they can be used to
communicate with blade servers.
The SoL remote text console function of the management module depends on a
VLAN provided by an Ethernet switch I/O module installed in I/O module bay 1.
This VLAN is automatically provided by the 4-port Gigabit Ethernet switch
module and the Nortel Networks Layer 2-7 Gigabit Ethernet switch module using
VLAN 4095.
89
For the Cisco Systems Intelligent Gigabit Ethernet switch module for the IBM
BladeCenter, the SoL VLAN ID is set as follows:
For Cisco I/O supervisor (IOS) releases earlier than 12.1(22)EA6a, you can
set the SoL VLAN ID to any value between 3 and 1001: it cannot be set to the
default value of 4095.
For Cisco IOS release 12.1(22)EA6a and later, you can set the SoL VLAN ID
to a default value of 4095. If necessary, you can still set your own defined
VLAN ID.
For further information, see BladeCenter Serial over LAN Setup Guide or the
Cisco Systems Intelligent Gigabit Ethernet Switch Module for the IBM eServer
BladeCenter, REDP-3869. The Cisco guide describes how to manually set up
the VLAN that is necessary to support the SoL remote text console function. You
can access these guides on the Internet by following the instructions in
“BladeCenter product documentation” on page 64.
6.5 Blade server configuration
Minimal configuration is required for each BladeCenter JS21 before installing an
operating system. The two main configuration tasks are:
Assigning a name to the blade server
Setting the blade server boot sequence
The easiest way to perform these tasks is through the management module Web
interface.
6.5.1 Assigning names to blade servers
Set the name of each blade server in a BladeCenter chassis by performing the
following steps:
1. From the management module Web interface, select Blade Tasks →
Configuration, as shown in Figure 6-15.
2. In the Blade Information section, type the name that you want to assign to
each blade server.
3. Scroll down the page and select the Save button to save the names that you
have assigned to each blade server.
90
Figure 6-15 Blade server naming
6.5.2 Setting the boot sequence
Set the boot sequence of each blade server by using this procedure:
1. From the management module Web interface, select Blade Tasks →
Configuration.
2. Scroll down the page to the Boot Sequence section to see the current boot
sequence for all the blade servers. If a blade server does not have the correct
boot sequence, you can change the boot sequence by selecting the blade
server name.
91
3. This displays the window shown in Figure 6-16. Set the necessary boot
sequence using the drop-down lists. Click the Save button to set the new boot
sequence.
Figure 6-16 Boot sequence of blades
The correct boot sequence depends on the method that you plan to use to install
an operating system on the blade server.
Note: You can also change the boot sequence using the System
Management Services (SMS) (see “Configuring boot device order” on
page 141) or the Open Firmware interface (see boot-device variable in 6.9.1,
“Activating the Open Firmware interface” on page 152). Independent of which
of the three methods you use to change the boot sequence, it will be stored in
the same location in the nonvolatile random access memory (NVRAM).
This means that when you use one method to change the boot sequence, the
changes are also reflected in the other interfaces with at least one exception:
If you select network - BOOTP in the management module’s management
interface (as shown in Figure 6-16), it does not give you an option to specify
the individual Ethernet ports. Using the network - BOOTP option results in two
entries in the boot device order where the secondary adapter (location code
-T8) shows up first as recommended for use with SoL (6.7, “Providing a
console for the BladeCenter JS21” on page 121). It also does not allow you to
specify IP addresses for remote IPL, which is possible in SMS and Open
Firmware.
92
You can see that the boot sequence changes that are made by the management
module’s interface (shown in Figure 6-16 on page 92) are also reflected in the
Open Firmware interface shown in Example 6-1.
Example 6-1 Open Firmware boot sequence
boot-device
network1:speed=auto,duplex=auto,0.0.0.0,,0.0.0.0,0.0.0.0
network:speed=auto,duplex=auto,0.0.0.0,,0.0.0.0,0.0.0.0 cdrom hdd0 hdd1
The Open Firmware aliases network and network1 definitions are shown in
Example 6-2, which shows that the secondary Ethernet port comes first.
Example 6-2 Open Firmware device aliases created by MM
network
network1
/pci@8000000f8000000/pci@2/ethernet@4
/pci@8000000f8000000/pci@2/ethernet@4,1
The boot sequence changes are also reflected in the SMS interface shown in
Figure 6-17.
PowerPC Firmware
Version MB240_470_013
SMS 1.6 (c) Copyright IBM Corp. 2000,2005 All rights reserved.
--------------------------------------------------------------------------Current Boot Sequence
1. Ethernet
( loc=U788D.001.23A1137-P1-T8 )
2. Ethernet
( loc=U788D.001.23A1137-P1-T7 )
3. USB CD-ROM
( loc=U788D.001.23A1137-P1-T1-L1-L3 )
4. SCSI 36401 MB Harddisk, part=1 ()
( loc=U788D.001.23A1137-P1-T10-L1-L0 )
5. SCSI 36401 MB Harddisk, part=1 ()
( loc=U788D.001.23A1137-P1-T11-L1-L0 )
--------------------------------------------------------------------------Navigation keys:
M = return to Main Menu
ESC key = return to previous screen
X = eXit System Management Services
--------------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-17 SMS Configure Boot Device Order: Displaying current setting
93
6.5.3 Assigning the media tray
To assign the media tray to a specific blade server:
1. Select Blade Tasks → Remote Control and select Start Remote Control.
2. A new window opens as shown in Figure 6-18. In the upper right, scroll down
the Change media tray owner menu and select a blade server. You can assign
the media tray to one specific blade server.
Figure 6-18 Blade server management: Remote control
Note: KVM, Remote Disk, and Remote Console are currently not supported
for the blade server JS21.
6.6 Firmware
This section deals with the firmware management of the BladeCenter JS21 and
the various hardware components used in combination with it.
Important: To provide a stable environment for production and to avoid
hard-to-detect problems, spend some time planning and implementing the
firmware management. You must determine if it is a good choice to upgrade to
the latest firmware or to synchronize the firmware versions to an earlier but
compatible level.
94
6.6.1 Getting the latest firmware, tools, and support
IBM provides a great variety of support pages to address different client
requirements. The following sections help you to find firmware and additional
utilities or information.
General support
To stay informed about the BladeCenter JS21 and to get the latest firmware for
BladeCenter JS21 components:
1. Visit the following IBM Web site:
http://www.ibm.com/pc/support
Note: It might happen that there is an automatic redirect to another URL,
but you can always identify the page by looking for Personal computing
support.
On this service Web page, in the Browse by product section, enter the
product or model number of the hardware. In the case of the BladeCenter
JS21, you have to enter the Type number 8844 and click the GO button.
2. In the new window that opens, you can optionally choose a model number, an
operating system, or choose both. Click the Continue button.
3. The BladeCenter JS21 service page opens. On this page, use the Downloads
and drivers link to check the firmware that is available.
IBM BladeCenter: Software and device drivers
To visit the Software and device drivers - IBM BladeCenter Web page, use the
following link:
You can choose BladeCenter JS21 or choose the Link for BladeCenter or
BladeCenter H to get the latest firmware especially for BladeCenter switches.
Finding AIX support for BladeCenter JS21
To find the AIX support Web page for BladeCenter JS21, perform the following
steps:
1. Start at the following Web site:
http://www.ibm.com
On the IBM home page, select Support & Downloads from the main menu at
the top of the page.
95
2. On the Support & downloads page, in the Support by product section, select
Servers.
3. On the Support for IBM Systems page, select the Blade servers link.
4. On the Support for BladeCenter page, in the Select your product section,
select BladeCenter JS21 as Hardware and AIX environment as Software.
Click Go. This opens the Support for BladeCenter JS21page.
Finding Linux support for the BladeCenter JS21 in the Internet
To find the Linux support Web page for BladeCenter JS21, perform the following
steps:
1. Start at the following Web site:
http://www.ibm.com
On the IBM homepage, select Support & Downloads, from the main menu
at the top of the page.
2. On the Support & downloads page, in the Support by product section, select
Servers.
3. On the Support for IBM System page, select the Blade servers link.
4. On the Support for BladeCenter page, in the Select your product section,
select BladeCenter JS21 as Hardware, and Linux on POWER environment
as Software. Click Go.
5. On the Support for BladeCenter JS21 page, in the Tools and utilities section,
select the Diagnostics for Linux on POWER link to get the tools for different
Linux distributions.
Tip: For hardware acquired from another vendor, it might be helpful to also
consult the Web page of the manufacturer.
6.6.2 Identifying and updating the latest firmware in general
If you are planning to use already existing hardware with a BladeCenter JS21,
check the related firmware readme file. It includes a release note about which
version of the BladeCenter JS21 is supported.
You can use the BladeCenter management module to identify many firmware
levels of installed hardware. See Table 6-3 on page 97, which lists the available
update methods and the possible impact on different features. In the
management module, you can access the information by using the menu on the
left. Navigate to Monitors → Firmware VPD. On the command-line interface of
the management module, type the info command to get this information.
96
Tip: To learn more about the BladeCenter management module
command-line interface, see the Management Module Command Line
Interface Reference Guide - IBM BladeCenter and BladeCenter T. To locate
these manuals, refer to the procedure described in “BladeCenter product
documentation” on page 64.
In Table 6-3, the column Update method comprises different approaches to
update the firmware:
Diag-CD
Stand-alone diagnostics CD V5.3.0.40 or later, based on
AIX and diag
OS
Operating system
Web-MM
Web interface of the BladeCenter management module
Com-MM
Command-line interface of the BladeCenter management
module using Trivial File Transfer Protocol (TFTP)
Web-SM
Web interface of BladeCenter I/O modules
Com-SM
Command-line interface of BladeCenter I/O modules
using File Transfer Protocol (FTP) or TFTP
Table 6-3 lists the hardware components and the firmware levels used during the
creation of this book.
Table 6-3 Firmware of common BladeCenter JS21 related hardware compounds
BladeCenter
firmware
components
Update
methoda
Version (build ID)
used for this
bookb
Version
information
available
using MM
Impact
BladeCenter
JS21system
firmware (BIOS)
Diag-CD,
OS
MB240_470_014
Yes
All BladeCenter JS21 related
functions must always be
synchronized with the BSMP
firmware.
BSMP of a
BladeCenter JS21
Diag-CD,
OS,
Web-MM,
Com-MM
BYBT07A
Yes
System management interface
using management module and
BladeCenter JS21
JS21 onboard
Ethernet adapter
Diag-CD,
OS
No
General Ethernet
communication, VLAN, network
boot, SoL
Linux: 3.18
AIX: 03180608
97
BladeCenter
firmware
components
Update
methoda
Version (build ID)
used for this
bookb
Version
information
available
using MM
Impact
Serial-Attached
SCSI (SAS)
Controller
Diag-CD,
OS
06160020
No
SAS interface and onboard
Redundant Array of Independent
Disks (RAID) functions
SAS drives
non-hot
swappable
Diag-CD,
OS
42353139
No
Hard drive and interface
functions
Management
module
Web-MM,
Com-MM
Main application:
1.21F (BRET85F)
Boot ROM: Not
important
Remote control:
1.21F (BRET85F)
Yes
Provided management functions
and all system management
related network communications
including SoL
Advanced
management
module
Web-MM,
Com-MM
V1.1 (BPET15D)
Yes
Cisco Systems
Intelligent Gigabit
Ethernet switch
modulec (Copper
and Fiber version)
Web-SM,
Com-SM
12.1(22)EA6a
Yes
Nortel Networks
Layer 2/3 Copper
or Fiber Gigabit
Ethernet switch
module
Com-SM
1.2.3.11
Yes
4-Port Ethernet
switch module
from IBM (DLink)
Web-SM,
Com-SM
1.08 or 00.00.96
(ibmrun.096)
Yes
IBM BladeCenter
2-port Fibre
Channel switch
module
COM-SM
v2.0.1.09
Yes
General Fibre Channel issues,
especially storage area network
(SAN) boot and interaction with
SAN Utility or SANsurfer
Qlogic 4Gb Fibre
Channel smallform-factor (SFF)
expansion card
Diag-CD,
Driver(IBM): 1.14
Firmware version:
4.00.22
No
General Fibre Channel issues,
especially important for SAN
boot
98
OS
Switch management interface
and all networking features
including VLAN and SoL
BladeCenter
firmware
components
Update
methoda
Version (build ID)
used for this
bookb
Version
information
available
using MM
Impact
IBM Ethernet
expansion card
(LFF/SFF)
Diag-CD,
3.38
No
General Ethernet
communication, VLAN, network
boot
Optical path-thru
module
Web-MM,
Com-MM
9
Yes
Copper path-thru
module
Web-MM,
Com-MM
7
OS
Yes
Interaction with MM, for
example, temperature control
a. The explanations for the abbreviations used are described in the previous section. The supported
update method might change with other firmware levels
b. We recommend that you use at least the mentioned firmware levels to minimize problems and
ensure BladeCenter JS21 support for external components
c. Switch module is an alternative name for I/O module
There are many firmware update possibilities and different hardware
components. Therefore, it is not possible to show all update procedures. The
descriptions in the following sections might be helpful even if the actual case is
slightly different. For example, the firmware update process of different
expansion cards with the stand-alone diagnostics CD is almost identical. In all
cases, we recommend that you read the readme file related to the firmware.
Tip: To automate all firmware update processes, a script language based on
the Tcl named Expect is useful. See the following Web site:
http://expect.nist.gov
6.6.3 Updating BladeCenter JS21 firmware using AIX
AIX 5L includes diag to update the BladeCenter JS21 firmware (BIOS). The
procedure is very similar to the stand-alone diagnostics CD, hence diag is used
by the stand-alone diagnostics CD. The flash procedure with diag is not
described in this section, see 6.6.9, “Updating the firmware using the stand-alone
diagnostics CD” on page 111 for more information about this topic.
99
It is also possible to use the update_flash utility as part of the AIX 5L diagnostics
package. The following section illustrates the usage of this utility (you require
root privileges):
1. Download the firmware update package from the IBM support Web site. See
6.6.1, “Getting the latest firmware, tools, and support” on page 95. The
firmware image itself is delivered using an IMG file.
Note: Even if the IMG file size is approximately 24 MB, it is meant to be
used as a flash image for the 4 MB flash memory. Typically, the IMG file
contains more than the firmware data of a BladeCenter JS21.
2. To use the update_flash utility, the operating system running on the
BladeCenter JS21 must have access to the IMG file. If the name of the
firmware IMG file is mb-240.470.013_ppc64 and the file is in the actual
directory, enter the following command to verify an IMG file:
/usr/lpp/diagnostics/bin/update_flash -v -f
ibm_fw_bios_mb-240.470.013_anyos_ppc64.img
3. Use the following command to update the firmware:
/usr/lpp/diagnostics/bin/update_flash -f
ibm_fw_bios_mb-240.470.013_anyos_ppc64.img
4. The update_flash utility asks for a confirmation as shown in Figure 6-19.
After confirmation, the update of the temporary system firmware starts. To
avoid the confirmation dialog, additionally use the parameter -q.
# /usr/lpp/diagnostics/bin/update_flash -f mb-240.470.013_anyos_ppc64.img
The image is valid and would update the temporary image to MB240_470_013.
The new firmware level for the permanent image would be MB240_470_012.
The current permanent system firmware image is MB240_470_012.
The current temporary system firmware image is MB240_470_012.
***** WARNING: Continuing will reboot the system! *****
Do you wish to continue?
Enter 1=Yes or 2=No
Figure 6-19 Confirmation dialog of the update_flash utility
100
5. If there are problems with the new firmware it is possible to use the firmware
image stored on the permanent side of the flash memory to reject the new
version. Enter the following command to copy the firmware level of the
permanent system firmware image to the temporary side:
/usr/lpp/diagnostics/bin/update_flash -r
Note: The firmware stored on the temporary side of the flash memory is
the default firmware used during a BladeCenter JS21 startup. It is possible
to boot the firmware stored on the permanent side of the flash memory by
using the SMS interface. See 6.8, “System Management Services
interface” on page 133.
Alternatively, if the new firmware works fine, you can commit the firmware
update and copy the image from the temporary side to the permanent side.
To do this, use the following command:
/usr/lpp/diagnostics/bin/update_flash -c
6.6.4 Updating BladeCenter JS21 firmware (BIOS) using Linux
IBM provides a script for Linux called update_flash that uses the interfaces
provided by rtas_flash. It supports the same options and syntax as the
update_flash utility under AIX. This tool uses the Run-Time Abstraction Services
(RTAS), which enables access to the firmware during the run time of the
operating system. The interface functions are included in most recent 2.4 kernels
and in all 2.6 kernels.
You can obtain the latest version of the update_flash script by installing the
diagnostic utilities for Linux on Advanced Performance Optimization with
Enhanced RISC (POWER) that IBM distributes from the Web at:
https://www14.software.ibm.com/webapp/set2/sas/f/lopdiags/home.html
Note: Some Linux distributions provide a version of the update_flash script
that is not fully functional. Always obtain the latest version of the update_flash
script from IBM before you attempt to update the firmware.
101
To use the update_flash script, follow these steps:
1. Download the latest firmware from the IBM support Web site.
2. Transfer it to a disk that is accessible to the operating system running on the
BladeCenter JS21.
3. Enter the update_flash command from the directory where the new firmware
is located. The update_flash command requires that you have root privileges.
The following example illustrates the usage of update_flash:
/usr/sbin/update_flash -f ibm_fw_bios_mb-240.470.013_anyos_ppc64.img
In this example, ibm_fw_bios_mb-240.470.013_anyos_ppc64.img is the
firmware image that was previously downloaded from the IBM support Web site.
The update_flash utility copies the new firmware image into the kernel and then
reboots the operating system to perform the actual firmware update.
Subsequently, if you have a problem with the new firmware and want to revert to
the previous firmware level, then use the following command:
/usr/sbin/update_flash -r
If you are satisfied with the new firmware, commit the firmware upgrade before
you install any future firmware by using the following command:
/usr/sbin/update_flash -c
6.6.5 Updating BladeCenter JS21 Ethernet adapter using Linux
Get the Broadcom flash diagnostic utility as a Red Hat Package Manager (RPM)
file. See “Finding Linux support for the BladeCenter JS21 in the Internet” on
page 96. This RPM is part of the Linux on POWER diagnostic package. The
software version used in this book is V1.24. With the RPM stored on the JS21 in
the /srv/data/tools/ directory, issue the following command:
rpm -i /srv/data/tools/bcmflashdiag-js20-1.24-1.ppc.rpm
After you install the RPM, you can obtain information about the installed firmware
level by issuing the following command:
/usr/sbin/bcmflashdiag ethX
102
Here x is the Ethernet port in the range of 0 to 3. The output of this command
depends on the actual firmware level, one possible output is shown in
Figure 6-20. With an installed Ethernet expansion card, the onboard Ethernet
adapter ports are in most cases eth2 and eth3. The firmware is always identical
for eth0 and eth1 and accordingly for eth2 and eth3 because each interface pair
uses one common firmware. Therefore, a firmware update for eth0 also affects
eth1 and the same is true for eth2 and eth3.
/usr/sbin/bcmflashdiag eth3
Firmware versions:
Type
--------BootCode
ASF CFG
ASF CPUB
ASF CPUA
ASF Init
Version
------5780s-v3.18
40
ASFIPMI v6.08
ASFIPMI v6.08
ASFIPMI v6.08
Figure 6-20 Firmware information of onboard Ethernet adapter eth3
To update to a new firmware level, if the firmware is located in
/srv/data/firmware/, use the following command:
/usr/sbin/bcmflashdiag -n /srv/data/firmware/FIRMWARE.BIN ethX
Here FIRMWARE.BIN is a placeholder for the actual name of the binary firmware
file. See the command line output produced by this command in Figure 6-21. The
flash process does not interrupt the data transfer on the interface, but you have to
restart the BladeCenter JS21 to activate the new firmware.
Attention: The binary firmware files for BladeCenter JS21 onboard Ethernet
and Ethernet expansion cards are not compatible.
To get a list with more options, issue /usr/sbin/bcmflashdiag without any
parameters.
103
/usr/sbin/bcmflashdiag -n js21_318_608_ver2.bin eth3
Firmware versions:
Type
--------BootCode
ASF CFG
ASF CPUB
ASF CPUA
ASF Init
Version
------5780s-v3.17
40
ASFIPMI v6.08
ASFIPMI v6.08
ASFIPMI v6.08
Start programming Broadcom firmware...
File Name
: js21_318_608_ver2.bin
File Size
: 131072
Offset
: 0x0000
Bytes Read : 131072
Restoring original MAC address 00-11-25-C9-0B-A6
Restoring original MAC address 00-11-25-C9-0B-A7
Restoring original Serial Number: 0123456789
Broadcom firmware has been upgraded successfully. Reboot the machine
to take effect.
Figure 6-21 Command line output during and after the flash process
6.6.6 Updating the management module and advanced management
module firmware using the Web interface
To upgrade the firmware of the BladeCenter management module itself using the
Web interface, perform the following steps:
6.6.1, “Getting the latest firmware, tools, and support” on page 95. This is
usually a compressed file that contains one or more files with a PKT
extension.
Note: The management module requires two PKT files. For the advanced
management module, we found the package to contain only one PKT file.
104
2. Extract the firmware update package into a directory on the workstation
where you are running the Web browser that you use to connect the
management module Web interface.
Note for X Server users: Make sure that the directory where the PKT file
is stored is accessible from the computer system that provides the Web
browser.
3. Update each PKT file to the BladeCenter management module as explained
in the following steps.
a. In the navigation panel of the management module Web interface, select
MM Control → Firmware Update, as shown in Figure 6-22.
b. In the Update MM Firmware pane, select the Browse button and locate
the directory where the PKT file is stored.
c. Click the Update button to send and check the PKT file to the
management module. This step might take some time to complete.
Note: The Web interface is locked during download or update
processes, but the command-line interface is still available.
Figure 6-22 Updating the management module firmware
105
d. A window opens that shows the progress of the download. Click the
Continue button when prompted to confirm the firmware update. This step
might also take some time to complete.
Another window opens that shows you the progress.
4. After all PKT files are updated, you must restart the management module to
activate the new firmware. Select MM Control → Restart MM in the
You can also use the update command on the BladeCenter management module
command-line interface in combination with a running TFTP server. See 9.2.3,
“Configuring a Trivial File Transfer Protocol service” on page 287.
6.6.7 Updating I/O module firmware using a Web interface
The method to update the firmware of a BladeCenter I/O module depends on the
type of the module. As an example, we present the firmware update of a 4-port
Gigabit Ethernet switch module from IBM.
For information about upgrading firmware of other BladeCenter I/O modules use
the search function in the following Web site:
http://www.redbooks.ibm.com
In the Search field, type BladeCenter AND Module.
To update the firmware of a 4-port Gigabit Ethernet switch module from IBM
using the switch module Web interface, follow this procedure:
6.6.1, “Getting the latest firmware, tools, and support” on page 95. This is
usually a compressed file that contains the firmware file. In this case it is
called ibmrun.096.
2. Extract the firmware update package into a directory on the workstation
where you are running the Web browser that you use to connect the
3. Before the firmware of the switch module can be updated, configure an IP
address for the switch module management interface. This enables the
capability to directly connect to the switch module Web interface. For details
about how to configure the IP address of the switch module management
interface, see 6.4, “I/O module configuration” on page 85.
106
4. Connect to the switch module Web interface using the management module
Web interface using the following procedure:
a. From the management module Web interface, select I/O Module
Tasks → Management.
b. Select the Advanced Management link for the I/O module bay where the
switch module is installed.
c. Scroll down to the bottom of the page to the Start Telnet/Web Session
category. Click the Start Web Session button.
d. A new window opens for the switch module Web interface. You are
prompted to provide a user ID and a password.
The default user ID is USERID, and the default password is PASSW0RD.
Here the number 0 (zero) is between the letters W and R in the password.
Consider changing the defaults in a production environment.
You can also connect directly to the switch module user interface from any
Web browser, if you know the IP address of the switch module.
5. Upgrade the firmware using the following procedure:
a. From the switch module Web interface, select Maintenance → Using
Browser → Upgrade Firmware/Configuration File.
b. You see the screen shown in Figure 6-23. Select the Browse button and
locate the switch module firmware file in the directory where you saved it
earlier.
c. Select the Start button to initiate the switch module firmware update
process.
107
Figure 6-23 Firmware upgrade of 4-port Gb Ethernet switch module
6. A window opens requesting confirmation. After you confirm the update, the
main window displays the status of the update until it completes and reboots
the switch module.
7. After the restart of the Ethernet switch module, verify that the IP address of
the switch module is still set correctly through the management module Web
interface.
6.6.8 Updating IBM BladeCenter 2-port Fibre Channel switch module
firmware using FTP
A graphical interface to this switch is provided by external tools such as the
Blade Center SAN Utility and in most cases, you must update the firmware using
these tools. See also Chapter 12, “Storage area network installation and booting”
on page 409.
In some cases, it might be necessary to perform a direct firmware update using
the FTP service running on the switch module. This is described in the following
section. We assume that the switch module is already configured using the
BladeCenter management module. See 6.4, “I/O module configuration” on
page 85.
108
Note: In other cases, it might be necessary to have an external FTP or TFTP
server running to update firmware. To learn how to set up these services, see
9.2, “Basic preparations for a Linux network installation” on page 283.
1. Store the firmware file in a directory on your workstation, for example,
/srv/data/firmware/IBM_2port_fc_switch/firmware_2.0.1.09
2. Connect to the FTP server running in the Fibre Channel switch module to
transfer the firmware file. In this case, the module host name is ibm_fcsm_1.
Log in as images, use the password images, and send the file to the Fibre
Channel switch module. Example 6-3 shows the command line dialog. The
commands are highlighted.
Example 6-3 FTP command line dialog
myworkstation:/srv/ftp/firmware # ftp ibm_fcsm_1
Connected to ibm_fcsm_1.
220 Sanbox2 FTP server (Version wu-2.6.1-18) ready.
Name (ibm_fcsm_1:root): images
331 Password required for images.
Password:******
230 User images logged in. Access restrictions apply.
Remote system type is UNIX.
Using binary mode to transfer files.
ftp> put firmware_2.0.1.09
local: firmware_2.0.1.09 remote: firmware_2.0.1.09
500 'EPSV': command not understood.
227 Entering Passive Mode (192,168,1,200,26,53)
150 Opening BINARY mode data connection for firmware_2.0.1.09.
100% |*************************************| 9148 KB
2.30 MB/s
00:00 ETA
226 Transfer complete.
9367804 bytes sent in 00:03 (2.29 MB/s)
ftp> quit
221-You have transferred 9367804 bytes in 1 files.
221-Total traffic for this session was 9368346 bytes in 1 transfers.
221 Thank you for using the FTP service on Sanbox2.
109
3. Connect using Telnet, with the user name USERID and password as PASSW0RD.
Issue the highlighted commands as shown in Example 6-4 to finish the
firmware activation.
Example 6-4 Telnet command line dialog
linux:/srv/ftp/firmware # telnet ibm_fcsm_1
Trying ibm_fcsm_1...
Connected to ibm_fcsm_1.
Escape character is '^]'.
Linux 2.4.18-xfs (Sanbox2) (01:28 on Friday, 01 January 1988)
Sanbox2 login: USERID
Password:********
Establishing connection...
Please wait.
*****************************************************
*
*
*
Command Line Interface SHell (CLISH)
*
*
*
*****************************************************
SystemDescription
Switch Module
Eth0NetworkAddress
FCNetworkAddress
MACAddress
WorldWideName
SymbolicName
ActiveSWVersion
ActiveTimestamp
DiagnosticsStatus
SecurityEnabled
IBM BladeCenter(TM) 2-port Fibre Channel
9.3.5.23 (use 'set setup system' to update)
0.0.0.0 (use 'set setup system' to update)
00:c0:dd:02:1c:21
10:00:00:c0:dd:02:1c:22
FCSM
V2.0.0.15-0
Wed Oct 29 21:18:28 2003
Passed
False
Alarms history ...
-----------------[1][Fri Jan 01 00:01:08.332 1988][A1005.0002][cmon: unable to set gw
addr]
FCSM: USERID> admin start
110
FCSM (admin): USERID> image list
firmware_2.0.1.09
FCSM (admin): USERID> image unpack firmware_2.0.1.09
Processing the command ...
A notification with the command result will appear
on the screen when the command has completed.
FCSM (admin): USERID>
image unpack command result: Passed.
The firmware will become active and the SWImageVersion
information will be updated after the next switch reset.
reset
The switch will be reset.
Please confirm (y/n): [n] y
Alarm Msg: [Fri Jan 01 01:30:25.880 1988][A1000.000B][SM][The switch
will be reset in 3 seconds]
Reset switch in progress...
Connection closed by foreign host.
6.6.9 Updating the firmware using the stand-alone diagnostics CD
The stand-alone diagnostics CD contains a bootable mini-AIX and uses diag to
flash various firmware as described in Table 6-3 on page 97. You can use a
Network Installation Manager (NIM) server to boot the stand-alone diagnostics to
avoid the usage of the BladeCenter CD/DVD drive. See Chapter 8, “Installing
AIX” on page 259. However, the following procedure uses the CD/DVD drive.
111
Tip: If you use a NIM server, you must select diag, as shown in Figure 8-8 on
page 269. If you copy the firmware image into usr/lib/microcode on the NIM
server's Shared Product Object Tree (SPOT), the client can access this image
from /usr/lib/microcode, which is the Network File System (NFS) mounted
from the NIM server during the network boot. You do not have to assign the
media tray or use a CD or DVD media in this case.
To enable the firmware update of onboard Ethernet or expansion cards using
diag, it is essential that the firmware file name starts with an uppercase
I.xxxxxxxxx and not with lowercase i.xxxxxxxxx.
As an example, we describe the update process for the Qlogic 4Gb Fibre
Channel expansion card.
1. Assign the CD/DVD drive to the BladeCenter JS21 where the expansion card
for the update is installed.
2. Set the CD/DVD drive as first boot device using the management module.
3. Start the BladeCenter JS21 and wait until the boot process is finished.
112
4. Type 1 and press Enter to activate the console, as shown in Figure 6-24. Then
follow the directions in the subsequent menus.
-------------------------------------------------------------------Welcome to AIX.
boot image timestamp: 15:58 12/19
The current time and date: 15:08:30 06/06/2006
number of processors: 4
size of memory: 3968MB
boot device:
/pci@8000000f8000000/pci@7/usb@0/hub@1/hub@1/cdrom@3:\ppc\chrp\bootf
ile.exe
kernel size: 13421062; 64 bit kernel
-------------------------------------------------------------------******* Please define the System Console. *******
Type a 1 and press Enter to use this terminal as the
system console.
Pour definir ce terminal comme console systeme, appuyez
sur 1 puis sur Entree.
Taste 1 und anschliessend die Eingabetaste druecken, um
diese Datenstation als Systemkonsole zu verwenden.
Premere il tasto 1 ed Invio per usare questo terminal
come console.
Escriba 1 y pulse Intro para utilizar esta terminal como
consola del sistema.
Escriviu 1 1 i premeu Intro per utilitzar aquest
terminal com a consola del sistema.
Digite um 1 e pressione Enter para utilizar este terminal
como console do sistema.
Figure 6-24 Selecting the system console
113
5. When the Function Selection menu opens, type 3 and press Enter, as shown
in Figure 6-25.
FUNCTION SELECTION
1 Diagnostic Routines
This selection will test the machine hardware. Wrap plugs and
other advanced functions will not be used.
2 Advanced Diagnostics Routines
other advanced functions will be used.
3 Task Selection (Diagnostics, Advanced Diagnostics, Service Aids,
etc.)
This selection will list the tasks supported by these procedures.
Once a task is selected, a resource menu may be presented showing
all resources supported by the task.
4 Resource Selection
This selection will list the resources in the system that are
supported
by these procedures. Once a resource is selected, a task menu will
be presented showing all tasks that can be run on the resource(s).
99 Exit Diagnostics
NOTE: The terminal is not properly initialized. You will be
prompted to
initialize the terminal after selecting one of the above
options.
To make a selection, type the number and press Enter.
Figure 6-25 Function Selection menu
114
[3 ]
6. In most cases, you can choose the vt340 terminal type in the Define Terminal
menu, as shown in Figure 6-26. With this terminal type, most functions keys
(F-keys) must work as expected. However, even if the F1 to F4 keys are not
working, you can press Esc and then use the regular 1, 2, 3, or 4 key to select
F1 to F4 in a menu.
DEFINE TERMINAL
The terminal is not properly initialized.
The following are some of the terminal types that are supported.
ibm3101
ibm3151
ibm3161
ibm3162
ibm3163
ibm3164
ibmpc
lft
NOTE:
tvi912
tvi925
tvi920
tvi950
vs100
vt100
vt320
sun
vt330
vt340
wyse30
wyse50
wyse60
wyse100
wyse350
If you are using a Graphics Display, such as a
5081 or 6091 display, enter 'lft' as the
terminal type.
If the next screen is unreadable, press <CTRL> C.
Please enter a terminal type, or press Enter to return.
Figure 6-26 Define Terminal menu
115
7. After the Resource Database is built, the Task Selection List menu opens.
Use the cursor keys to scroll down the list and choose Microcode Tasks, as
shown in Figure 6-27.
TASKS SELECTION LIST
801004
From the list below, select a task by moving the cursor to
the task and pressing 'Enter'.
To list the resources for the task highlighted, press 'List'.
[MORE...18]
Display or Change Bootlist
Format Media
Hot Plug Task
Identify and Attention Indicators
Local Area Network Analyzer
Microcode Tasks
Process Supplemental Media
RAID Array Manager
SSA Service Aids
This selection provides tools for diagnosing and resolving
problems on SSA attached devices.
Update and Manage System Flash
[BOTTOM]
F1=Help
F3=Previous Menu
F4=List
F10=Exit
Enter
Figure 6-27 Task Selection List menu with highlighted Microcode Tasks selection
116
8. In the Microcode Tasks menu, choose Download Microcode to update the
firmware of expansion cards or onboard devices, as shown in Figure 6-28.
Microcode Tasks
801004
Move cursor to desired item and press Enter.
Display Microcode Level
Download Microcode
Generic Microcode Download
F1=Help
F3=Previous Menu
F4=List
F10=Exit
Enter
Figure 6-28 Microcode Tasks
117
9. The Resource Selection List menu opens. The selectable devices depends
on the specific BladeCenter JS21 configuration. Move the cursor to the
device you require for a firmware update, as shown in Figure 6-29. Press F7
to commit the selection.
RESOURCE SELECTION LIST
801006
From the list below, select any number of resources by moving
the cursor to the resource and pressing 'Enter'.
To cancel the selection, press 'Enter' again.
To list the supported tasks for the resource highlighted, press
'List'.
Once all selections have been made, press 'Commit'.
To avoid selecting a resource, press 'Previous Menu'.
All Resources
This selection will select all the resources currently
displayed.
U788D.001.23A1137sisioa0
P1
PCI-XDDR Dual Channel SAS
RAID Adapter
+ fcs0
P1-C5-T1
FC Adapter
fcs1
P1-C5-T2
FC Adapter
ent0
P1-T7
Gigabit Ethernet-SX PCI-X
Adapter
(14101403)
ent1
P1-T8
Gigabit Ethernet-SX PCI-X
Adapter
(14101403)
F1=Help
F3=Previous Menu
F4=List
F7=Commit
Figure 6-29 Resource Selection List with fcs0 selected for firmware update
118
F10=Exit
10.If the device has two ports, for example, fcs0 and fcs1, both the ports will be
updated, even if you have selected only one port. See Figure 6-30.
fcs0
INSTALL MICROCODE
FC Adapter
802113
Please stand by.
lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x
x
x
x
x [TOP]
x
x
*** NOTICE *** NOTICE *** NOTICE ***
x
x
x
x fcs0 represents a port on a dual-port
x
x fibre channel adapter. To update microcode on
x
x this adapter, microcode needs to be installed
x
x on both fcs0 and fcs1.
x
x Selecting either fcs0 or fcs1 installs
x
x microcode on both ports.
x
x [MORE...11]
x
x
x
x F3=Cancel
F10=Exit
Enter
x
F3=Cancel
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj
Figure 6-30 Notice that both ports will be updated
11.In the next menu, select cd0 and press Enter.
119
12.Insert the CD with the microcode file stored in the root directory of the CD and
confirm that the medium is in the CD/DVD drive. If the system does not find
firmware files in the /etc/microcode directory on the CD-ROM, another
message is shown. If you use the stand-alone diagnostics CD, this is always
the case. If a valid microcode file is found, it is shown in the menu, as shown
in Figure 6-31. Press Enter to start the update process.
fcs0
INSTALL MICROCODE
FC Adapter
802116
The current microcode level for fcs0
is 0003040010.
Available levels to install are listed below.
Select the microcode level to be installed.
Use Help for explanations of "M", "L", "C"
and "P".
Make selection, use Enter to continue.
M
F1=Help
0117040022
F10=Exit
F3=Previous Menu
Figure 6-31 Diag found a valid microcode on the CD
120
After the firmware update, you get the message that the installation is
completed successfully, as shown in Figure 6-32.
INSTALL MICROCODE
fcs0
802118
FC Adapter
Installation of the microcode has completed successfully
on fcs0 and fcs1.
The current microcode level is 0003040016.
Please run diagnostics on the resources fcs0 and fcs1 to
ensure that the adapter is functioning properly.
Use Enter to continue.
F3=Cancel
F10=Exit
Enter
Figure 6-32 Update successful message
6.7 Providing a console for the BladeCenter JS21
The BladeCenter servers JS20 and JS21 differ from other blade servers that are
available for the BladeCenter in that they do not provide an interface to a KVM
console. Therefore, you must set up the SoL remote text console function to
provide a console (American Standard Code for Information Interchange (ASCII)
terminal) for a BladeCenter JS20 and JS21. The use of SoL is optional for other
types of blade servers that support KVM consoles.
SoL is a virtual serial connection over Ethernet. It overcomes the limitations
posed by a blade server with no video controller, and also offers an alternative
way of connecting to an operational blade server.
Note: You can use SoL to connect to running systems as well. However, we
recommend that you use standard network access methods over the
production network path if the network connection is working well because it
provides a much better performance, and there is less risk of dropping
connections. SoL is mostly intended during installation or when a normal
network connection fails for some reason.
121
SoL provides a connection between the BladeCenter server and the
management workstation in the external management network as shown in
Figure 6-33 on page 123. Typically, there are two connections: One from the
management workstation to the management module using Telnet or SSH, and
one from the management module to the BSMP using SoL. The management
module acts as a proxy between these two connections. Therefore, you cannot
reach the BSMP directly from the management workstation.
Note: The SoL ends at the BSMP that emulates the console (ASCII terminal)
and not at the BladeCenter server’s IBM Power PC® processor. Therefore,
the SoL session can be active even when the BladeCenter server is powered
off.
In addition, the ESM in switch bay 1 is used to create the path from the
management module to the BSMP. However, its function is transparent if it is set
up correctly.
Note: For SoL to work, it is essential to have a supported and properly setup
ESM in I/O module bay 1. SoL will not work over a copper or optical pass-thru
module in bay 1 or an ESM in bay 2.
122
BladeCenter
Chassis
Management Module (MM)
INT (eth1)
Management
Interface
Telnet/SSH
SoL Proxy
EXT (eth0)
MANAGEMENT MODULE BAY 1
MGT 1
EXT 1
MGT 2
EXT 2
INT 1
EXT 3
..
Port 2 (eth1)
Port 1 (eth0)
INT 14
EXT 4
I/O MODULE BAY 1
Management
Workstation
SoL Data
BSMP
Management
Network
Build in Ethernet
Controller
Network (LAN)
BladeCenter JS21
BAY 1
SoL VLAN
default ID = 4095
Figure 6-33 SoL schematics
Note: Although it is possible to have concurrent SoL and regular Ethernet
traffic across ESM 1 and the eth0 port of the blade server (Figure 6-33), there
is a tendency for the SoL session to drop because of the traffic volume and
resets of the eth0 port caused by the running software. The production
network is typically configured to use ESM 2 and the eth1 Ethernet controller
of the blade server, which requires at least two ESMs.
The SoL remote console function works as follows:
1. Using a Telnet or SSH client, connect to the BladeCenter management
module CLI. This is usually using an external management network that is
connected to the management module’s 10/100BaseT Ethernet interface.
2. From the management module’s CLI, initiate an SoL remote console session
to the blade server that you want.
123
3. The management module uses a private VLAN (typically VLAN ID 4095)
provided by the Ethernet switch module in I/O module bay 1 (ESM 1), to
transport the SoL data stream to the Ethernet interface of the target blade
server’s primary Ethernet interface (eth0).
4. The Ethernet controller (network interface controller (NIC)) of the target blade
server passes the SoL data stream received from the private VLAN to the
BSMP using I2 C, which manages the ASCII text console for the blade server.
Note: The management module can handle 20 login sessions in parallel.
Therefore, it is possible to have up to 14 SoL sessions (one for each blade
server) active at the same time leaving at least 6 additional logins open for
other purposes.
6.7.1 Configuring Serial over LAN
Before you attempt to configure the SoL remote text console function, verify that
you have all the prerequisites in place:
A supported LAN switch module installed in I/O module bay 1
This switch module is used to provide a VLAN (typical VID 4095) that
connects the management module to the first Ethernet interface (eth0) on
each blade server.
The minimum firmware levels described in 6.6, “Firmware” on page 94
This is important if you install a BladeCenter JS21 in an existing BladeCenter
chassis that might have earlier firmware in the management module or
Ethernet switch modules.
A reliable Telnet or SSH client on the management workstation
Note: Depending on the operating system where you run the Telnet or
SSH client, there might be differences in the mapping of special keys such
as function keys, cursor keys, Backspace, Delete, and Insert keys.
Especially when you log on to AIX or Linux, you might encounter difficulties
in System Management Interface Tool (SMIT) or Yet Another Setup Tool
(YaST) menus. For most purposes, we found PUTTY to be a good choice
for the client.
A network connecting the management workstation to the BladeCenter
management module external 10/100BaseT Ethernet interface
An identified range of IP addresses that is used by the management module
to communicate with the BSMP on each blade server using the private VLAN.
This is discussed in Chapter 5, “Planning considerations” on page 45.
124
The easiest way to configure the SoL remote text console function is through the
management module Web interface as explained in the following steps.
Alternatively, you can perform the same task using the management module’s
CLI.
To configure the SoL remote text console function, use the following procedure:
1. From the management module Web interface, select Blade Tasks → Serial
Over LAN, as shown in Figure 6-34. In the right pane, scroll down to the
Serial Over LAN Configuration section. Complete the following tasks:
a. From the Serial over LAN list, select the Enabled option.
b. Leave the value for SoL VLAN ID at the default (4095) if you have either a
4-port Gigabit Ethernet switch module or a Nortel Networks Layer 2-7
Gigabit Ethernet switch module installed in I/O module bay 1.
If you have a Cisco Systems Intelligent Gigabit Ethernet switch module
installed in I/O module bay 1, set the VLAN ID to the same value that you
used when you configured the Cisco Systems Intelligent Gigabit Ethernet
switch module as described in 6.4.3, “Enabling the external I/O module
ports” on page 88.
c. In the BSMP IP address range field, enter the start of the IP address range
that will be used by the management module to communicate with the
BSMP on each blade server.
d. Leave the values for Accumulate timeout, Send threshold, Retry count,
and Retry interval at their defaults (5, 250, 3, and 250).
e. In the User Defined Keystroke Sequences section, leave the values at
their defaults.
f. Click the Save button.
125
Figure 6-34 Serial over LAN configuration
2. Restart the management module. To do this, in the management module
Web interface, select MM Control → Restart MM. Click the Restart button
on the displayed page. You are prompted to confirm the restart before it
occurs.
Important: When you configure the SoL VLAN on the management
module, it never asks you to reboot after saving. We recommended that
you restart the management module after you configure the SoL VLAN
properly, especially if the SoL session status does not become ready.
3. After you restart the management module and reconnect to the management
module Web interface from your Web browser, enable the SoL remote text
console for each blade server by doing the following steps.
a. Select Blade Tasks → Serial Over LAN from the management module
Web interface. In the right pane, scroll down until you see the Serial Over
LAN Status section (see Figure 6-35 on page 127).
126
b. Select the blade servers you want to enable for SoL. You can choose all of
them by selecting the check box at the top of the table.
c. Click the Enable Serial Over LAN link below the table. You might have to
scroll down to see this link.
d. After a few seconds, the window refreshes. In the SoL column of the table,
verify that each blade server has a status of Enabled and the SoL Session
column has a status of Ready.
Note: If the blade server does not show the Ready state, use Blade
Tasks → Power/Restart → Restart Blade System Mgmt Processor or
Blade Tasks → Power/Restart → Restart Blade.
Figure 6-35 Serial over LAN status
The configuration of the SoL remote text console function is now complete.
127
6.7.2 Using Serial over LAN
Access to the SoL remote text console function is through the management
module CLI. The CLI is documented in the IBM BladeCenter Management
Module Command-Line Interface Reference Guide, which is one of the
documents provided on the CD supplied with your BladeCenter. You can also find
it on the product document site described in “BladeCenter product
1. Connect to the management module CLI using either a Telnet or SSH client.
Note: To use SSH, you must first configure the management module using
MM Control → Security as described in Advanced Management Module
and Management Module User's Guide - IBM BladeCenter, BladeCenter T,
BladeCenter H.
2. You are prompted to enter a user ID and password.
Note: The default user ID is USERID and the default password is
PASSW0RD, where 0 is a zero. All the letters are in uppercase. Consider
changing the defaults in a production environment.
Figure 6-36 shows an active Telnet session that displays the help command.
128
username: USERID
password: ********
system> help
?- Display commands
alertentries- View/edit remote alert recipients
boot- Boot target
clear- Clear the config
clearlog- Clear the event log
console- Start SOL session to a blade
dhcpinfo- View DHCP server assigned settings
displaylog- Display log entries
dns- View/edit DNS config
env- Set persistent command target
exit- Log off
fuelg- Power management
health- View system health status
help- Display command list
history- Display command history
identify- Control target location LED
ifconfig- View/edit network interface config
info- Display identity and config of target
list- Display installed targets
power- Control target power
reset- Reset target
shutdown- Shutdown target
slp- View/edit SLP parameters
smtp- View/edit SMTP config
snmp- View/edit SNMP config
sol- View SOL status and view/edit SOL config
tcpcmdmode- View/edit TCP command mode config
telnetcfg- View/edit telnet config
update- Update firmware from TFTP server
uplink- View/edit failover on network uplink loss config
users- View/edit user login profiles
Type "<command> -h"
[ ] is used
< > denotes
{ } denotes
| denotes
system>
for individual command syntax help.
for indexing (by bay number)
a variable
optional arguments
choice
Figure 6-36 Management module’s command-line interface
129
The management module CLI is context-sensitive. When you issue CLI
commands, you can accept the current context or override the context using the
-T option provided on most commands. In our examples, we use the -T option to
make it clear on which entity the command is operating. You can change the
context for the management module CLI by using the env -T command.
There are different commands that you can use to access the SoL remote text
console function. Table 6-4 lists the most useful commands.
Table 6-4 Management module commands for SoL
Command
Function
console
Opens an SoL remote text console for the blade server. This command fails if another
SoL remote text console is already open for the blade server.
console -o
Terminates any existing SoL remote text console for the blade server and opens an
SoL remote text console for the blade server.
boot -c
Resets the blade server, and then opens an SoL remote text console for the blade
server.
env -T
One of the built-in commands. Sets the target (management module, BladeCenter,
blade server, or switch) for the current session.
reset -c
This is functionally equivalent to the boot -c command when used in a blade server
context.
power -on -c
Powers on the blade server and then opens an SoL remote text console for the blade
server.
power -cycle -c
Powers on the blade server and then opens an SoL remote text console for the blade
server. If the blade server is already powered on, powers it off first, and then powers on.
We provide some examples of how to use these commands.
To open an SoL remote text console to the blade server in bay 3, use the
following command:
console -T system:blade[3]
Note: You can open many types of consoles to an installed BladeCenter
JS21: SoL, SSH, Virtual Network Computing (VNC), and others. You can
have only one active SoL remote text console connection to a BladeCenter
JS21. When you try to open a second SoL session, you get the message:
SOL session is already active. To force a console to be established if it
is already active from elsewhere use:
console -T system:blade[3] -o
This interrupts the already existing SoL session.
130
To reset the blade server in bay 2 and then open an SoL remote text console
to the blade server, use the following command:
boot -c -T system:blade[2]
To set the management module in bay 1 as the persistent target for
commands during the current session:
env -T system:mm[1]
Note: To terminate an active SoL remote text console, press the Esc key
followed by an open parenthesis “(“ (Shift+9 on U.S. keyboards).
You can see active SoL sessions in the Web interface as shown for the blade
server in bay 3 in Figure 6-37. Using the CLI, the following command shows you
the status for only one blade server at a time:
sol -T system:blade[3]
Figure 6-37 Active SoL session for blade server in bay 3
When the SoL remote text console ends, you return to the management module
CLI prompt.
131
Inactivity timeout
The management module CLI has an aggressive default inactivity timeout. If you
do not use the CLI or an SoL remote text console accessed through the CLI for
2 minutes, you are disconnected. You can increase the timeout using the
telnetcfg command. For example, the following command increases the
inactivity timeout to 10 minutes (600 seconds):
telnetcfg -t 600 -T system:mm[1]
Note: The command telnetcfg -t 0 means no timeout.
Setting carriage return-line feed
Over the years, there have been many variations on the interpretation of the
Enter key in a Telnet session. Some Telnet daemons expect carriage return-line
feed (CRLF) and some expect carriage return alone (CR). From our testing, CR
seems to be the most compatible. During your SoL session, check whether the
key acts as though it is pressed twice and not once when you press the Enter
key. If it does, then you have to correct the CRLF option. Under Windows 2000
Telnet, do this as shown in Figure 6-38 before you open a connection to the
management module.
Microsoft (R) Windows 2000 (TM) Version 5.00 (Build 2195)
Welcome to Microsoft Telnet Client
Telnet Client Build 5.00.99206.1
Escape Character is 'CTRL+]'
Microsoft Telnet> unset crlf
Microsoft Telnet> open bcmm
Figure 6-38 Setting CRLF in Windows 2000 Telnet
Refer to the documentation of your client for other Telnet implementations.
132
6.8 System Management Services interface
This section explains how to access the SMS interface and how to use some of
the functions that it provides. The SMS is the preferred interface to the
BladeCenter JS21 firmware (BIOS) where you can configure initial and basic
hardware settings such as boot order, network boot parameters, and firmware
temporary versus permanent selection.
Note: The BladeCenter JS21 supports the SMS and Open Firmware
interface, but the BladeCenter JS20 supports only the Open Firmware
interface.
6.8.1 Activating the System Management Services interface
You can access the SMS interface whenever you power on a BladeCenter JS21.
Before you access the SMS interface, set up the SoL remote text console
function as described in 6.7, “Providing a console for the BladeCenter JS21” on
page 121.
Use the following procedure to access the SMS interface:
1. Using a Telnet or SSH client, connect to the management module external
Ethernet interface IP address.
2. When prompted, enter a valid user ID and password. The default
management module user ID is USERID, and the default password is
PASSW0RD, where the 0 is a zero. Consider changing the defaults in any
production environment.
3. Power cycle the blade and start an SoL console by using the power -cycle -c
command. For example, to power cycle and start an SoL remote text console
with the blade server in the first bay (bay 1), use the command:
power -cycle -c -T system:blade[1]
To open a console with a blade that is already powered on, use the
command:
console -T system:blade[1]
4. After approximately 30 seconds, you see a sequence of checkpoint codes
displayed on the console. These codes are generated by the Power On Self
Test (POST). The meaning of the checkpoint codes is documented in the
Problem Determination and Service Guide - IBM BladeCenter JS21 (Type
8844), which is on the CD that is shipped with your system. You can find the
latest update of the document on the Web site described in “BladeCenter
133
5. If you get the message To select this console as the active console
press 0, then press 0 (zero) to confirm the console.
6. When the POST menu and indicators are displayed as shown in Figure 6-39,
type 1 after the word Keyboard is displayed and before the word Speaker is
displayed.
...
CA00D003
CA00D004
CA00E139
CA00E1FB
CA00E100
CA00D008
CA00E1DC
1 = SMS Menu
5 = Default Boot
8 = Open Firmware Prompt
6 = Stored Boot List
List
Memory
Keyboard
Network
Figure 6-39 POST checkpoint codes: SMS
134
SCSI
Speaker
7. Type 1 and wait until the SMS menu is displayed as shown in Figure 6-40. If
you do not type any key when the POST menu and indicators are displayed,
the blade server proceeds to boot using the default boot sequence. To learn
how to configure the boot sequence, see 6.5, “Blade server configuration” on
page 90.
PowerPC Firmware
-------------------------------------------------------------------Main Menu
1.
Select Language
2.
Setup Remote IPL (Initial Program Load)
3.
Change SCSI Settings
4.
Select Console
5.
Select Boot Options
6.
Firmware Boot Side Options
7.
Progress Indicator History
-------------------------------------------------------------------Navigation Keys:
-------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-40 PowerPC Firmware: SMS menu
6.8.2 Setting up remote initial program load
This section describes how to set up a network boot that is necessary for a
network installation of an operating system. When you want to boot or install from
a local media (such as CD or DVD), perform the procedure as described in
“Booting from CD/DVD” on page 145. Ensure that you have the information in the
worksheet shown in Table 6-5 for your JS21 before you proceed with the
installation.
Table 6-5 Network configuration information worksheet
Network attribute
Value
Network interface
Host name
IP address
_______.________.________.________
135
Network attribute
Value
Network mask
_______.________.________.________
Name server
_______.________.________.________
Domain name
Gateway
_______.________.________.________
Depending on the operating system you want to install, refer to 8.3, “Preparing
AIX network installation using NIM” on page 261; 9.3, “Installing SLES using the
network” on page 290; or 9.4, “Installing Red Hat Enterprise Linux AS 4 Update 3
using the network” on page 321.
Broadcast Bootstrap Protocol versus directed Bootstrap
Protocol
You can use a broadcast Bootstrap Protocol (BOOTP) or directed BOOTP to
install a JS21 blade from a network installation server. Broadcast BOOTP
requires that you define the client’s hardware MAC address in the server’s
BOOTP or DHCP configuration and additionally that client and server are in the
same broadcast domain. However, a directed BOOTP does not require the
network installation server to be on the same subnet as the client. This option
also does not require that you define the MAC address of the client’s network
adapter on the server. To perform a directed BOOTP, you require an SoL
connection to the blade so that you can specify the IP parameters to SMS or
Open Firmware.
Attention: If your server uses DHCP instead of BOOTP, directed BOOTP
might not work. NIM on AIX uses BOOTP, although on Linux the preferred
method is DHCP. Our experience shows that directed BOOTP does not work
with a DHCP server.
136
Specifying IP parameters in System Management Services
For a network installation, perform these steps:
1. Select option 2 to set up the remote IPL.
2. The screen shown in Figure 6-41 opens. As described in 6.7, “Providing a
console for the BladeCenter JS21” on page 121, use the second port of the
blade server’s network interface when using the standard built-in Ethernet
interface. In this case, select the adapter with the location code “...-T8” by
entering 2.
PowerPC Firmware
--------------------------------------------------------------------------NIC Adapters
Device
Location Code
Hardware
Address
1. Port 1-IBM 2 PORT 1000 Base-SX U788D.001.23A1137-P1-T7 001125c90ba6
2. Port 2-IBM 2 PORT 1000 Base-SX U788D.001.23A1137-P1-T8 001125c90ba7
--------------------------------------------------------------------------Navigation keys:
--------------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-41 SMS remote IPL: NIC adapters
137
3. In the screen shown in Figure 6-42, type 1.
PowerPC Firmware
------------------------------------------------------------------Network Parameters
Port 2-IBM 2 PORT 1000 Base-SX PCI-X Adapter:
U788D.001.23A1137-P1-T8
1.
IP Parameters
2.
Adapter Configuration
3.
Ping Test
4.
Advanced Setup: BOOTP
-------------------------------------------------------------------Navigation keys:
X = eXit System Management
Services
------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:
Figure 6-42 SMS remote IPL: Network parameters
138
4. In the screen shown in Figure 6-43, enter the IP parameters.
PowerPC Firmware
-------------------------------------------------------------------IP Parameters
Port 2-IBM 2 PORT 1000 Base-SX PCI-X Adapter:
U788D.001.23A1137-P1-T8
1.
Client IP Address
[9.3.5.231]
2.
Server IP Address
[9.3.5.228]
3.
Gateway IP Address
[0.0.0.0]
4.
Subnet Mask
[255.255.255.000]
-------------------------------------------------------------------Navigation keys:
Services
Figure 6-43 SMS remote IPL: IP parameters
If you want, you can also check other menu items such as Adapter Configuration
in Figure 6-42 on page 138 and Spanning Tree Enabled, which we recommend
that you disable. Although this option was enabled in earlier firmware versions, it
is already disabled in the current firmware.
You do not have to change the speed and duplex settings for blade servers.
Note: You can always get back to the main menu by pressing M (without
pressing Enter).
139
6.8.3 Selecting boot options
In the main menu, type 5 for Select Boot Options. You can set the default Boot
Device Order and the Install/Boot Device as shown in Figure 6-44.
PowerPC Firmware
-------------------------------------------------------------------Multiboot
1.
Select Install/Boot Device
2.
Configure Boot Device Order
3.
Multiboot Startup <OFF>
-------------------------------------------------------------------Navigation keys:
Services
Figure 6-44 SMS Select Boot Options: Multiboot
140
Configuring boot device order
To Configure the boot device order:
1. Type 2 to display (option 6) or select a Boot Device (options 1- 5), as shown
in Figure 6-45. Type 1 to select the first boot device.
PowerPC Firmware
-------------------------------------------------------------------Configure Boot Device Order
1.
Select 1st Boot Device
2.
Select 2nd Boot Device
3.
Select 3rd Boot Device
4.
Select 4th Boot Device
5.
Select 5th Boot Device
6.
Display Current Setting
7.
Restore Default Setting
-------------------------------------------------------------------Navigation keys:
Services
Figure 6-45 SMS Select Boot Options: Configuring boot device order
141
2. The screen shown in Figure 6-46 opens. In this screen, specify 5 for hard
drive.
PowerPC Firmware
-------------------------------------------------------------------Select Device Type
1.
Diskette
2.
Tape
3.
CD/DVD
4.
IDE
5.
Hard Drive
6.
Network
7.
None
8.
List All Devices
-------------------------------------------------------------------Navigation keys:
Services
Figure 6-46 SMS Select Boot Options: Selecting device type
142
3. In the screen shown in Figure 6-47, select the first adapter with the location
code ending -T10.
PowerPC Firmware
-------------------------------------------------------------------Select Media Adapter
1.
U788D.001.23A1137-P1-T10
/pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@0
2.
U788D.001.23A1137-P1-T11
3.
U788D.001.23A1137-P1-T12-T1
/pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@ff
4.
None
5.
List all devices
-------------------------------------------------------------------Navigation keys:
Services
Figure 6-47 SMS Select Boot Options: Selecting media adapter
143
4. In the screen shown in Figure 6-48, select the device that you want.
check /pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@0/sd@1,0
PowerPC Firmware
-------------------------------------------------------------------Select Device
Device Current Device
Number Position Name
1.
SCSI 36401 MB Harddisk, part=1 ()
( loc=U788D.001.23A1137-P1-T10-L1-L0 )
2.
None
-------------------------------------------------------------------Navigation keys:
Services
Figure 6-48 SMS Select Boot Options: Selecting device hard drive SCSI
After you select the necessary disk, you can see the current boot device order
and go back to the main menu.
Attention: If the firmware does not detect the disk as bootable (that is, having
an operating system installed), it will not display the disks here. In particular,
new drives or drives that have been reformatted with the RAID functions as
described in 6.12, “SAS hardware RAID configuration” on page 170 cannot be
seen here. They first show up in the installation dialog of the corresponding
operating system or diagnostics.
144
Selecting install/boot device
This section shows you how to select the install/boot device.
Booting from CD/DVD
To boot from CD/DVD, perform these steps:
1. Select Select Boot Options → Select Install/Boot Device.
2. The menu shown in Figure 6-49 opens. Select 7. List all Devices.
PowerPC Firmware
-------------------------------------------------------------------Select Device Type
1.
Diskette
2.
Tape
3.
CD/DVD
4.
IDE
5.
Hard Drive
6.
Network
7.
List all Devices
-------------------------------------------------------------------Navigation keys:
Services
Figure 6-49 SMS Select Install/Boot Device: Selecting device type
Attention: To select CD/DVD in a BladeCenter environment, you must
select item 7. List all Devices and not items 3. CD/DVD or 4. IDE. This is
because the CD-ROM in the BladeCenter is a Universal Serial Bus (USB)
CD-ROM.
145
3. The screen in Figure 6-50 opens. This screen might look different on your
blade server depending on the hardware configuration. We can see two
interesting items in this screen:
– Because a Virtual I/O Server (VIOS) is installed on this JS21, the virtual
Ethernet adapters are also shown. However, the VIOS is not currently
running because we are in the SMS.
– More items can be displayed at once. Therefore, consider the option N =
Next page of list within the navigation keys.
In this screen, type N.
PowerPC Firmware
-------------------------------------------------------------------Select Device
1.
- Virtual Ethernet
( loc=U8844.51X.23A0248-V1-C3-T1 )
2.
Virtual Ethernet
( loc=U8844.51X.23A0248-V1-C4-T1 )
3.
Virtual Ethernet
( loc=U8844.51X.23A0248-V1-C5-T1 )
4.
Virtual Ethernet
( loc=U8844.51X.23A0248-V1-C6-T1 )
5.
Ethernet
( loc=U788D.001.23A0248-P1-T7 )
6.
Ethernet
( loc=U788D.001.23A0248-P1-T8 )
-------------------------------------------------------------------Navigation keys:
N = Next page of list
Services
Figure 6-50 SMS Select Install/Boot Device: Listing all devices
146
4. In the screen in Figure 6-51, select USB CD-ROM.
PowerPC Firmware
-------------------------------------------------------------------Select Device
7.
- USB CD-ROM
( loc=U788D.001.23A0248-P1-T1-L1-L3 )
8.
1
SCSI 36401 MB Harddisk, part=2 (AIX 5.3.0)
( loc=U788D.001.23A0248-P1-T10-L1-L0 )
-------------------------------------------------------------------Navigation keys:
P = Previous page of list
Services
Figure 6-51 SMS Select Install/Boot Device: USB CD-ROM
147
5. In the next screen (Figure 6-52), select Normal Mode Boot.
PowerPC Firmware
-------------------------------------------------------------------Select Task
USB CD-ROM
( loc=U788D.001.23A0248-P1-T1-L1-L3 )
1.
2.
3.
Information
Normal Mode Boot
Service Mode Boot
-------------------------------------------------------------------Navigation keys:
Services
------------------------------------------------------------------Type menu item number and press Enter or select Navigation key:2
Figure 6-52 SMS Select Install/Boot Device: CD-ROM Select Task
Proceed with “Initiating boot” on page 151.
148
Booting from network
To select the install/boot device, we provide an example of how to use a network
boot.
1. From the main menu, select Select Boot Options → Select Install/Boot
Device → Network.
2. The screen in Figure 6-53 opens. In this screen, you do not see any virtual
Ethernet interfaces, only physical ones. In other cases, you might see only
virtual Ethernet interfaces if this is a logical partition (LPAR). Select the
adapter with the location code ending with -T8 because the first adapter is
used for SoL.
PowerPC Firmware
-------------------------------------------------------------------Select Device
1.
Ethernet
( loc=U788D.001.23A1137-P1-T7 )
2.
Ethernet
( loc=U788D.001.23A1137-P1-T8 )
3.
None
-------------------------------------------------------------------Navigation keys:
Services
Figure 6-53 SMS Select Install/Boot Device: Network
149
3. In Figure 6-54, choose Normal Mode Boot.
PowerPC Firmware
-------------------------------------------------------------------Select Task
Ethernet
( loc=U788D.001.23A1137-P1-T8 )
1.
2.
3.
Information
Normal Mode Boot
Service Mode Boot
-------------------------------------------------------------------Navigation keys:
Services
Figure 6-54 SMS Select Install/Boot Device: Selecting the task
150
Initiating boot
To boot from your selected device, you must exit the SMS menu. To do this,
select 1 in Figure 6-55.
PowerPC Firmware
-------------------------------------------------------------------Are you sure you want to exit System Management Services?
1.
Yes
2.
No
-------------------------------------------------------------------Navigation Keys:
Figure 6-55 SMS Select Boot Options: Exiting SMS
The system tries to boot from the selected media. In the case of a network boot,
you see a screen similar to Figure 6-56. However, this requires that the network
boot is prepared as described in Chapter 8, “Installing AIX” on page 259 or
Chapter 9, “Installing Linux” on page 281.
STARTING SOFTWARE
PLEASE WAIT...
BOOTP:
BOOTP:
BOOTP:
BOOTP:
BOOTP:
BOOTP:
BOOTP:
BOOTP:
chosen-network-type = ethernet,auto,none,auto
server
IP =
9.3.5.228
requested filename =
client
IP =
9.3.5.231
client
HW addr =
0 11 25 c9 b a7
gateway IP =
0.0.0.0
device
loc-code U788D.001.23A1137-P1-T8
Elapsed time since release of system processors: 0 mins 48 secs
Figure 6-56 Starting software BOOTP
151
Other SMS options
For other SMS options such as Select Language, Change SCSI Settings, Select
Console, Firmware Boot Side Options, and Progress Indicator History, see
Installation and User's Guide - IBM BladeCenter JS21 (Type 8844). To access
these documents, follow the procedure described in “BladeCenter product
6.9 Open Firmware interface
In some situations, you might want to access the Open Firmware interface to the
BladeCenter JS21 firmware, either for debugging or because you are already
familiar with Open Firmware. This section explains how to access the Open
Firmware interface and how to use some of the functions they provide. Because
SMS is the recommended interface to use, we show only a few examples. You
can find further information about Open Firmware technology on the following
Web site:
http://www.firmworks.com
There is no guarantee that everything you find in this site will exactly match the
JS21’s Open Firmware implementation.
6.9.1 Activating the Open Firmware interface
You can access the Open Firmware interface whenever you power on a
BladeCenter JS21. Before you access the Open Firmware interface, set up the
SoL remote text console function as described in 6.7, “Providing a console for the
BladeCenter JS21” on page 121.
Use the following procedure to access the Open Firmware interface:
1. Using a Telnet or SSH client, connect to the management module external
Ethernet interface IP address.
2. When prompted, enter a valid user ID and password. The default
management module user ID is USERID, and the default password is
PASSW0RD, where 0 is a zero. Consider changing the defaults in any
production environment.
3. Power cycle the blade and start an SoL console by using the power -cycle -c
command. For example, to power cycle and start a SoL remote text console
with the blade server in the first bay (bay 1), use the command:
152
4. After approximately 30 seconds, you see a sequence of checkpoint codes
displayed on the console. These codes are generated by the Power On Self
Test (POST). The meaning of the checkpoint codes is documented in the
Problem Determination and Service Guide - IBM BladeCenter JS21 (Type
8844), which is on the CD that is shipped with your system. You can also find
the latest update of the document on the Web site described in “BladeCenter
5. If you get the message To select this console as the active console
press 0, then press 0 (zero) to confirm the console.
6. When the POST menu and indicators are displayed as shown in Figure 6-57,
type 8 after the word Keyboard is displayed and before the word Speaker is
displayed.
...
CA00D003
CA00D004
CA00E139
CA00E1FB
CA00E100
CA00D008
CA00E1DC
1 = SMS Menu 5 = Default Boot List
Memory
Keyboard
Network
SCSI
Speaker
Figure 6-57 POST checkpoint codes: Open Firmware
7. Type 8 and wait until the Open Firmware prompt is shown. If you do not type
any key when the POST menu and indicators are displayed, the blade server
proceeds to boot using the default boot sequence. To learn how to configure
the boot sequence, see 6.5, “Blade server configuration” on page 90.
In the following sections, we provide some examples of how to use the Open
Firmware interface.
153
Printenv command
The printenv [variable] command shows the values of variables. If you do not
specify any variable name, it shows all the configuration variables and their
current and default values. Figure 6-58 and Figure 6-59 on page 155 show the
output of the printenv command on a completely new BladeCenter JS21.
0 > printenv
---------- Partition: of-config -------- Signature: 0x50 ---------ibm,fw-dc-select
100
0
ibm,fw-default-mac-address? false
false
ibm,fw-forced-boot
ibm,fw-n-bc
255.255.255.255
255.255.255.255
ibm,fw-n-bretry
00
00
ibm,fw-n-tretry
00
00
ibm,fw-n-dbfp
00000000
00000000
ibm,fw-n-dafp
00000000
00000000
ibm,fw-n-rc
A
A
ibm,fw-n-ru
Y
Y
---------- Partition: common -------- Signature: 0x70 -------------little-endian?
false
false
real-mode?
true
true
auto-boot?
true
true
diag-switch?
false
false
fcode-debug?
false
false
oem-banner?
false
false
oem-logo?
false
false
use-nvramrc?
true
false
ibm,fw-tty-language
1
1
ibm,fw-new-mem-def
false
false
ibm,fw-prev-boot-vpd
00000000: 08 07 07 0c 09 ff 07 07 07 07 07
07 07 07 07 07 |................|
ibm,fw-keyboard
real-base
virt-base
real-size
virt-size
load-base
screen-#columns
screen-#rows
selftest-#megs
1
c00000
ffffffff
1000000
ffffffff
4000
64
28
0
Figure 6-58 Open Firmware printenv output: Page 1 of 2
154
1
c00000
ffffffff
1000000
ffffffff
4000
64
28
0
boot-device
boot-file
diag-device
diag-file
output-device
input-device
oem-banner
oem-logo
nvramrc
boot-command
reboot-command
menu?
ibm,fw-find-tape-alias
ibm,fw-find-cdrom-alias
ibm,dasd-spin-interval
security-#badlogins
security-mode
security-password
pw-status
ok
0 >
hdd0 network1 network cdrom hdd1
hdd0 network1 network cdrom hdd1
diag
diag
/vdevice/vty@30000000 com1
/vdevice/vty@30000000 com1
Defined : use NVEDIT related words to view
boot
boot
false
false
false
5
0
none
false
true
true
5
Figure 6-59 Open Firmware printenv output: Page 2 of 2
After you set the parameters for remote IPL and install AIX, you can see
additional entries as shown in Figure 6-60.
Note: When you have already configured a JS21 blade, you might find
additional entries in the output of the printenv command as shown in
Figure 6-60.
155
...
pw-status
bootinfo-aix
/pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@0/s
d@1,0:1
nu-001125c9113c
auto,auto,000.000.000.000,000.000.000.000,000.000.000.000,255.255.25
5.000,Standard,No,,5,512,5
ibm,fw-menu-001125c9113d
auto,auto,000.000.000.000,000.000.000.000,000.000.000.000,255.255.25
5.000,Standard,No,,5,512,5
ok
0 >
Figure 6-60 Open Firmware: Printenv configured output
Devalias command
In Open Firmware you can define aliases to specify devices in a more convenient
way instead of using full path names. Use the following command to set aliases
for the current session:
devalias alias device-path
Alternatively, use the following command to set them as persistent in the
NVRAM.
nvalias alias device-path
156
Figure 6-61 shows the aliases that we found on JS21 blade using the firmware
as stated in 6.6, “Firmware” on page 94.
Attention: Be aware that the aliases found in a completely new JS21 might
not be set up correctly already. You might not yet find the aliases net and
cdrom, and the aliases for hdd* and network* might not point to the correct
Peripheral Component Interconnect (PCI) adapter. Correct this if you define
the boot sequence as described in 6.5.2, “Setting the boot sequence” on
page 91.
0 > devalias
ibm,sp
hdd1
hdd0
disk
network
net
network1
scsi
cdrom
nvram
rtc
screen
ok
0 >
/vdevice/IBM,sp@4000
/pci@8000000f8000000/pci@1/pci1014,028C@1/scsi@1/sd@1,0
/pci@8000000f8000000/pci@7/usb@0/hub@1/hub@1/cdrom@3
/vdevice/nvram@4002
/vdevice/rtc@4001
/vdevice/vty@30000000
Figure 6-61 Open Firmware: Devalias configured output
Ls command
Use the ls command to display the names of the current node’s children as
shown in Figure 6-62 on page 158 and Figure 6-63 on page 159. The output
might be useful if you do not have an alias already and want to specify a device
or define an alias. It is also useful to check whether the aliases that are already
defined (see “Devalias command” on page 156) are correct. Be aware that the
output is actually a tree example. Therefore, the second Ethernet adapter is:
This is the adapter referred to by the network1 alias in Figure 6-61.
157
0 > ls
000000c887d0:
000000c89658:
000000c89898:
000000c89990:
000000c8fd68:
000000cd0288:
000000cd0a48:
000000cd60f0:
000000cd7190:
000000cdbc10:
000000cdbef8:
000000ceab88:
000000cf0cd8:
000000cf14f8:
000000cf3870:
000000cfd0f8:
000000d16518:
000000d1c060:
000000d1d420:
000000d31578:
000000d31758:
000000d32008:
000000d357f0:
000000d36568:
000000d36660:
000000de6ba0:
000000e02340:
000000e61338:
000000ca25a0:
000000ca6a90:
000000ca7af8:
000000ca81b0:
000000ca8dd8:
000000ca99c8:
000000ca9df8:
000000caa9e8:
000000caae18:
000000cb0788:
000000cb0968:
000000cb0b48:
000000cb0d28:
/ibm,serial
/chosen
/packages
/disassembler
/assembler
/dev-tree
/lpevents
/deblocker
/disk-label
/tape-label
/obp-tftp
/ip
/prep-boot
/fat-files
/iso-13346-files
/utilities
/net
/iso-9660-files
/boot-mgr
/chrp-loader
/pe-loader
/elf-loader
/nls-support
/terminal-emulator
/dynamic-reconfig
/gui
/iscsi
/post
/cpus
/PowerPC,POWER@0
/l2-cache@2000
/PowerPC,POWER@1
/PowerPC,POWER@2
/l2-cache@2001
/PowerPC,POWER@3
/l2-cache@2002
/memory@0
/memory@8000000
/memory@9000000
/memory@a000000
/memory@b000000 ...
Figure 6-62 Open Firmware ls: Page 1 of 2
158
000000ccc5c8:
000000ccc7a8:
000000cccdc8:
000000ccdd08:
000000d36b28:
000000d38998:
000000d3a4c8:
000000d3a630:
000000d3be60:
000000d3dc08:
000000d43ef0:
000000d46f38:
000000d47ea8:
000000d49188:
000000d4a220:
000000d4ab10:
000000d4b148:
000000d4d5f0:
000000d55088:
000000d6e6d0:
000000d785e0:
000000d7ac40:
000000d7c660:
000000d7e1a8:
000000d80808:
000000d82228:
000000d83d70:
000000d863d0:
000000d87df0:
000000d5bd30:
000000d89a10:
000000d990b0:
000000d629b8:
000000da8750:
000000dc4c68:
000000dc5f58:
000000de2470:
000000d6c6c8:
ok
0 >
/memory@f6000000
/memory@f7000000
/options
/aliases
/openprom
/event-sources
/epow-events
/interrupt-controller@0
/interrupt-controller@f8000400
/rtas
/vdevice
/vty@30000000
/vty@30000001
/IBM,sp@4000
/rtc@4001
/nvram@4002
/tpm@100f4003000
/pci@8000000f8000000
/pci@1
/pci1014,028C@1
/scsi@0
/sd
/st
/scsi@1
/sd
/st
/scsi@ff
/sd
/st
/pci@2
/ethernet@4
/ethernet@4,1
/pci@7
/usb@0
/hub@1
/usb@0,1
/hub@1
/tnk-memory-map
Figure 6-63 Open Firmware ls: Page 2 of 2
159
6.9.2 Specifying IP parameters to Open Firmware
You have to specify the IP parameters only if you want to use a directed BOOTP
as explained in “Broadcast Bootstrap Protocol versus directed Bootstrap
Protocol” on page 136.
Specifying a directed BOOTP from the JS21
Typically, you can initiate a directed BOOTP in Open Firmware by typing:
boot net:bootp,server_ip,,client_ip,gateway_ip
Here server_ip, client_ip and gateway_ip are the IP addresses of the network
installation server, the client, and an optional gateway (note the double comma).
If there is no gateway, use 0.0.0.0 for the gateway_ip address.
Because SoL uses ent0, use ent1 as explained in 6.7, “Providing a console for
the BladeCenter JS21” on page 121. First you have to check whether an alias net
is defined using the devalias command, as shown in “Devalias command” on
page 156. Figure 6-61 on page 157 shows that the alias net points to the first
Ethernet port which is ent0. Therefore, you can use the ls command (as shown
in “Ls command” on page 157) to determine the full path name of the second
Ethernet port and run a command similar to:
boot
/pci@8000000f8000000/pci@2/ethernet@4,1:bootp,9.3.4.9,,9.3.5.3,9.3.5.1
Alternatively, you can use the alias network1 (as shown in Figure 6-61 on
page 157). This alias refers to /pci@8000000f8000000/pci@2/ethernet@4,1:
boot network1:bootp,9.3.4.9,,9.3.5.3,9.3.5.1
160
Note: You must specify the full device path name with this command. To
determine the full path to your device, list the device tree by running the ls
command at the Open Firmware prompt. This command displays an output
similar to Figure 6-64. The highlighted items are the path to the second
Ethernet adapter. Pass this information to the boot command to initiate a
network boot from the second Ethernet adapter.
0 > ls
000000c887d0:
000000c89658:
000000c89898:
...
000000d4b148:
000000d4d5f0:
000000d55088:
000000d6e6d0:
000000d785e0:
000000d7ac40:
000000d7c660:
000000d7e1a8:
000000d80808:
000000d82228:
000000d83d70:
000000d863d0:
000000d87df0:
000000d5bd30:
000000d89a10:
000000d990b0:
000000d629b8:
/ibm,serial
/chosen
/packages
/tpm@100f4003000
/pci@8000000f8000000
/pci@1
/pci1014,028C@1
/scsi@0
/sd
/st
/scsi@1
/sd
/st
/scsi@ff
/sd
/st
/pci@2
/ethernet@4
/ethernet@4,1
/pci@7
Figure 6-64 ls command output
After you run the boot command, the network installation begins and you see an
output similar to Figure 6-56 on page 151.
161
6.10 Blade server Ethernet controller enumeration
The Standard (73P9030) and SFF (26K4842) Gigabit Ethernet expansion cards
increase the number of Gigabit Ethernet network interfaces on the BladeCenter
JS21 from two to four.
The enumeration of the Ethernet controllers in a blade server is operating system
dependent. You must number the two interfaces (ports) consecutively on the
controller itself with a lower number for the primary and a higher number for the
secondary interface. Because of the changes in the hardware layout, the
enumeration of the Ethernet controllers on the JS21 is different from the JS20.
Attention: If you install an Ethernet expansion card (daughter card) in a JS21
blade server, the expansion card might be configured first, with the logical
names eth0 and eth1, and the integrated Ethernet interfaces get the names
eth2 and eth3.
On AIX, this happens if you have already installed the Ethernet expansion card
before installing AIX. However, when you install the Ethernet expansion after
installing AIX, the names eth0 and eth1 remain on the integrated Ethernet
interfaces because AIX has saved this information in the Object Data Manager
(ODM).
If this is an issue, you must first install the operating system to allow the onboard
ports to be recognized and configured before the ports on the expansion card. If
you install the Ethernet expansion card before you install the operating system,
be aware that the expansion card ports are assigned before the onboard ports.
Changing Ethernet controller enumeration on AIX
For AIX, there are other ways to change the Ethernet controller enumeration.
Example 6-5 on page 163 shows one of these methods. This method changes
the ODM. Example 6-6 on page 167 shows a method that deletes the device first
and recreates it with the name that you want.
Example: Changing the Object Data Manager
Example 6-5 shows you how to change the name of an Ethernet interface in the
ODM. The main functions of the code are:
Changing devices to defined state because later we do not want to add a
drive into the ODM, which is not already there but has the available state
Retrieving all related configuration data from the customized ODM and saving
it in a text file. For Ethernet interfaces, we also require enX and etX besides
the entX.
162
Changing the name in the text file using standard string manipulation (we use
sed)
Removing the old device
Adding the changed text file’s data back to the ODM
Configuring the new device to the available state
Example 6-5 Changing Ethernet enumeration using ODM
#!/bin/ksh
Abort ()
{ echo "Error: $*" 1>&2 ; exit 1; }
if [ $# -ne 2 ] ; then
echo "Usage: $0 entX entY ; where entX & entY are old & new adapter
names"
exit 1
fi
if [ "ìd -un`" != "root" ] ; then
Abort "$0 must be run as root"
fi
case "$1" in
ent[0-9]*) entX="$1" ;;
*) Abort "ERROR: entX format is incorrect" ;;
esac
case "$2" in
ent[0-9]*) entY="$2";;
*) Abort "ERROR: entY format is incorrect" ;;
esac
# remove "ent" to get number only
idX=${entX##ent}
idY=${entY##ent}
# get correspondin en and et device name
enX=en$idX
etX=et$idX
enY=en$idY
etY=et$idY
163
# check if entX is present
lsattr -El $entX >/dev/null 2>&1
if [ $? -ne 0 ] ; then
Abort "\"$entX\" does not exist"
fi
# while entY must not exist yet
lsattr -El $entY >/dev/null 2>&1
if [ $? -eq 0 ] ; then
Abort "\"$entY\" exists already"
fi
echo "Changing $entX $en and $et to Defined status"
for i in $enX $etX $entX
do
rmdev -l $i
if [ $? -ne 0 ] ; then
Abort "\"rmdev -l $i\" FAILED"
fi
done
echo "Gathering ODM data for $entX"
odmget -q name="$entX" CuDv >/tmp/$entX
if [ $? -ne 0 ] ; then
Abort "\"odmget -q name=$entX CuDv\" FAILED"
fi
odmget -q name="$entX" CuAt >>/tmp/$entX
if [ $? -ne 0 ] ; then
Abort "\"odmget -q name=$entX CuAt\" FAILED"
fi
odmget -q value3="$entX" CuDvDr >>/tmp/$entX
if [ $? -ne 0 ] ; then
Abort "\"odmget -q value3=$entX CuDvDr\" FAILED"
fi
odmget -q name="$entX" CuVPD >>/tmp/$entX
if [ $? -ne 0 ] ; then
Abort "\"odmget -q name=$entX CuVPD\" FAILED"
fi
odmget -q name="$enX" CuDv >>/tmp/$entX 2>/dev/null
if [ $? -ne 0 ] ; then
164
Abort "\"odmget -q name=$enX CuDv\" FAILED"
fi
odmget -q name="$enX" CuAt >>/tmp/$entX 2>/dev/null
if [ $? -ne 0 ] ; then
Abort "\"odmget -q name=$enX CuAt\" FAILED"
fi
odmget -q name="$etX" CuDv >>/tmp/$entX 2>/dev/null
if [ $? -ne 0 ] ; then
Abort "\"odmget -q name=$et CuDv\" FAILED"
fi
odmget -q name="$et" CuAt >>/tmp/$entX 2>/dev/null
if [ $? -ne 0 ] ; then
Abort "\"odmget -q name=$et CuAt\" FAILED"
fi
echo "Preparing ODM directives to change $entX to $entY"
sed "s/= \"$entX\"/= \"$entY\"/
s/= \"$enX\"/= \"$enY\"/
s/= \"$etX\"/= \"$etY\"/g" /tmp/$entX >/tmp/$entY
if [ $? -ne 0 ] ; then
Abort "\"sed\" of ODM directives FAILED"
fi
echo "Removing $entX and deleting definition"
for i in $enX $etX $entX
do
rmdev -dl $i
if [ $? -ne 0 ] ; then
Abort "\"rmdev -dl $i\" FAILED"
fi
done
echo "Critical point - Modifying ODM to add $entY"
odmadd /tmp/$entY
if [ $? -ne 0 ] ; then
Abort "\"odmadd /tmp/$entY\" FAILED"
fi
165
echo "Remaking device $entY"
mkdev -l $entY
if [ $? -ne 0 ] ; then
Abort "\"mkdev -l $entY\" FAILED"
fi
echo "Done"
Example: Using the device method
The second way to change the Ethernet device name is to use the device
method. The advantage is that you do not have to directly manipulate the ODM.
The disadvantage is that the device information is deleted and all the settings
such as IP address are lost. But this does not cause any harm because we
typically perform this operation on a newly installed machine.
Use the following command to manually create an Ethernet interface and specify
the name that you want:
/usr/lib/methods/define_rspc -c adapter -s pci -t e414a816 -p pci1 -w 8
-d -L '01-08' -l ent5
In this example, we use ent5 as the required name.
Note: This is not an officially supported AIX command. Therefore, it is not
documented and its availability or function might change in the future.
The main problem is to get the necessary parameters, which depend on the
hardware of the blade server and the type of expansion card. Use the following
command to extract the information from the ODM, when the interface has to be
configured.
odmget -q name=ent0 CuDv
166
Figure 6-65 shows that the class (-c flag), subclass (-s), and the type (-t) come
from the PdDvLn entry. The parent (-p), location (-L), and connection
(connwhere, -w) have separate entries.
CuDv:
name = "ent0"
status = 1
chgstatus = 0
ddins = "pci/bentdd"
location = "01-08"
parent = "pci1"
connwhere = "8"
PdDvLn = "adapter/pci/e414a816"
Figure 6-65 Ethernet controller ODM data
The actual task shown in Example 6-6 performs the following functions:
Extract ODM data
Remove the enX, etX, and entX interface
Create the new entY interface with the name that you want
Run cfgmgr to change the entY from defined to available state and to create the
corresponding interfaces enY and etY.
Example 6-6 Changing Ethernet enumeration using the device method
#!/bin/ksh
Abort ()
{ echo "Error: $*" 1>&2 ; exit 1; }
if [ $# -ne 2 ] ; then
echo "Usage: $0 entX entY ; where entX & entY are old & new adapter
names"
exit 1
fi
if [ "ìd -un`" != "root" ] ; then
Abort "$0 must be run as root"
fi
case "$1" in
ent[0-9]*) entX="$1" ;;
*) Abort "ERROR: entX format is incorrect" ;;
esac
167
case "$2" in
ent[0-9]*) entY="$2";;
*) Abort "ERROR: entY format is incorrect" ;;
esac
idX=${entX##ent}
idY=${entY##ent}
lsattr -El $entX >/dev/null 2>&1
if [ $? -ne 0 ] ; then
Abort "\"$entX\" does not exist"
fi
lsattr -El $entY >/dev/null 2>&1
if [ $? -eq 0 ] ; then
Abort "\"$entY\" already not exist"
fi
odmget -q name=$entX CuDv | awk -v IdX=$idX -v IdY=$idY '
{ gsub("\"","") }
$1
$1
$1
$1
==
==
==
==
"PdDvLn" { split($3,Pd,"/") }
"location" { Loc=$3 }
"connwhere" { Conn=$3 }
"parent"
{ Parent=$3 }
END { args="-c " Pd[1] " -s " Pd[2] " -t " Pd[3] " -p "
args=args " -w " Conn " -d -L " Loc " -l ent" IdY
system ("rmdev -dl en" IdX )
system ("rmdev -dl et" IdX )
system ("rmdev -dl ent" IdX )
system ("/usr/lib/methods/define_rspc " args)
print ""
system ("cfgmgr")
}
'
168
Parent
6.11 Blade server I/O module routing
The routing of an Ethernet controller to a particular I/O module bay depends on
the type of blade server and the type of the expansion card. For the JS21, the
routing is as shown in Table 6-6.
Table 6-6 BladeCenter I/O routing
Controller
Interface
I/O module bay
Integrated Ethernet
Interface 1
1
Interface 2
2
Interface 1
3
Interface 2
4
Interface 1
4
Interface 2
3
Expansion card
(except 73P9030)
Standard (73P9030) Gigabit Ethernet
expansion card
You can verify which Ethernet controller is routed to which I/O module bay by
using the following test:
1. Install only one Ethernet switch module or pass-thru module in I/O module
bay 1.
2. Ensure that the ports on the switch module or pass-thru module are enabled
(click I/O Module Tasks → Admin/Power/Restart in the management
module Web interface).
3. Enable only one of the Ethernet controllers on the blade server. Note the
designation that the blade server operating system has for the controller.
4. Use ping in an external computer in the network that is connected to the
switch module or pass-thru module. If you can use ping in the external
computer, the Ethernet controller that you enabled is associated with the
switch module or pass-thru module in I/O-module bay 1. The other Ethernet
controller in the blade server is associated with the switch module or
pass-thru module in I/O-module bay 2.
If you have installed an I/O expansion card in the blade server, communications
from the expansion card are routed to I/O module bay 3 and module bay 4, if
these bays are supported by your BladeCenter unit. You can verify which
controller on the card is routed to which I/O module bay by performing the same
test, and using a controller on the expansion card and a compatible switch
module or pass-thru module in I/O module bay 3 or module bay 4.
169
6.12 SAS hardware RAID configuration
The storage subsystem on the BladeCenter JS21 is based on the 2-port SAS
controller attached to the PCI-X bus running at 100 MHz. The subsystem
provides RAID 0 or RAID 1 mirroring functionality.
As RAID is considered an option, the disks are shipped blank, ready for use as
individual disk drives. If you want to configure RAID for any onboard drives, run
the RAID configuration tools first to prepare the disks to be used in a RAID.
6.12.1 RAID configuration methods
The RAID configuration occurs within the AIX 5L or Linux operating system.
However, you cannot perform this function from an operating system running on
these disks. Because not more than two internal drives are available in a JS21,
boot the operating system from CD/DVD, network, or an external disk. A typical
method to configure a RAID array is to boot the AIX stand-alone diagnostics from
the CD/DVD or network (use the NIM operation diag = enable a machine to boot
a diagnostic image), and configure the RAID from there. This function is also
available when booting into rescue mode on the SUSE Linux Enterprise Server 9
(SLES9) Service Pack 3 (SP3) installation CD.
Before you can create a RAID array, you must reformat the hard disk drives so
that the sector size of the drives changes from 512 bytes to 522 bytes. Later if
you decide to remove the hard disk drives, delete the RAID array before you
remove the drives. If you decide to delete the RAID array and reuse the hard disk
drives, you must reformat the drives so that the sector size of the drives changes
from 522 bytes to 512 bytes.
170
6.12.2 Configuring RAID using the AIX stand-alone diagnostics
This section provides an example of how to configure a RAID using the AIX
stand-alone diagnostics.
1. After diagnostics starts, select 3 Task Selection in the Function Selection
menu, as shown in Figure 6-66.
FUNCTION SELECTION
1 Diagnostic Routines
other advanced functions will not be used.
2 Advanced Diagnostics Routines
other advanced functions will be used.
3 Task Selection (Diagnostics, Advanced Diagnostics, Service Aids, etc.)
This selection will list the tasks supported by these procedures.
Once a task is selected, a resource menu may be presented showing
all resources supported by the task.
4 Resource Selection
This selection will list the resources in the system that are supported
by these procedures. Once a resource is selected, a task menu will
be presented showing all tasks that can be run on the resource(s).
99 Exit Diagnostics
NOTE:
The terminal is not properly initialized. You will be prompted to
initialize the terminal after selecting one of the above options.
To make a selection, type the number and press Enter.
[1 ]
Figure 6-66 RAID diag: Selecting the function
171
2. Define the terminal type and scroll down the Tasks Selection menu
(Figure 6-67) and select Raid Array Manager.
TASKS SELECTION LIST
801004
From the list below, select a task by moving the cursor to
the task and pressing 'Enter'.
To list the resources for the task highlighted, press 'List'.
[MORE...17]
Display USB Devices
Display or Change Bootlist
Format Media
Hot Plug Task
Identify and Attention Indicators
Local Area Network Analyzer
Microcode Tasks
RAID Array Manager
SSA Service Aids
This selection provides tools for diagnosing and resolving
problems on SSA attached devices.
[BOTTOM]
F1=Help
F3=Previous Menu
F4=List
F10=Exit
Enter
Figure 6-67 RAID diag: Selecting the task
3. Select PCI-X SCSI Disk Array Manager, as shown in Figure 6-68.
RAID Array Manager
PCI SCSI Disk Array Manager
PCI-X SCSI Disk Array Manager
Figure 6-68 Selecting the RAID Array Manager
172
801004
4. From the Disk Array Manager screen (Figure 6-69), select List PCI-X SCSI
Disk Array Configuration.
List PCI-X SCSI Disk Array Configuration
Create an Array Candidate pdisk and Format to 522 Byte Sectors
Create a PCI-X SCSI Disk Array
Delete a PCI-X SCSI Disk Array
Add Disks to an Existing PCI-X SCSI Disk Array
Configure a Defined PCI-X SCSI Disk Array
Change/Show Characteristics of a PCI-X SCSI Disk Array
Reconstruct a PCI-X SCSI Disk Array
Change/Show PCI-X SCSI pdisk Status
Diagnostics and Recovery Options
F1=Help
F9=Shell
F2=Refresh
F10=Exit
F3=Cancel
Enter=Do
F8=Image
Figure 6-69 Disk Array Manager: Main dialog
173
If you have not yet configured a RAID, the output looks similar to Figure 6-70.
COMMAND STATUS
Command: OK
stdout: yes
stderr: no
Before command completion, additional instructions may appear below.
---------------------------------------------------------------------------Name
Location
State
Description
Size
---------------------------------------------------------------------------sisioa0
01-08
Available
PCI-XDDR Dual Channel SAS RAID Adapter
hdisk0
hdisk1
01-08-00-1,0
01-08-01-1,0
F1=Help
F8=Image
n=Find Next
Available
Available
F2=Refresh
F9=Shell
SCSI Disk Drive
SCSI Disk Drive
36.4GB
36.4GB
F3=Cancel
F10=Exit
Figure 6-70 List PCI-X SCSI disk array configuration: No array candidates
174
F6=Command
/=Find
5. Create array candidates from the native hdisks. Select the adapter as shown
in Figure 6-71.
????????????????????????????????????????????????????????????????????????
?
Available Controllers
?
?
?
? Move cursor to desired item and press F7.
?
?
ONE OR MORE items can be selected.
?
? Press Enter AFTER making all selections.
?
?
?
?
sisioa0 Available 01-08 PCI-XDDR Dual Channel SAS RAID Adapter
?
?
?
? F1=Help
F2=Refresh
F3=Cancel
?
? F7=Select
F8=Image
F10=Exit
?
F1? Enter=Do
/=Find
n=Find Next
?
F9????????????????????????????????????????????????????????????????????????
Figure 6-71 RAID diag: Available controllers
175
6. Select the disks as shown in Figure 6-72.
????????????????????????????????????????????????????????????????????????
?
Create an Array Candidate pdisk and Format to 522 Byte Sectors ?
?
?
? Move cursor to desired item and press F7. Use arrow keys to scroll. ?
?
?
?
?
?
? > hdisk0
01-08-00-1,0 Available
SCSI Disk Drive
36.4GB
?
? > hdisk1
01-08-01-1,0 Available
SCSI Disk Drive
36.4GB
?
?
?
? F1=Help
F2=Refresh
F3=Cancel
?
? F7=Select
F8=Image
F10=Exit
?
F1? Enter=Do
/=Find
n=Find Next
?
F9????????????????????????????????????????????????????????????????????????
Figure 6-72 Creating an array candidate pdisk and formatting to 522-byte sectors
The reformat to 522-byte sectors starts as shown in Figure 6-73. This takes
some time to complete.
Format in progress
|#################################################-| | 98%
hdisk1 deleted
|##################################################| / 99%
hdisk0 deleted
|##################################################| - 100%
Formats complete.
Press the Enter key to continue ...
Figure 6-73 RAID diag format in progress
176
7. Now when you perform a List PCI-X SCSI Disk Array Configuration in the
main screen (Figure 6-69 on page 173), you get the output shown in
Figure 6-74. Compared to Figure 6-70 on page 174, the disk names have
changed from hdisks to pdisks and the description is now Array Candidate.
COMMAND STATUS
Command: OK
stdout: yes
stderr: no
-------------------------------------------------------------------Name
Location
State
Description
Size
--------------------------------------------------------------------sisioa0
01-08
Available
pdisk1
pdisk0
01-08-01-1,0
01-08-00-1,0
F1=Help
F8=Image
n=Find Next
Active
Active
F2=Refresh
F9=Shell
Array Candidate
Array Candidate
F3=Cancel
F10=Exit
34.8GB Zeroed
34.8GB Zeroed
F6=Command
/=Find
Figure 6-74 List PCI-X SCSI disk array configuration: With array candidates (pdisks)
177
8. You can create the array by selecting Create a PCI-X SCSI Disk Array in the
main screen. Figure 6-75 shows that first you have to select the RAID level. In
this example, we use RAID level 0.
????????????????????????????????????????????????????????????????????????
?
Select a Raid Level
?
?
?
? Move cursor to desired item and press Enter.
?
?
?
?
0
?
?
5
?
?
10
?
?
?
? F1=Help
F2=Refresh
F3=Cancel
?
? F8=Image
F10=Exit
Enter=Do
?
F1? /=Find
n=Find Next
?
F9????????????????????????????????????????????????????????????????????????
Figure 6-75 Creating the array: Selecting the RAID level
Restriction: Figure 6-75 shows that the Disk Array Manager supports
RAID levels 0, 5, and 10. The JS21 has only two physical drives. RAID
level 5 requires a minimum of three drives. RAID level 10 requires a
minimum of three drives. RAID level 0 is striping, which you can do with
two disks. When you select RAID level 10 with two disks, we suspect that
the system is performing RAID 1, which is mirroring with two disks.
178
9. Select the Stripe Size as shown in Figure 6-76.
????????????????????????????????????????????????????????????????????????
?
Select a Stripe Size (in Kb)
?
?
?
?
?
?
?
16 Kb
?
?
64 Kb (recommended)
?
?
256 Kb
?
?
?
? F1=Help
F2=Refresh
F3=Cancel
?
? F8=Image
F10=Exit
Enter=Do
?
F1? /=Find
n=Find Next
?
F9????????????????????????????????????????????????????????????????????????
Figure 6-76 Creating the array: Selecting the stripe size
179
10.Select the Array Candidates (pdisks) that you want to use, as shown in
Figure 6-77.
????????????????????????????????????????????????????????????????????????
?
Select Disks to Use in the Array
?
?
?
? Move cursor to desired item and press F7. Use arrow keys to scroll. ?
?
?
?
?
?
?
# RAID 0 supports a minimum of 1 and a maximum of 18 drives.
?
?
?
?
?
? > pdisk0
01-08-00-1,0 Active
Array Candidate
34.8GB
?
? > pdisk1
01-08-01-1,0 Active
Array Candidate
34.8GB
?
?
?
? F1=Help
F2=Refresh
F3=Cancel
?
? F7=Select
F8=Image
F10=Exit
?
F1? Enter=Do
/=Find
n=Find Next
?
F9????????????????????????????????????????????????????????????????????????
Figure 6-77 Creating the array: Selecting the pdisks
180
The system shows a final dialog with the chosen settings (Figure 6-78).
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
sisioa0
0
64
pdisk0 pdisk1
Controller
RAID Level
Stripe Size in KB
Selected Disks
F1=Help
Esc+5=Reset
F9=Shell
F2=Refresh
F6=Command
F10=Exit
F3=Cancel
F7=Edit
Enter=Do
F4=List
F8=Image
Figure 6-78 Creating the array: Final dialog
Figure 6-79 shows the successful creation of the array ().
COMMAND STATUS
Command: OK
stdout: yes
stderr: no
hdisk0 Available
F1=Help
F8=Image
n=Find Next
F2=Refresh
F9=Shell
F3=Cancel
F10=Exit
F6=Command
/=Find
Figure 6-79 Creating the array: hdisk created
181
11.When you select the List PCI-X SCSI Disk Array Configuration again in the
main dialog, you get an output similar to Figure 6-80. You can see the RAID
array (hdisk0) and also the corresponding array members (pdisks).
COMMAND STATUS
Command: OK
stdout: yes
stderr: no
-----------------------------------------------------------------------Name
Location
State
Description
Size
-----------------------------------------------------------------------sisioa0
01-08
Available
hdisk0
pdisk0
pdisk1
01-08-ff-0,0
01-08-00-1,0
01-08-01-1,0
F1=Help
F8=Image
n=Find Next
Optimal
Active
Active
RAID 0 Array
Array Member
Array Member
F2=Refresh
F9=Shell
69.7GB
34.8GB
34.8GB
F3=Cancel
F10=Exit
F6=Command
/=Find
Figure 6-80 List PCI-X SCSI disk array configuration: RAID 0 array
Example 6-7 shows how the RAID array disk is displayed by different disk-related
commands in AIX 5L.
Example 6-7 Commands output for RAID disk
# lsdev -Ccdisk
hdisk0 Available 01-08-ff-0,0 SCSI RAID 0 Disk Array
# lscfg -vp
hdisk0
U788D.001.23A1137-P1-T12-T1-L0-L0
# lsattr -El hdisk0
pvid
000004718f5fc27f0000000000000000
queue_depth
8
raid_level
0
serial_number B5FA7003
182
SCSI RAID 0 Disk Array
Physical volume identifier False
Queue DEPTH
False
RAID Level
False
Serial Number
False
sis_unique_id 1D08B5FA700303IBM0885CB1410scsi
size_in_mb
69797
stripe_size
64
Unique device identifier
Size in Megabytes
Stripe Size in KB
False
False
False
Figure 6-81 shows how the RAID array disk shows up in the SMS.
PowerPC Firmware
----------------------------------------------------------------Select Device
1.
1
SCSI 69793 MB Harddisk, part=2 (AIX 5.3.0)
( loc=U788D.001.23A1137-P1-T12-T1-L0-L0 )
Figure 6-81 RAID array in SMS
6.12.3 Configuring RAID using iprconfig on Linux
On Linux, you can use the iprconfig tool to configure the RAID. In this example,
we use SLES9 from a CD-ROM drive.
Attention: The drivers for the Small Computer System Interface (SCSI)
adapter are on the SLES SP3 CD-ROM. You must boot from CD 1 of the SP
CD-ROMs or a network location of SLES where SP3 is installed. If you boot
from the SLES9 CD 1, you will get a message that no hard drives are found.
1. Insert CD-ROM 1 of the SLES9 SP3 into the CD-ROM drive.
2. Ensure that the BladeCenter media tray is connected to the JS21 that you will
configure. You can do this in the Web interface of the management module or
by pressing the appropriate button in front of the blade.
3. Within the management module, change the boot order so that the CD-ROM
is the first bootable device.
4. Start a Telnet session to the management module.
5. Start an SoL session using the following command (in this example, we use
blade 3)
183
6. When the CD-ROM is ready to load the kernel, you see the screen shown in
Figure 6-82. Type the word install to start the process.
Welcome to SuSE Linux (SLES9)!
Use
Use
"install"
"install32"
to boot the pSeries 64bit kernel
to boot the 32bit RS/6000 kernel
You can pass the option "noinitrd" to skip the installer.
Example: install noinitrd root=/dev/sda4
Welcome to yaboot version 1.3.11.SuSE
Enter "help" to get some basic usage information
boot: install
Figure 6-82 Booting SLES9 SP3 CD-ROM 1
7. Wait for the screen shown in Figure 6-83. Type 1 and press Enter to continue.
>>> SUSE LINUX Enterprise Server 9 installation program v1.6.53 (c)
1996-2004 SUSE LINUX AG <<<
Starting hardware detection...
Activating usb devices... done
Searching for info file...
Driver Update: Service Pack 3
Make sure that CD number 1 is in your drive.
1) OK
2) Back
> 1
Figure 6-83 SLES9 SP3 booting
184
8. The screen shown in Figure 6-84 opens. Select the language in which you
want to see the screens.
Select the language.
1)
2)
3)
4)
5)
6)
7)
8)
9)
10)
11)
12)
13)
14)
15)
16)
17)
18)
Bosnia
Cestina
Deutsch
English
Español
Français
Hellenic
Italiano
Japanese
Magyar
Nederlands
Polski
Português
Português Brasileiro
Russian
Slovencina
Simplified Chinese
Traditional Chinese
> 4
Figure 6-84 Choosing the installation language
185
9. You are now ready to start the process as shown in Figure 6-85. Type 4 to
start the installation.
>>> Linuxrc v1.6 (Kernel 2.6.5-7.244-pseries64) (c) 1996-2004 SUSE
LINUX AG <<<
Main Menu
1)
2)
3)
4)
5)
6)
7)
Settings
System Information
Kernel Modules (Hardware Drivers)
Start Installation or System
Eject CD
Exit or Reboot
Power off
> 4
Figure 6-85 Starting the installation process
10.You go into the rescue mode as shown in Figure 6-86. Type 3 to start the
rescue system.
1) Start Installation or Update
2) Boot Installed System
3) Start Rescue System
> 3
Figure 6-86 Starting the rescue mode
186
11.You are prompted for the source medium as shown in Figure 6-87. Choose 1
to start the rescue system from the CD-ROM.
Choose the source medium.
1)
2)
3)
4)
CD-ROM
Network
Hard Disk
Floppy
> 1
Figure 6-87 Choosing the source of the SUSE installation
12.Log in as root and run the iprconfig command, as shown in Figure 6-88.
INIT: Entering runlevel: 3
Boot logging started on /dev/hvc0(/dev/console) at Wed Jun 21
19:42:01 2006
Master Resource Control: previous runlevel: N, switching to
runlevel:3
Initializing random number generator done
Starting syslog services done
Starting RPC portmap daemon done
Importing Net File System (NFS) unused
Master Resource Control: runlevel 3 has been reached
Skipped services in runlevel 3: nfsboot nfs
Rescue login: root
iprconfig
Figure 6-88 Logging in as root
187
13.The first iprconfig screen is shown in Figure 6-89. Specify 1 to verify the
hardware status.
IBM Power RAID Configuration Utility
Select one of the following:
1.
2.
3.
4.
5.
6.
7.
8.
Display hardware status
Work with disk arrays
Work with disk unit recovery
Work with SCSI bus configuration
Work with driver configuration
Work with disk configuration
Download microcode
Analyze log
Selection: 1
Figure 6-89 iprconfig main screen
The output is shown in Figure 6-90.
Display Hardware Status
Type option, press Enter.
1=Display hardware resource information details
OPT Name
PCI/SCSI Location
--- ------ -------------------------0000:01:01.0/2:
sda
0000:01:01.0/2:0:1:0
sdb
0000:01:01.0/2:1:1:0
Description
------------------------PCI-X SAS RAID Adapter
Physical Disk
Physical Disk
Status
---------------Operational
Active
Active
e=Exit q=Quit r=Refresh t=Toggle
Figure 6-90 Output of iprconfig: Hardware status display
Tip: If you get a message that there are no hard drives, then check the
CD-ROM that you booted from. If you booted from the SLES9 CD-ROM 1,
you will get this message. You must boot from the SLES9 SP 3 CD-ROM 1
for this to work.
188
14.Type q to return to the main menu shown in Figure 6-89 on page 188.
15.Type 2 to work with disk arrays.
16.Type 5 to format a device for a disk array as shown in Figure 6-91.
Work with Disk Arrays
Select one of the following:
1.
2.
3.
4.
5.
6.
7.
8.
9.
Display disk array status
Create a disk array
Delete a disk array
Add a device to a disk array
Format device for RAID function
Format device for JBOD function (512)
Create a hot spare
Delete a hot spare
Force RAID Consistency Check
Selection: 5
Figure 6-91 iprconfig: Working with disk arrays
17.. You see a list of disks that you can format as shown in Figure 6-92. Type 1
beside each hard drive that you want to format for RAID. We put a 1 beside
both the hard drives.
Select Disks to format for RAID Function
1=Select
OPT
--1
1
Name
-----sda
sdb
PCI/SCSI Location
-------------------------0000:01:01.0/2:0:1:0
0000:01:01.0/2:0:1:0
Vendor
-------IBM-ESXS
IBM-ESXS
Product ID
-----------ST936701SS
ST936701SS
Status
-------Active
Active
e=Exit q=Cancel t=Toggle
Figure 6-92 iprconfig: Selecting the disks to format
189
18.You see a confirmation screen as shown in Figure 6-93. Type c to confirm the
formatting of the disks.
Confirm Initialize and Format Disks
ATTENTION! System crash may occur if selected device is in use. Data loss will
occur on selected device. Proceed with caution.
Press 'c' to confirm your choice for 1=Initialize and format.
q=Return to change your choice.
OPT
--1
1
Name
-----sda
sdb
PCI/SCSI Location
-------------------------0000:01:01.0/2:0:1:0
0000:01:01.0/2:1:1:0
Vendor
-------IBM-ESXS
IBM-ESXS
Product ID
------------ST936701SS
ST936701SS
Status
-------Active
Active
c=Confirm q=Cancel t=Toggle
Figure 6-93 iprconfig: Formatting disks confirmation
19.It takes a few minutes to format the disks. When they are finished, you see
the disk array screen shown in Figure 6-91 on page 189. Type 2 to create a
disk array.
20.This opens the screen shown in Figure 6-94. Type 1 beside the adapter that
owns the hard drives that you just formatted.
Create a Disk Array
Select the adapter.
Type choice, press Enter.
1=create a disk array
OPT Name
PCI/SCSI Location
Vendor
Product ID
Status
--- ------ -------------------------- -------- ------------ --------0000:01:01.0/2:
IBM
572E001
Operational
e=Exit q=Cancel t=Toggle
Figure 6-94 Selecting the adapter
190
21.You see the list of formatted disks as shown in Figure 6-95. Type 1 beside
both the hard drives to add them both to the array.
Select Disk Units for Disk Array
1=Select
OPT Name
PCI/SCSI Location
--- ------ -------------------------0000:01:01.0/2:0:1:0
0000:01:01.0/2:1:1:0
Vendor
-------IBM-ESXS
IBM-ESXS
Product ID
------------ST936701SS
ST936701SS
Status
------Zeroed
Zeroed
Figure 6-95 List of formatted disks
22.When you press Enter you see the screen to select the protection level as
shown in Figure 6-96. If you select c to change the setting, you will see an
option of either RAID 0 or RAID 10. RAID 0 is just stripping. RAID 10 is
normally stripping and mirroring, but because we only have two drives
internally in the JS21, it becomes just mirroring. This can be helpful to guard
against a disk crash. When you press Enter on the screen it creates the array
and takes you back to the Work with Disk Arrays screen shown in Figure 6-91
on page 189.
Select Protection Level and Stripe Size
Default array configurations are shown. To change
setting hit "c" for options menu. Highlight desired
option then hit Enter
c=Change Setting
Protection Level . . . . . . . . . . . . : Raid 10
Stripe Size. . . . . . . . . . . . . . . : 64 K
Queue Depth. . . . . . . . . . . . . . . : 8
Figure 6-96 Selecting the protection level
23.Reboot the JS21.
You have finished creating the disk array. If you are going to install SLES9 from a
CD-DROM, you have to boot from CD-ROM 1 of the SLES9 SP3. After you choose
to start the installation, you are prompted for CD-ROM 1 of the SLES9 installation
media. Insert this CD-ROM and follow the instructions in the screen menus.
191
192
7
Chapter 7.
Installing and managing the
Virtual I/O Server
The Integrated Virtualization Manager (IVM) is a feature of the Virtual
Input/Output (I/O) Server (VIOS). This chapter describes how to install and
manage the VIOS on a BladeCenter JS21. It further describes the two different
interfaces to the IVM: The graphic interface and the command line based
interface. It also provides examples of how to create and modify partitions, and
configure storage and network.
193
7.1 VIOS installation in a JS21
This chapter describes how to install the VIOS on a BladeCenter JS21using
either a DVD or the network.
7.1.1 VIOS image installation from a DVD
VIOS Version 1.2.1 is shipped as a single DVD media that contains a bootable
image of the software. It also contains the IVM component. CD media is not
supported for the VIOS.
Attention: When using the JS21 in a BladeCenter chassis that does not have
a DVD drive in the media tray, you can install the VIOS through the network
from a Network Installation Manager (NIM) server or a Linux server.
When you install from a DVD, assign the media tray to the blade that you want
(see 6.5.3, “Assigning the media tray” on page 94) and mount the VIOS
installation media in the DVD drive of the media tray. The remaining steps are
similar to a normal AIX installation as described in 8.2, “Preparing AIX installation
from CD/DVD” on page 260.
7.1.2 VIOS image installation from a NIM server
You can install the VIOS from a NIM server, especially when the BladeCenter
chassis has no DVD drive.
Note: The method described in this section is different from the option where
you type smitty nim_power5 from an AIX command-line, and then select
Virtual I/O Server and Integrated Virtualization Manager Installation
Tasks (/usr/sbin/installios command). This option requires a Hardware
Management Console (HMC). Because we have no HMC in an IVM
environment, we require another method in our case.
To install the VIOS using NIM, perform the following steps:
1. Install or define an existing server running AIX that you can configure as a
NIM server.
2. If your NIM server does not have a DVD drive, get access to a computer with
a DVD drive and a network connection. This computer can run an operating
system other than AIX, for example, Linux or Windows.
194
Note: The AIX version must be at least 5300, technology level 04. To verify
this, use one of the following commands:
/usr/bin/oslevel -rf
instfix -i | grep AIX_ML
Use the following command to see the service pack level:
/usr/bin/oslevel -s
3. Configure the NIM server as described in 8.3, “Preparing AIX network
installation using NIM” on page 261.
4. Mount the VIOS installation DVD in the computer and transfer the files
mksysb and bosinst.data from the /nimol/ioserver_res directory on the DVD to
the NIM server:
# mount –oro –vcdrfs /dev/cd0 /mnt
# cp /mnt/nimol/ioserver_res/mksysb /export/vios
# cp /mnt/nimol/ioserver_res/bosinst.data /export/vios
Note: If your operating system is not capable of resolving links, then you
might have to copy the file mksysb from the directory /usr/sys/inst.images
instead. The file mksysb must have a size of approximately 600 MB.
Chapter 7. Installing and managing the Virtual I/O Server
195
Configuring NIM using System Management Interface Tool
To configure NIM using System Management Interface Tool (SMIT):
1. Go to the NIM server and define a mksysb resource from the mksysb file you
copied from the DVD. To do this, type smitty nim_mkres, select mksysb and
enter the details as shown in Figure 7-1.
Define a Resource
[Entry
Fields]
* Resource Name
* Resource Type
* Server of Resource
[master]
+
* Location of Resource
[/export/vios/mksysb]
[MORE...9]
F1=Help
F5=Reset
F9=Shell
F2=Refresh
F6=Command
F10=Exit
Figure 7-1 NIM: Defining VIOS mksysb resource
196
[vios_mksysb]
mksysb
F3=Cancel
F7=Edit
Enter=Do
F4=List
F8=Image
2. Define a bosinst_data resource in the smitty nim_mkres fast path by selecting
bosinst_data and enter the details as shown in Figure 7-2.
Define a Resource
[Entry
Fields]
* Resource Name
* Resource Type
* Server of Resource
+
* Location of Resource
[vios_bid]
bosinst_data
[master]
/export/vios/bosinst.data]
Comments
[]
Source for Replication
[] +
F1=Help
F5=Reset
F9=Shell
F2=Refresh
F6=Command
F10=Exit
F3=Cancel
F7=Edit
Enter=Do
F4=List
F8=Image
Figure 7-2 NIM: Defining VIOS bosinst_data resource
Note: By default this bosinst.data file performs a non-prompted installation.
This means that it selects the target hard disk automatically. If you do not
want this, then change PROMPT = no to PROMPT = yes in the
bosinst.data file. You can do this even if the resource is already defined
and is active immediately.
197
3. Define a Shared Product Object Tree (SPOT) resource from the mksysb
resource defined in Figure 7-1 on page 196. From the smitty nim_mkres fast
path, select spot and enter the details as shown in Figure 7-3.
Define a Resource
*
*
*
*
*
[Entry Fields]
[vios_spot]
spot
[master]
[vios_mksysb]
[/export/vios]
yes
[]
Resource Name
Resource Type
Server of Resource
Source of Install Images
Location of Resource
Expand file systems if space needed?
Comments
installp Flags
COMMIT software updates?
SAVE replaced files?
AUTOMATICALLY install requisite software?
OVERWRITE same or newer versions?
VERIFY install and check file sizes?
F1=Help
F5=Reset
F9=Shell
F2=Refresh
F6=Command
F10=Exit
F3=Cancel
F7=Edit
Enter=Do
no
yes
yes
no
no
+
+
/
+
+
+
+
+
+
F4=List
F8=Image
Figure 7-3 NIM: Defining VIOS SPOT resource
Important: It is essential to define the SPOT from the just created mksysb
resource and not from the lpp_source of the NIM server as highlighted in
Figure 7-3. This guarantees that the SPOT and the corresponding network
boot image match the operating system version of the VIOS to be installed.
198
4. Define a client machine as described in 8.3.4, “Defining the NIM client” on
page 266. Allocate the mksysb (vios_mksysb), bosinst_data (vios_bid), and
spot (vios_spot) resources as described in 8.3.5, “Allocating resources to a
client” on page 268. Figure 7-4 shows the resources that you must allocate to
the client.
Manage Network Install Resource Allocation
Mo????????????????????????????????????????????????????????????????????????
?
Available Network Install Resources
?
?
?
? Move cursor to desired item and press F7.
?
?
?
?
?
?
? [MORE...10]
?
? > vios_bid
bosinst_data
?
? > vios_spot
spot
?
? > vios_mksysb
mksysb
?
? [BOTTOM]
?
?
?
? F1=Help
F2=Refresh
F3=Cancel
?
? F7=Select
F8=Image
F10=Exit
?
F1? Enter=Do
/=Find
n=Find Next
?
F9????????????????????????????????????????????????????????????????????????
Figure 7-4 NIM: Allocating VIOS resources
199
5. Activate the installation as described in 8.3.6, “Activating the client
installation” on page 268. Here we select Source for BOS Runtime Files =
mksysb as shown in Figure 7-5.
Perform a Network Install
[Entry
Fields]
Target Name
Source for BOS Runtime Files
installp Flags
Fileset Names
Remain NIM client after install?
Initiate Boot Operation on Client?
Set Boot List if Boot not Initiated on Client?
Force Unattended Installation Enablement?
ACCEPT new license agreements?
F1=Help
F5=Reset
F9=Shell
F2=Refresh
F6=Command
F10=Exit
F3=Cancel
F7=Edit
Enter=Do
Figure 7-5 NIM: Activating VIOS installation from mksysb
200
js21_vios
mksysb
[-agX]
[]
no
no
no
no
[no]
+
+
+
+
+
+
F4=List
F8=Image
If you get the warning as shown in Figure 7-6, you can ignore it.
COMMAND STATUS
Command: OK
stdout: yes
stderr: no
warning: 0042-360 m_bos_inst: The SPOT level is older than the
mksysb level. Therefore,
the BOS installation may encounter problems.
Update the SPOT to match the mksysb level or create a
new SPOT that has the same level.
F1=Help
F8=Image
n=Find Next
F2=Refresh
F9=Shell
F3=Cancel
F10=Exit
F6=Command
/=Find
Figure 7-6 NIM: Activating VIOS installation warning message
The rest of the installation is similar to 8.4, “Installing AIX on the client” on
page 270.
Configuring NIM using the command line
You can also configure NIM (described in the previous section) using the
command line.
NIM resources
The following commands define NIM resources.
bosinst_data resource:
# nim –o define –t bosinst_data –a server=master –a
location=/export/vios/bosinst.data vios_bid
mksysb resource:
# nim –o define –t mksysb –a server=master –a
SPOT resource (/usr file system):
# nim –o define –t spot –a server=master –a source=vio_mksysb –a
location=/export/vios vios_spot
201
Client definition of Virtual I/O Server
For the client definition of VIOS, perform the following steps:
1. Define the client:
# nim –o define –t standalone –a if1=“network1
js21_vios.itsc.austin.ibm.com 0” –a cable_type1=tp js21_vios
2. Initiate a pull installation from the NIM master:
# nim –o bos_inst –a source=mksysb –a mksysb=vios_mksysb -a
bosinst_data=vios_bid –a spot=vios_spot –a no_client_boot=yes
js21_vios
This allocates all the necessary resources, but waits for the VIOS to initiate the
installation itself. To finish the installation, proceed with the steps described in
8.4, “Installing AIX on the client” on page 270.
7.1.3 VIOS image installation from a Linux server
We describe a method that uses a Linux server for the network installation in the
case where there is no AIX server available. Therefore, you cannot use the nimol
Red Hat Package Manager (RPM) because it ships with the AIX license CD.
This section describes how to set up the Linux server for a VIOS network
installation. The following description is based on SUSE Linux Enterprise Server
9, Service Pack 3 (SLES9 SP3).
General notes
We use the following names and values in the configuration file examples of this
section. You might have to change these based on your environment:
Subnet: 9.153.99.0
Subnet mask: 255.255.255.0
Gateway: 9.153.99.1
Table 7-1 lists the VIOS installation network settings.
Table 7-1 VIOS installation network settings
202
Host name
Internet Protocol (IP) address
Medium Access Control
(MAC) address
Client: bch-6s
9.153.99.65
00:11:25:c9:30:b7
Server: bc-4s
9.153.99.49
Note: The server must be able to resolve the client's host name, by either the
Domain Name System (DNS), /etc/hosts, or another method. Depending on
the setup, you might have to use the fully qualified domain name (FQDN).
Installing RPM packages
You have to install the following packages using rpm -i, if they are not already
there:
tftp-0.36-44.4.ppc.rpm
dhcp-3.0.1rc13-28.18.ppc.rpm
dhcp-server-3.0.1rc13-28.20.ppc.rpm
Preparing Linux server resources
The network installation requires some resources on the server that we create
now.
Directories
Create the directories for storing the VIOS installation source:
mkdir /tftpboot
mkdir -p /export/vios
VIOS resources from the DVD
Mount the VIOS DVD and copy the resources booti.chrp.mp.ent.Z, bosinst.data,
ispot.tar.Z, and mksysb from the directory /nimol/ioserver_res on the DVD to the
local directory /export/vios:
cp /mnt/nimol/ioserver_res/*
/export/vios
If you do not want an unattended installation later, you have to edit bosinst.data
and change PROMPT=no to PROMPT=yes. Later this opens the installation
dialog and offers you the possibility to select the target hard disk.
Unpacking the SPOT
You have to unpack the so-called SPOT from the compressed image. This
creates the directory SPOT under the current directory /export/vios:
cd /export/vios
tar -xzf ispot.tar.Z
203
Preparing the boot image and information file
To prepare the boot image and information file:
1. Uncompress the boot image from the DVD so that you can use it for the
network installation.
cd /tftpboot
gunzip < /export/vios/booti.chrp.mp.ent.Z > bch-6s
2. Create a file named bch-6s.info with the content shown in Example 7-1.
Example 7-1 Syntax of the information for file installation
#------------------ Network Install Manager --------------export NIM_SERVER_TYPE=linux
export NIM_SYSLOG_PORT=514
export NIM_SYSLOG_FACILITY=local2
export NIM_NAME=bch-6s
export NIM_HOSTNAME=bch-6s
export NIM_CONFIGURATION=standalone
export NIM_MASTER_HOSTNAME=bc-4s
export REMAIN_NIM_CLIENT=no
export RC_CONFIG=rc.bos_inst
export
NIM_BOSINST_ENV="/../SPOT/usr/lpp/bos.sysmgt/nim/methods/c_bosinst_env"
export
NIM_BOSINST_RECOVER="/../SPOT/usr/lpp/bos.sysmgt/nim/methods/c_bosinst_
env -a hostname=bch-6s"
export NIM_BOSINST_DATA=/NIM_BOSINST_DATA
export SPOT=bc-4s:/export/vios/SPOT/usr
export NIM_BOS_IMAGE=/NIM_BOS_IMAGE
export NIM_BOS_FORMAT=mksysb
export NIM_HOSTS=" 9.153.99.65:bch-6s 9.153.99.49:bc-4s "
export NIM_MOUNTS="
bc-4s:/export/vios/bosinst.data:/NIM_BOSINST_DATA:file
bc-4s:/export/vios/mksysb:/NIM_BOS_IMAGE:file "
export ROUTES=" default:0:9.153.99.1 "
204
Network File System exports and server daemons
To configure the Network File System (NFS) service, perform the following steps:
1. Add the following entries to /etc/exports as shown in Example 7-2.
Example 7-2 /etc/exports
/export/vios/mksysb
/export/vios/SPOT/usr
/export/vios/bosinst.data
*(ro,insecure,no_root_squash)
2. Check if portmap is already running on SLES9 SP3. This was true with the
default installation on our system:
/etc/init.d/portmap status
If not, start portmap:
/etc/init.d/portmap start
3. Start the Network File System (NFS) server:
/etc/init.d/nfsserver start
Preparing Trivial File Transfer Protocol service
To prepare Trivial File Transfer Protocol (TFTP) service:
1. Change the setting for disable to no in /etc/xinetd.d/tftp, as shown in
Example 7-3.
Example 7-3 /etc/xinetd.d/tftp
service tftp
{
socket_type
protocol
wait
user
server
server_args
disable
}
=
=
=
=
=
=
=
dgram
udp
yes
root
/usr/sbin/in.tftpd
-s /tftpboot
no
2. Restart the xinited daemon to activate your change:
/etc/init.d/xinetd restart
205
Configuring syslog
To configure syslog:
1. Edit the file /etc/sysconfig/syslog and add the -r option to
SYSLOGD_PARAMS. Depending on the Linux distribution, this variable
might also be named SYSLOGD_OPTIONS:
SYSLOGD_PARAMS="-r "
2. Edit the file /etc/syslog.conf and make the changes as shown in Example 7-4.
Example 7-4 /etc/syslog.conf
#local2,local3.*
local3.*
local4,local5.*
local6,local7.*
local2.*
-/var/log/localmessages
/var/log/nimol.log
3. Restart the syslogd daemon:
/etc/init.d/syslog restart
Configuring the firewall
If the firewall is active, you have to allow the client to update the server's syslog.
Run the following command. On the default SLES9 system, this is not necessary.
iptables -I INPUT 1 -s bch-6s -j ACCEPT
Setting up Dynamic Host Configuration Protocol
To set up Dynamic Host Configuration Protocol (DHCP):
1. Edit the file /etc/dhcpd.conf and add the lines shown in Example 7-5.
Example 7-5 /etc/dhcpd.conf
always-reply-rfc1048 true;
allow bootp;
deny unknown-clients;
not authoritative;
default-lease-time 600;
max-lease-time 7200;
ddns-update-style none;
206
subnet 9.153.99.0 netmask 255.255.255.0 {
host bch-6s {
fixed-address 9.153.99.65;
hardware ethernet 00:11:25:c9:30:b7;
next-server 9.153.99.49;
filename "bch-6s";
}
}
Note: The value for the file name does not necessarily have to be identical
with the host name, but it must match with the file name that we use in
/tftpboot.
2. Make the following changes in /etc/sysconfig/dhcpd to allow your server to
handle DHCP requests on Ethernet port eth1:
DHCPD_INTERFACE="eth1"
3. Restart your dhcpd server to activate the changed configuration:
/etc/init.d/dhcpd restart
Installing on the client
This section describes the actual installation process on the client.
Important: Linux is configured to boot efficiently by only starting the required
daemons. Depending on the setup on the Linux server, you might have to start
the services xinetd, dhcp, portmap, and NFS server manually after a reboot.
You can also update the configuration so that they are automatically started at
the next reboot.
Start a Serial over LAN (SoL) session to the client and power on the JS21. Enter
the System Management Services (SMS) menu and start the network installation
as described in 6.7, “Providing a console for the BladeCenter JS21” on
page 121; 6.8.1, “Activating the System Management Services interface” on
page 133; and 8.4, “Installing AIX on the client” on page 270.
207
Important: Directed Bootstrap Protocol (BOOTP) does not work with the
BOOTP function provided by DHCP. Therefore, the IP address of the client
and the server must be 0.0.0.0 in the SMS IP parameters as shown in
Example 7-6. Specify the MAC address of the client in the server's
/etc/dhcpd.conf file as shown in Example 7-5 on page 206 to use broadcast
BOOTP. However, broadcast BOOTP works only if the client and server are in
the same subnet. Directed BOOTP with the BOOTPD implementation on
Linux was not verified during the writing of this book.
Example 7-6 SMS IP parameters
PowerPC Firmware
----------------------------------------------------------------------IP Parameters
Port 2-IBM 2 PORT 1000 Base-SX PCI-X Adapter: U788D.001.99DWL3F-P1-T8
1.
Client IP Address
[0.0.0.0]
2.
Server IP Address
[0.0.0.0]
3.
Gateway IP Address
[000.000.000.000]
4.
Subnet Mask
[255.255.255.000]
You can monitor the progress of the client installation on the server by monitoring
the syslog file that the installation creates:
tail -f /var/log/nimol.log
Tip: If your installation environment is temporary, then you must consider
removing access to the installation server and stopping the installation
services after a successful installation (NFS, DHCP daemon (DHCPD),
TFTP).
7.2 Initial VIOS configuration
The new VIOS requires a simple configuration setup using the command-line
interface (CLI), as described in 7.3.2, “IVM command-line interface” on page 213.
Later, you can use the Web interface besides the CLI, as described in 7.3.1, “IVM
graphical user interface” on page 211.
1. Log on to the VIOS using the user padmin and the default password padmin.
When prompted, change the login password to a secure one.
2. Accept the VIOS licence by issuing the license -accept command.
208
7.2.1 Virtualization setup
First we check whether the four virtual Ethernet interfaces that VIOS manages
are already created during the VIOS installation. Perform the lsdev command
shown in Example 7-7 on page 209 to verify whether the four virtual I/O Ethernet
adapters are already available. If they are not yet available, the administrator
must issue the mkgencfg command to create them with the following syntax:
mkgencfg -o init [-i "configuration data"]
For example:
mkgencfg -o init -i "mac_prefix=06ABC0"
Note: If the virtual Ethernet adapters are already available after first booting
the VIOS (which is possible if a VIOS is already installed), you might have to
run:
lpcfgop -o clear
Restart the system and then run mkgencfg. The help function in the IVM’s CLI
does not display the mkgencf and the lpcfgop command. You can get the
description by using:
man mkgencf
You can use the optional configuration data to define the prefix of the MAC
address of all four virtual Ethernet adapters of the VIOS and to define the
maximum number of partitions supported by the VIOS after the next restart.
Example 7-7 shows the output of the mkgencfg command.
Example 7-7 Creating virtual Ethernet adapters
$ lsdev | grep ênt
ent0
Available
ent1
Available
Gigabit Ethernet-SX PCI-X Adapter (14101403)
$ mkgencfg -o init
$ lsdev | grep ênt
ent0
Available
ent1
Available
ent2
Available
ent3
Available
ent4
Available
ent5
Available
Gigabit
Gigabit
Virtual
Virtual
Virtual
Virtual
Ethernet-SX PCI-X Adapter (14101403)
Ethernet-SX PCI-X Adapter (14101403)
I/O Ethernet Adapter (l-lan)
209
Attention: Although you can change the maximum number of partitions later
using the IVM Web graphical user interface (GUI), you can no longer modify
the MAC address easily.
Here is a method that worked in our environment. You will not get a network
connection after the reboot and therefore have to use SoL.
$ bkprofdata -o backup -f profile.bak
$ lpcfgop -o clear
# backup VIOS profile
# clear VIOS profile
Restart the system and then run:
$
$
#
#
$
mkgencfg -o init -i "mac_prefix=06ABC0"
oem_setup_env
mkdev -l inet0
exit
rstprofdata -l 1 -f profile.bak --ignoremac
#
#
#
#
#
change mac prefix
change to root user
fix IP routes
back to padmin
restore VIOS profile
7.2.2 Setting the date and time
Use the chdate command to set the VIOS date and time, using the following
syntax:
chdate [-year YYyy] [-month mm] [-day dd] [-hour HH] [-minute MM]
[-timezone TZ]
chdate mmddHHMM[YYyy | yy] [-timezone TZ]
7.2.3 Initial network setup
The IVM Web interface requires a valid network configuration to work. Configure
the IP address by choosing a physical network adapter and issuing the mktcpip
command from the command line, using the following syntax:
mktcpip -hostname HostName -inetaddr Address -interface Interface
[-start] [-netmask SubnetMask] [-cabletype CableType]
[-gateway Gateway] [-nsrvaddr NameServerAddress
[-nsrvdomain Domain]]
Example 7-8 shows how to set the host name, address, and IP address for the
VIOS.
Example 7-8 VIOS network setup at the command line
$ mktcpip -hostname js21_vios -inetaddr 9.3.5.229 -interface en0
-netmask 255.255.255.0 -gateway 9.3.5.41 -nsrvaddr 9.3.4.2 -nsrvdomain
itsc.austin.ibm.com
210
Important: The IVM, like all Web servers, requires a valid name resolution to
work correctly. If DNS is involved, check that both the name and the IP
address resolution of the IVM host name are correct.
After the IVM Web server has access to the network, it is possible to use the
Web GUI with the Hypertext Transfer Protocol (HTTP) or the Hypertext Transfer
Protocol-Secure (HTTPS) protocol pointing to the IP address of the IVM server
application. Authentication requires the use of the padmin user, unless other
users have been created.
7.3 VIOS management using IVM
The IVM provides a unique environment to administer logical partition capable
servers. It provides two ways to configure and manage logical partitions
(LPARs):
A GUI designed to be as simple and intuitive as possible, incorporating
partition management, storage management, serviceability, and monitoring
capabilities (see 7.3.1, “IVM graphical user interface”)
A CLI to enable scripting capabilities (see 7.3.2, “IVM command-line
interface” on page 213)
You can use either of the two interfaces to create, delete, and update the logical
partitions and perform non-dynamic operations on LPARs including the partition
of the VIOS itself.
As described in 4.1.2, “Dynamic logical partitioning” on page 43, the JS21does
not support dynamic logical partitioning (DLPAR) on the client partitions. This
means that you can change some options such as memory and processor
assignment only when the corresponding LPAR is stopped. However, you can
assign and remove storage and optical media when the LPAR is running.
The following examples use both methods, GUI and CLI.
7.3.1 IVM graphical user interface
The GUI is a Hypertext Markup Language (HTML)-based interface. It enables
you to create LPARs on a single managed system, manage the virtual storage
and virtual Ethernet on the managed system, perform maintenance, and view
service information related to the managed system.
211
Connecting to the IVM
Open a Web browser window and connect using HTTP or HTTPS to the IP
address that you assigned to the VIOS during the installation process (see 7.2.3,
“Initial network setup” on page 210).
As a result, a Welcome window that contains the login and the password
prompts opens, as shown in Figure 7-7. The default user ID is padmin, and the
password is the one you defined during the VIOS installation. The default
password is padmin until you change it.
Figure 7-7 IVM login page
After the authentication process, the default IVM console window opens, as
shown in Figure 7-8 on page 213. The IVM GUI consists of several elements.
The following elements are the most important:
212
Navigation area
The navigation area displays the tasks that you can
access in the work area.
Work area
The work area contains information related to the
management tasks that you perform using the IVM and
related to the objects on which you can perform
management tasks.
Task area
The task area lists the tasks that you can perform for
items displayed in the work area. The tasks listed in the
task area can change depending on the page that is
displayed in the work area, or even depending on the tab
that is selected in the work area.
Work Area
Navigation area
Task area
Figure 7-8 IVM console
7.3.2 IVM command-line interface
The text-based console with the CLI is accessible through an SoL terminal
connection or through network connectivity using Telnet. The IP address is the
same as the one used to connect to the GUI, and which you defined during the
installation process. The CLI requires more experience to master than the GUI,
but it offers more possibilities to tune the partition’s definitions and can be
automated using scripts.
213
Use Telnet to connect to the VIOS login as padmin as shown in Example 7-9.
Example 7-9 CLI login to the VIOS
telnet (js21a2)
IBM Virtual I/O Server
login: padmin
padmin's Password:
$
Enter help to show an overview of the available commands. Example 7-10 shows
the output of the help command.
Example 7-10 IVM CLI global help
$ help
Install Commands
ioslevel
license
lssw
oem_platform_level
oem_setup_env
remote_management
updateios
LAN Commands
cfglnagg
cfgnamesrv
entstat
hostmap
hostname
lsnetsvc
lstcpip
mktcpip
netstat
optimizenet
ping
rmtcpip
startnetsvc
stopnetsvc
traceroute
Device Commands
214
Security Commands
lsfailedlogin
lsgcl
UserID Commands
chuser
lsuser
mkuser
passwd
rmuser
Maintenance Commands
backupios
bootlist
cattracerpt
chdate
chlang
diagmenu
errlog
fsck
invscout
ldfware
loginmsg
lsfware
lslparinfo
motd
mount
chdev
chpath
cfgdev
lsdev
lsmap
lspath
mkpath
mkvdev
mkvt
rmdev
rmpath
rmvdev
pdump
restorevgstruct
savevgstruct
showmount
shutdown
snap
startsysdump
starttrace
stoptrace
sysstat
topas
unmount
viostat
Physical Volume Commands
lspv
migratepv
Logical Volume Commands
chlv
cplv
extendlv
lslv
mklv
mklvcopy
rmlv
rmlvcopy
Volume Group Commands
activatevg
chvg
deactivatevg
exportvg
extendvg
importvg
lsvg
mirrorios
mkvg
redefvg
reducevg
syncvg
unmirrorios
Shell Commands
awk
cat
chmod
clear
cp
date
ftp
grep
head
ls
man
mkdir
more
mv
rm
sed
stty
tail
tee
vi
wall
wc
who
Storage Pool Commands
chsp
lssp
215
mkbdsp
mksp
rmbdsp
$
Help relating to an individual command is available with the -h flag. Example 7-11
shows the help for the mkvt command.
Example 7-11 IVM CLI mkvt help
$ mkvt -h
Usage: mkvt
{-id PartitionID}
Opens a virtual terminal session for an AIX or virtual I/O
server
partition in a managed system.
-id
The ID of the partition for which to open the virtual
terminal session.
7.4 Storage management
The VIOS uses the following storage management concepts:
Physical volume: A physical disk or a logical unit number (LUN) on a storage
area network (SAN). They are all owned by the VIOS. You can assign a
physical volume not belonging to any storage pool whole to a single partition
as a virtual device.
Storage pool: A set of physical volumes treated as a single entity. There can
be multiple pools and they cannot share physical disks. One pool is defined
as the default storage pool.
Logical volume: A portion of a storage pool, which can belong to more than
one physical volume
Virtual disk: Logical volume that the VIOS assigns to a single partition as a
virtual device
You can assign both physical volumes and virtual disks to an LPAR to provide
disk space. Each of them is represented by the LPAR operating system as a
single disk. For example, if you assign a 73.4 GB physical disk and a 3 GB virtual
disk to an LPAR running AIX 5L, then the operating system creates two hdisk
devices.
216
Note: In practice, there is no difference for the user of the IVM between virtual
disk and logical volume, because a virtual disk is stored in a logical volume
that acts as a container for the virtual disk. However, in the GUI, you find
differences in the menu options. For example, creating a virtual disk is done in
the default storage pool and creating a logical volume gives you the possibility
to select the storage pool (see “Creating the logical volume” on page 222).
At installation time of the VIOS, there is only one storage pool named rootvg,
typically containing only one physical volume. All the remaining physical volumes
are available but not assigned to any pool.
Note: Typically a storage pool is the same as a volume group in AIX. While
the VIOS is running on an underlying AIX, it uses volume groups as storage
pools, which you can especially see with the name rootvg. The different name
is related to the possibility of implementing VIOS on Linux platforms too.
The rootvg storage pool is used for the VIOS. We recommend that you do not
use it to provide disk space to logical partitions. Because it is the only pool
available at installation time, it is also defined as the default pool. Create another
pool and set it as the default before creating other partitions.
Important: Create at least one additional storage pool so that the rootvg pool
is not the default storage pool.
However, if you have only one disk, which might be the case if you have
configured all the disks as a RAID array as described in 6.12, “SAS hardware
RAID configuration” on page 170, your only choice is to use rootvg. In this case,
you must be very careful not to overwrite rootvg, for example, when reinstalling
the VIOS. Be sure to back up all essential data.
From any storage pool, logical volumes can be defined and configured as virtual
disks. They can be created in several ways, depending on the IVM menu that is
in use:
During LPAR creation: A logical volume is created in the default storage pool
and assigned to the partition.
Using the Create Devices link: A logical volume is not assigned to any
partition and it is created in the default storage pool. There is an Advanced
tab that enables the storage pool selection.
We discuss basic storage management in the following section, and the
advanced setup in 7.7, “Advanced storage management” on page 244.
217
Storage pool disk management
During the installation of the VIOS, a default storage pool is created and named
rootvg. During the creation of the logical partition, the IVM automatically creates
virtual disks in the default storage pool. We recommend that you create another
storage pool and add virtual disks to it for the logical partitions. For advanced
configuration of the storage pool, refer to 7.7, “Advanced storage management”
on page 244.
Creating the storage pool
A storage pool consists of a set of physical disks that can be different types and
sizes. You can create multiple storage pools. However, a disk can only be a
member of a single storage pool.
Important: All the data of a physical volume (hard disk) is erased when you
add this volume to a storage pool.
To create a storage pool, perform the following steps:
1. From the Storage Management menu in the navigation area, click the Create
Devices link.
2. Select the Advanced Create Devices tab in the work area. Click Create
Storage Pool in the work area, as shown in Figure 7-9.
Figure 7-9 Creating the storage pool
218
3. In the new window that opens:
a. Enter a name in the Storage pool name field, as shown in Figure 7-10.
b. Select the required disks.
c. Click OK to create the storage pool.
Figure 7-10 Storage pool name
Default storage pool
The default storage pool that is created during the VIOS installation is rootvg.
This is because rootvg is the only volume group created at that time.
Important: Create at least one additional storage pool. Because the rootvf
storage pool is the default storage pool, this causes VIOS data and user data
to be merged. Therefore, do not use the rootvg storage pool for virtual disk
storage.
Because the VIOS is installed in rootvg, when VIOS is reinstalled, the rootvg
storage pool is overwritten. Change the default storage pool to another one to
avoid creating virtual disks within the rootvg by default. This prevents the loss of
user data during a VIOS update.
To change the default storage pool, perform the following steps:
1. From the Storage Management menu in the navigation area, click Advanced
View/Modify Devices.
219
2. Select the storage pool you want as the default. Click Assign as default
storage pool in the task area, as shown in Figure 7-11.
Figure 7-11 Storage pools list
3. A summary with the current and the next default storage pool opens, as
shown in Figure 7-12. Click OK to validate the change.
Figure 7-12 Assigning default storage pool
220
7.5 VIOS partition configuration
The first panel that opens after the login process is the partition configuration, as
Figure 7-13 Initial partition configuration
After the initial installation of the VIOS, there is only one VIOS partition on the
system with the following characteristics:
The ID is 1
The name is equal to the system’s serial number
The state is Running
The allocated memory is the maximum value between 512 MB and
one-eighth of the installed system memory
The number of allocated processors is equal to the number of physical
processors, and the processing units is equal to 0.1 times the number of
allocated processors
The default configuration for the partition is designed to be appropriate for most
VIOS installations. If the administrators want to change memory or processing
unit allocation of the VIOS partition, they can perform a re-configuration using
either the Web GUI or the command line, as described in 7.5.8, “Logical partition
configuration changes” on page 235.
221
7.5.1 Creating the logical volume
Logical volumes belong to a storage pool and are also known as virtual disks.
Logical volumes are used to provide disk space to logical partitions, but they are
not assigned to logical partitions when you create them. They can be created in
several ways, depending on the menu that is in use:
During LPAR creation: A logical volume is created in the default storage pool
and assigned to the partition.
After or before LPAR creation: A logical volume is not assigned to any
partition and is created in the default storage pool. There is an Advanced tab
that enables the storage pool selection.
To create a new logical volume, perform the following steps:
1. From the Storage Management menu in the navigation area, click Create
Devices.
2. Select the Advanced Create Devices tab in the work area. Click Create
Logical Volume in the work area.
3. In the new window that opens:
a.
b.
c.
d.
Enter a name for the logical volume, as shown in Figure 7-14.
Select a storage pool name from the drop-down list.
Enter a size for the logical volume.
Click OK to create the logical volume.
Figure 7-14 Logical volume name and size
222
To view your new logical volume and use it, select the View/Modify Devices link
in the Storage Management menu in the navigation area. The list of available
logical volumes is shown in the work area.
7.5.2 Creating logical partitions
A logical partition is a set of resources: Processors, memory, and I/O devices.
Each resource assigned to a logical partition is allocated regardless of whether
the logical partition is running or not. The VIOS does not allow the
overcommitment of resources.
To create a logical partition, perform the following steps:
1. From the Partition Management menu in the navigation area, click Create
Partitions. Click the Start Wizard button in the work area.
2. In the new window that opens, enter a name for the new partition, as shown in
Figure 7-15. Click Next.
Figure 7-15 Creating LPAR: Partition name
223
3. In the new window, enter the amount of memory necessary, as shown in
Figure 7-16 Creating LPAR: Assigned memory
224
4. In the new window, select the number of processors necessary and choose a
processing mode, as shown in Figure 7-17. In shared mode, each virtual
processor uses 0.1 processing units. Click Next.
Figure 7-17 Creating LPAR: Selecting processors and processing mode
225
5. Each partition has two virtual Ethernet adapters that you can configure to one
of the four available virtual Ethernet networks. In Figure 7-18, adapter one
uses the virtual Ethernet ID 1. The Virtual Ethernet Bridge Overview section
of the panel shows the physical network interface on which every virtual
network is bridged. In Figure 7-18, no bridge is created. Perform the
procedure described in “Ethernet bridging” on page 241 later to enable the
partition to connect to the physical network. Click Next.
Figure 7-18 Creating LPAR: Virtual Ethernet
226
6. In the new window, select Assign existing virtual disks and physical
volumes, as shown in Figure 7-19.
You can also let the IVM create the virtual disk for you by selecting Create
virtual disk, but be aware that the virtual disk is created in the default storage
pool. To create storage pool and virtual disks or change the default storage
pool, refer to , “Storage pool disk management” on page 218.
Click Next.
Figure 7-19 Creating LPAR: Storage type
227
7. In the new window, select the required virtual disks from the list, as shown in
Figure 7-20 Creating LPAR: Assigning storage
228
8. In the new window, select the required optical devices, as shown in
Figure 7-21. If there is no optical device available, see 7.5.7, “Optical device
sharing” on page 233 for further information. Click Next.
Figure 7-21 Creating LPAR: Assigning optical devices
229
9. A summary of the partition to be created is shown in Figure 7-22. Click Finish
to create the logical partition.
Figure 7-22 Creating LPAR: Summary
To view the new logical partition and use it, from the Partition Management menu
in the navigation area, click the View/Modify Partitions link. A list opens in the
work area.
230
7.5.3 Updating the logical partition profile
Example 7-12 shows how to change the name of a logical partition with the
chsyscfg command using the CLI.
Example 7-12 Updating the profile
$ lssyscfg -r prof --filter "lpar_names=LPAR2" -F lpar_name
LPAR2
$ chsyscfg -r prof -i "lpar_name=LPAR2,new_name=LPAR2_new_name"
$ lssyscfg -r prof --filter "lpar_names=LPAR2_new_name" -F lpar_name
LPAR2_new_name
7.5.4 Powering on a logical partition
Example 7-13 shows how to start a logical partition using the chsysstate
command. To follow the boot process, use the lsrefcode command.
Example 7-13 Powering on a partition
$ chsysstate -o on -r lpar -n LPAR2
$ lsrefcode -r lpar --filter "lpar_names=LPAR2" -F refcode
CA00E1F1
$ lsrefcode -r lpar --filter "lpar_names=LPAR2" -F refcode
CA00E14D
7.5.5 Opening a virtual terminal to a logical partition
Sometimes you have to access a logical partition using a virtual terminal, for
example, during installation or when there is no network connection to the logical
partition. To open a virtual terminal, perform the following steps:
1. Log on to the VIOS partition.
2. Open a virtual terminal for the logical partition to be installed with the mkvt
command. You have to specify the ID of the logical partition, as shown in
Example 7-14.
Note: When connecting a virtual terminal to an already running partition,
press Enter again after you issue mkvt to interact with the running system.
If you get the message Virtual terminal is already connected, you can
use the rmvt command to free up the session.
231
Example 7-14 Opening a virtual terminal
$ mkvt -id 3
AIX Version 5
(C) Copyrights by IBM and by others 1982, 2005.
Console login:
7.5.6 Installing an operating system on a logical partition
The operating system installation process is similar to the process for
stand-alone systems. The main steps are:
1. Log on to the VIOS partition.
command as shown in Example 7-14.
3. Start the logical partition in SMS mode. To do so, you can change the boot
mode in the properties of the partition’s profile before starting it. Alternatively,
enter 1 on the virtual terminal at the start of the boot process, as shown in
Example 7-15.
Example 7-15 Boot display
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
1 = SMS Menu
Memory
Keyboard
Network
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
IBM
5 = Default Boot List
SCSI
Speaker
4. To install an operating system, proceed as described in Chapter 8, “Installing
AIX” on page 259, or Chapter 9, “Installing Linux” on page 281.
232
7.5.7 Optical device sharing
You can dynamically add, move, or remove optical devices from or to any logical
partition, regardless of whether the logical partition is running or not.
Attention: In a BladeCenter environment, there are two levels of device
sharing. Within the BladeCenter by the media tray assignment as described in
6.5.3, “Assigning the media tray” on page 94, and the device sharing provided
by the VIOS. The following description requires that the media tray is assigned
to the blade server running the VIOS. When the media tray is assigned to the
VIOS after it is booted, you must run cfgdev within the CLI of the IVM to make
the optical drive Available. Use the lsdev command to see the presence and
the status of the required devices.
To change the assignment of an optical device, perform the following steps:
1. From the Storage Management menu in the navigation area, click
2. Select the Optical Devices tab in the work area. Select the optical device you
want to modify. Click Modify partition assignment in the tasks area, as
Figure 7-23 Selecting the optical devices
233
3. In the new window, select the name of the logical partition to which you want
to assign the optical device, as shown in Figure 7-24. You can also remove
the optical device from the current logical partition by selecting None. Click
OK.
Figure 7-24 Assigning the optical device partition
4. If you move or remove an optical device from a running logical partition, you
are prompted to confirm the forced removal before the optical device is
removed. Because the optical device becomes unavailable, log on to the
logical partition and unmount the optical device before going further. Click the
Eject button. If the drawer opens, this is an indication that the device is not
mounted. Click OK.
Note: We strongly recommend that you unmount the optical device’s file
systems within the operating system before unassigning the device to
avoid endless loop conditions.
5. The new list of optical devices is shown with the changes you made. Log on
to the related logical partition and use the appropriate command to discover
the new optical device. On AIX 5L, use the cfgmgr command.
234
7.5.8 Logical partition configuration changes
You might want to modify the properties of the VIOS or logical partitions, if
necessary.
Logical partition operations on a VIOS partition
You can allocate or release resources such as processors and memory on the
VIOS partition. You can run these operations either on the GUI or the CLI.
Although changing the processing units has immediate effect, changing
processors and memory remains as pending until the next reboot of the VIOS
partition. The GUI displays a yellow triangle with an exclamation mark inside for
pending updates as shown in Figure 7-27 on page 237. To reboot the VIOS, you
must stop all other client partitions (LPARs) before.
Logical partition operation on memory using the GUI
The following steps describe how to increase memory size dynamically for the
VIOS partition:
1. From the Partition Management menu in the navigation area, click
View/Modify Partitions.
2. Select the VIOS partition (see Figure 7-25). Click Properties in the task area.
Figure 7-25 LPAR memory operation: Selecting logical partition
235
3. In the new window, modify the pending values as necessary. In Figure 7-26,
the assigned memory is increased to 1 GB. Click OK.
Figure 7-26 LPAR memory operation: Increasing the memory size
236
Memory is not added or removed immediately. The update is pending until
the next reboot of the VIOS partition. The pending update is shown by a
yellow triangle with an exclamation mark inside, as shown in Figure 7-27.
Figure 7-27 LPAR change memory pending
Logical partition operation on virtual processors using the CLI
Log on to the VIOS using the CLI and run your LPAR operation. Example 7-16
shows how to add a 0.1 processing unit dynamically to the VIOS using the
chsyscfg command.
Example 7-16 LPAR virtual processor operation
$ lshwres -r proc --level lpar --filter "lpar_names=VIOS" -F curr_proc_units
0.20
$ chsyscfg -r prof -i lpar_name=VIOS,desired_proc_units=0.3
$ lshwres -r proc --level lpar --filter "lpar_names=VIOS" -F curr_proc_units
0.30
Client logical partition resources management
The IVM does not allow dynamic operations on resources such as the processor,
memory, and virtual Ethernet on a client logical partition. But dynamic operations
on the disks, optical devices, partition name, and boot mode are allowed.
237
Logical partition operation on virtual disks using the GUI
The following steps describe how to assign logical volumes to a partition using
the GUI:
1. From the Storage Management menu in the navigation area, click Advanced
2. Select the Logical Volumes tab in the work area. Select the required logical
volumes, as shown in Figure 7-28. Click Modify partition assignment in the
task area.
Figure 7-28 Selecting the logical volumes
238
3. In the new window, select the partition name that you want to assign the
logical volumes, as shown in Figure 7-29. Click OK to validate the logical
volume partition assignment.
Figure 7-29 Assigning logical volume partition
4. Log on to the related logical partition and discover the new disks. On AIX 5L,
use the cfgmgr command. Example 7-17 shows how the partition discovers
the two new virtual disks on AIX 5L.
Example 7-17 Virtual disk discovery
# lsdev -Ccdisk
hdisk0 Available
Virtual SCSI Disk Drive
# cfgmgr
# lsdev -Ccdisk
hdisk0 Available
hdisk1 Available
hdisk2 Available
You can also assign virtual disks by selecting Partition Management →
View/Modify Partitions → Properties (Tasks) → Storage (Tab).
239
Operation on partition definitions using the CLI
You can perform the same operations with the CLI as with the GUI. The CLI also
enables some non-dynamic updates on the partition’s definitions. Example 7-18
shows how to decrease processing units for a logical partition using the chsyscfg
command.
Example 7-18 Decreasing the processing units
$ lssyscfg -r prof --filter "lpar_names=AppServer" -F desired_proc_units
0.40
$ chsyscfg -r prof -i "lpar_name=AppServer,desired_proc_units=0.3"
$ lssyscfg -r prof --filter "lpar_names=AppServer" -F desired_proc_units
0.30
These updates become pending and are taken in account only at the next reboot
of the related logical partition. A triangle with an exclamation mark inside it is
displayed in the View/Modify Partitions screen if the current and pending values
are not synchronized. This also shows that changes made by the CLI are
immediately reflected by the GUI.
7.6 Network management
When installed, the VIOS configures one network device for each physical
Ethernet interface present on the system and creates four virtual Ethernet
adapters, each belonging to a separate virtual network. All physical Ethernet
adapters installed in the system are managed by the VIOS.
Any client logical partition can be created with its own virtual adapters connected
to any of the four available virtual networks. Client logical partitions can only
have up to two virtual Ethernet adapters, each connected to one of the four
virtual networks present in the system. No bridging is provided with physical
adapters at installation time.The VIOS enables any virtual network to be bridged
with any physical adapter, provided that the same physical adapter is not used to
bridge more than one virtual network.
To allow partitions to access an external corporate network, every virtual network
can be bridged to a physical adapter interface. For each network, a separate
adapter interface is required. IVM provides a Web interface to configure bridging.
240
Ethernet bridging
To configure Ethernet bridging:
1. From the Virtual Ethernet Management menu in the navigation area, click
View/Modify Virtual Ethernet.
2. In the work area, the virtual Ethernet panel shows the partitions that are
connected to the four available networks. Select the Virtual Ethernet Bridge
tab to configure bridging, as shown in Figure 7-30. For each virtual Ethernet,
you can select one physical device. Use the drop-down menu to select the
physical Ethernet. Click Apply to create the bridging device.
Note: If the physical Ethernet that you selected for bridging is already
configured with an IP address using the CLI, all connections to that address
are reset. This might drop an SoL session when configuring the interface
associated with I/O module 1 as described in 6.7, “Providing a console for the
BladeCenter JS21” on page 121.
Figure 7-30 Creating the virtual Ethernet bridge
241
The Web GUI hides the details of the VIOS network configuration. Example 7-19
describes the VIOS configuration before the creation of the bridge. For each
physical and virtual network adapter, an Ethernet device is configured. The VIOS
is connected to a physical network using the en0 interface and four virtual
network adapters are available (ent1 and the corresponding en1 interface are
currently not used).
Example 7-19 VIOS Ethernet adapters with no bridging
$ lsdev | grep ên
en0
Available
en1
Defined
en2
Defined
en3
Defined
en4
Defined
en5
Defined
ent0
Available
ent1
Available
ent2
Available
ent3
Available
ent4
Available
ent5
Available
$ lstcpip
Name Mtu
en0
1500
en0
1500
lo0
16896
lo0
16896
lo0
16896
Network
link#2
9.3.5
link#1
127
::1
Standard Ethernet Network Interface
Virtual I/O Ethernet Adapter (l-lan)
Address
0.11.25.c9.11.3d
js21_vios.itsc.au
localhost
Ipkts Ierrs
1379465
1379465
82
82
82
0
0
0
0
0
Opkts Oerrs
44040
44040
295
295
295
Coll
0
0
0
0
0
0
0
0
0
0
When a virtual Ethernet bridge is created, a new shared Ethernet adapter (SEA)
is defined, binding the physical device with the virtual device. If a network
interface is configured on the physical adapter, the IP address is migrated to the
new SEA.
242
Example 7-20 shows the result of bridging virtual network 1 with the physical
adapter ent0 when the VIOS uses the network interface en0. A new ent6 SEA
device is created, and the IP address of the en0 interface is migrated to the en6
interface. Because of the migration, all active network connections on en0 are
reset.
Example 7-20 Shared Ethernet adapter configuration
$ lsdev | grep ên
en0
Defined
en1
Defined
en2
Defined
en3
Defined
en4
Defined
en5
Defined
en6
Available
ent0
Available
ent1
Available
ent2
Available
ent3
Available
ent4
Available
ent5
Available
ent6
Available
$ lstcpip
Name Mtu
en6
1500
en6
1500
lo0
16896
lo0
16896
lo0
16896
Network
link#2
9.3.5
link#1
127
::1
Shared Ethernet Adapter
Address
0.11.25.c9.11.3d
js21_vios.itsc.au
localhost
Ipkts Ierrs
1380345
1380345
82
82
82
0
0
0
0
0
Opkts Oerrs
44078
44078
295
295
295
Coll
0
0
0
0
0
0
0
0
0
0
Use the following command to display the associated SEA devices, as shown in
Example 7-21.
lsmap -net -all
Example 7-21 SEA lsmap output
$ lsmap -net -all
SVEA
Physloc
------ -------------------------------------------ent2
U8844.5CZ.23A1137-V1-C3-T1
SEA
Backing device
ent6
ent0
243
Physloc
U788D.001.23A1137-P1-T7
SVEA
Physloc
------ -------------------------------------------ent3
U8844.5CZ.23A1137-V1-C4-T1
SEA
NO SHARED ETHERNET ADAPTER FOUND
SVEA
Physloc
------ -------------------------------------------ent4
U8844.5CZ.23A1137-V1-C5-T1
SEA
SVEA
Physloc
------ -------------------------------------------ent5
U8844.5CZ.23A1137-V1-C6-T1
SEA
7.7 Advanced storage management
You can assign virtual disks and physical volumes to any logical partition, one at
a time. You can change storage allocation over time, and the content of the
virtual storage is kept. When you create a virtual disk using a logical volume, you
can also increase its size. Data protection against single disk failure is available
using software mirroring:
On the VIOS to protect the VIOS itself but not the LPAR data
Using two virtual disks for each of the managed system’s LPAR to protect the
LPAR data
7.7.1 Virtual storage assignment to a partition
Unassigned virtual disks and physical volumes can be associated to a running
partition. After the operation is complete, the LPAR’s operating system must
issue its device discovery procedure to detect the newly added disk. In an AIX 5L
environment, do this by issuing the cfgmgr command.
Before removing a physical disk or a virtual disk from a running partition, the
operating system must remove the corresponding disk device because it
becomes unavailable. In an AIX 5L environment, do this by using the rmdev
command.
244
On the Web GUI, it is possible to remove a virtual disk or a physical volume from
a running LPAR, but a warning sign always appears requiring an additional
confirmation. Figure 7-31 shows an example of this message.
Figure 7-31 Forced removal of a physical volume
7.7.2 Virtual disk extension
Several options are available to provide additional disk space to a logical
partition. The primary solution is to create a new virtual disk or select an entire
physical disk and dynamically assign it to a partition. Because you can do this
operation when the partition is running, it is the preferred one. After the partition's
operating system issues its own device reconfiguration process, a new virtual
Small Computer System Interface (SCSI) disk is available for use. You can use
this disk to extend existing data structures when using Linux with a logical
volume manager or AIX 5L.
245
When disk space is provided to a partition using a virtual disk, a secondary
solution is to extend it using the IVM. You can perform this operation when the
partition is running, but you must take the virtual disk offline to activate the
change. Schedule disk outages carefully so that they do not impact the overall
application availability. Consider using this solution to increase an existing
operating system's volume in size and when you cannot add a new virtual SCSI
disk for this purpose, that is, when using Linux without a logical volume manager.
Important: We recommend that you do not perform virtual disk extension
when using AIX 5L, because the same result is achieved by adding a new
virtual disk. If the virtual disk is used by a rootvg volume group, you cannot
extend it and you must create a new virtual disk.
The following steps describe how to extend a virtual disk:
1. On the operating system, stop any activity on the disk to be extended. If this is
not possible, shut down the partition. On AIX 5L, issue the varyoff command
on the volume group to which the disk belongs.
2. From the Storage Management menu in the IVM navigation area, click
3. From the work area, select the virtual disk and click Extend. Enter the disk
space to be added and click OK.
246
4. If the virtual disk is owned by a running partition, a warning message opens,
as shown in Figure 7-32. You must select the “Force extend on running
partition” check box to force the expansion. The additional disk space is
allocated to the virtual disk, but it is not available to the operating system.
Figure 7-32 Forced expansion of a virtual disk
5. From the Storage Management menu in the IVM navigation area, click
6. From the work area, select the virtual disk. Click Modify partition
assignment.
247
7. Unassign the virtual disk by selecting None in the New partition field. If the
disk is owned by a running partition, a warning message opens, as shown in
Figure 7-33. You must select the “Force device removal from running
operations” check box to force the expansion.
Figure 7-33 Forced unassignment of a virtual disk
8. Perform the same action as in step 5, but assign the virtual disk back to the
partition.
9. On the operating system, issue the appropriate procedure to recognize the
new disk size. On AIX 5L, issue the varyonvg command on the volume group
to which the disk belongs and, as suggested by a warning message, issue the
chvg -g command on the volume group to recompute the volume group size.
248
7.7.3 VIOS system disk mirroring
To prevent a VIOS outage due to system disk failure, make the rootvg storage
pool of the VIOS redundant. The default installation of VIOS uses only one
physical disk.
Disk mirroring on the VIOS is an advanced feature that, at the time of writing, is
not available on the Web GUI. You can configure it by using VIOS capabilities on
the CLI, but only system logical volumes can be mirrored. The following steps
describe how to provide a mirrored configuration for the rootvg storage pool.
Important: Mirrored logical volumes are not supported as virtual disks. This
procedure mirrors all logical volumes defined in rootvg and must not be run if
rootvg contains virtual disks.
1. Use the IVM to add a second disk to rootvg. From the Storage Management
menu in the navigation area, click Advanced View/Modify Devices.
2. Select the Physical Volumes tab. Select a disk not assigned to any storage
pool. Click Add to storage pool, as shown in Figure 7-34.
Figure 7-34 Adding a second disk to rootvg
249
3. In the Storage Pool field, select rootvg and click OK.
4. The actual mirroring is done using the IVM command line. Log in as the
padmin user ID and issue the mirrorios command, as shown in
Example 7-22. The command asks for confirmation and causes a VIOS
reboot to activate the configuration after performing data mirroring.
Example 7-22 rootvg mirroring at command line
$ mirrorios
This command causes a reboot. Continue [y|n]?
y
SHUTDOWN PROGRAM
Mon Aug 15 10:20:20 CDT 2005
Wait for 'Rebooting...' before stopping.
AIX 5L mirroring on the managed system LPARs
The AIX 5L logical volume manager is capable of data mirroring. You can also
use this feature when the partition is provided twice the number of virtual disks. A
VIOS administrator must create virtual storage that is used by AIX 5L for
mirroring purposes with careful consideration of data placement. The virtual
storage must not have any physical disks in common to avoid a disk failure that
affects both the mirror copies.
On the VIOS, virtual disks are created out of storage pools. They are created
using the minimum number of physical disks in the pool. If there is no sufficient
space on a single disk, they can span multiple disks. If the virtual disks are
expanded, the same allocation algorithm is applied. To guarantee mirror copy
separation, we recommend that you create two storage pools and create one
virtual disk from each of them. After virtual storage is created and made available
as an hdisk to AIX 5L, it is important to correctly map it.
On the IVM, the CLI is required. On the IVM, the lsmap command provides all the
mapping between each physical and virtual device. For each partition, there is a
separate stanza, as shown in Example 7-23. Each logical or physical volume
displayed in the IVM GUI is defined as a backing device, and the command
provides the virtual storage’s assigned LUN value.
250
Example 7-23 IVM command line mapping of virtual storage
$ lsmap -all
...
SVSA
Physloc
Client
Partition ID
--------------- ------------------------------------ ----------------vhost1
U9113.550.105E9DE-V1-C13
0x00000003
VTD
LUN
Backing device
Physloc
vtscsi1
0x8100000000000000
aixboot1
VTD
LUN
Backing device
Physloc
vtscsi2
0x8200000000000000
extlv
VTD
LUN
Backing device
Physloc
vtscsi3
0x8300000000000000
hdisk6
U787B.001.DNW108F-P1-T14-L5-L0
VTD
LUN
Backing device
Physloc
...
vtscsi4
0x8400000000000000
hdisk7
U787B.001.DNW108F-P1-T14-L8-L0
On AIX 5L, use the lscfg command to identify the hdisk using the same LUN
used by the IVM. Example 7-24 shows the command output with the 12-digit
hexadecimal number representing the virtual disk’s LUN number.
Example 7-24 Identification of AIX 5L virtual SCSI disk’s logical unit number
# lscfg -vpl hdisk0
hdisk0
U9113.550.105E9DE-V3-C2-T1-L810000000000
SCSI Disk Drive
Virtual
PLATFORM SPECIFIC
Name: disk
Node: disk
Device Type:
block
251
SCSI RAID adapter use
On a system equipped with an SCSI RAID adapter, you can protect data using
the adapter’s capabilities, avoiding any software mirroring. Use all physical disks
managed by each adapter’s SCSI chain to create a single RAID 5 array, and
install VIOS on it.
Configure the adapter to create the array before installing the VIOS. To do this
operation, boot the system with the stand-alone diagnostic CD and enter the
adapter’s setup menu as described in 6.12, “SAS hardware RAID configuration”
on page 170. After you create the array and finish formatting, install the VIOS.
During the installation, the VIOS partition’s rootvg is created on the array. Provide
disk space for logical partitions using the logical volumes created on the rootvg
storage pool.
Perform adapter maintenance using the IVM command line with the diagmenu
command to access diagnostic routines. Figure 7-35 shows the menu related to
the SCSI RAID adapter. It enables you to modify the array configuration and to
handle events such as the replacement of a failing physical disk.
Reconstruct a PCI-X SCSI Disk Array
Change/Show PCI-X SCSI pdisk Status
Diagnostics and Recovery Options
F1=Help
F9=Shell
F2=Refresh
F10=Exit
F3=Cancel
Enter=Do
Figure 7-35 The diagmenu menu for SCSI RAID adapter
252
F8=Image
7.8 Maintenance
This section provides information about maintenance operations on the IVM. It
discusses the following topics:
VIOS backup and restore
Logical partition backup and restore
VIOS upgrade
Managed system firmware update
VIOS migration
Command logging
7.8.1 VIOS maintenance
You can perform operations such as backup, restore, or upgrade using the IVM.
Some operations are available using the GUI or using the CLI, depending on the
type of operation.
Backing up and restoring the logical partition definitions
You can back up logical partition configuration information to a file. You can use
this file to restore information if required, and you can also export it to another
system. To back up the logical partition configuration:
1. From the Service Management menu in the navigation area, click
Backup/Restore.
2. Select Generate Backup in the work area, as shown in Figure 7-36. A file
named profile.bak is generated and stored under the user’s home directory.
There is only one unique backup file on the VIOS at a time, and a new backup
file replaces an existing one. In the work area, you can select this file name and
save it on your workstation, where the browser is running. This allows you to
specify a file name and save several copies.
253
Figure 7-36 Backing up the partition configuration
The backup file contains the logical partition’s configuration such as processors,
memory, and network and virtual disk assignment. The content of the virtual
disks is not included in the backup file.
To perform a restore operation, the system must not have any logical partition
configuration defined. If necessary, use the following command from the CLI
before the restore.
lpcfgop -o clear
254
In Figure 7-36, select the Restore Partition Configuration button to restore the
last backed up file. If you want to restore a backup file stored on your disk, do the
following steps:
1. Click Browse and locate the file.
2. Click Upload Backup File. The uploaded file replaces the existing backup
file.
3. Click Restore Partition Configuration to restore the uploaded backup file.
You can also back up and restore logical partition configuration information from
the CLI. Use the bkprofdata command to back up the configuration information
and the rstprofdata command to restore it. See the VIO Server and PLM
command descriptions in the Information Center at the following Web page for
more information:
http://publib.boulder.ibm.com/infocenter/eserver/v1r3s/index.jsp?topic=
/iphb1/iphb1_vios_commandslist.htm
Backing up and restoring the VIOS operating system
The only way to back up the VIOS operating system is with the backupios
command. No operating system backup operation is available within the GUI.
This command creates a bootable image that includes the VIOS partition’s
rootvg and can also contain the storage pool structure depending on the flags
used.
Important: You can use AIX or Linux commands to back up and restore the
virtual disks or physical volumes assigned to the logical partitions.
To create a backup that you can use for the NIM installation described in 7.1,
“VIOS installation in a JS21” on page 194, create a directory, for example,
backupios -file /home/padmin/ and use the following syntax:
backupios -file /home/padmin/backup_loc
When this is completed, a file nim_resources.tar is available in this directory.
Look at the file using the following command. Look for a file name such as
xxxx_mksysb:
tar -tf /home/padmin/backup_loc/nim_resources.tar
With VIOS 1.2.1, we find file 5300-04_mksysb in the nim_resources.tar file.
Extract this file and the file bosinst.data. Use these two files for a NIM installation.
Tip: The directory used for the backup can also be a NFS mounted directory,
for example, from the NIM server.
255
For further options of the backupios command, see Advanced POWER
Virtualization on IBM System p5, SG24-7940.
VIOS updates
To update VIOS to the latest fix pack, use the updateios command. Before it
installs the update, the updateios command makes a preview installation and
displays the results. You are prompted to continue or exit at the end of the
preview. Do not install the updates if the preview installation fails for any reason.
Assuming that you have located the update images in /home/padmin/update, the
command syntax is:
updateios -dev /home/padmin/update
The log file /var/adm/ras/install_all_updates.log is created.
Important: Ensure that you have the right level of firmware before updating
the VIOS.
7.8.2 Logical partition maintenance
Each logical partition hosted by the VIOS works like a stand-alone system. For
example, you can back up and restore, boot in maintenance mode, and perform
an operating system update or a migration.
Backing up the operating system
There are many ways to back up logical partitions hosted by the VIOS,
depending on the operating system installed. The main options for the AIX
operating system are:
The mksysb command creates a bootable image of the rootvg volume group
either in a file or onto a tape.
The mkcd command creates a system backup image (mksysb) to
CD-Recordable (CD-R) or DVD-Recordable (DVD-RAM) media from the
system rootvg or from a previously created mksysb image.
NIM creates a system backup image from a logical partition rootvg using the
network.
256
Restoring the operating system
The restoration process is exactly the same as on stand-alone systems. The
main steps are:
1. Log on to the VIOS.
command, providing the ID of the logical partition to be restored.
3. Start the logical partition in SMS mode.
4. Select the boot device that you used for the backup such as CD, DVD-RAM,
or network.
5. Boot the logical partition.
6. Follow the specific operating system’s restore procedures.
7.8.3 Command logs
All IVM actions are logged on the system. The log contains all the commands
that the IVM Web GUI runs and all IVM-specific commands issued by an
administrator on the command line.
The log contains the following information for each action:
User name
Date and time
The command including all the parameters
The following steps describe how to access the log:
1. From the Service Management menu in the navigation area, click
Application Logs.
2. In the work area, use the provided filters to restrict the log search. Click
Apply.
257
This generates the selected log entries, as shown in Figure 7-37.
Figure 7-37 Command logs
258
8
Chapter 8.
Installing AIX
This chapter describes the installation of AIX on an IBM BladeCenter JS21. The
BladeCenter JS21 supports the installation of AIX 5L Version 5.3 Technology
Level 5300-04 and later. We discuss the following basic topics:
General topics
AIX installation from CD
Preparing a Network Installation Manager (NIM) master
AIX network installation using NIM
259
8.1 General topics
The simplest method to install AIX is installing from the CD/DVD. However, we
recommend the network installation procedure. Typical blade server
environments and logical partitioning require many operating system
installations. Installing over a network saves time and can be fully automated
without the need to mount physical media.
AIX offers an easy to configure network installation server called NIM, which you
can configure on one or more machines in the network. The so-called NIM
master can either be outside the BladeCenter environment or even an IBM
BladeCenter JS20 or JS21. It must be a machine running AIX 5L. If the NIM
master is the first machine running AIX 5L in the given environment, you have to
install it from CD, DVD, or tape.
Note: Before you install any operating system, we recommend that you
upgrade all firmware to the latest available level. You can apply the Flash
BIOS Update of the JS21 itself and the Broadcom Firmware Update for the
integrated Ethernet adapter only by using an already installed operating
system (AIX or Linux) or with the stand-alone diagnostics booted from CD or
network (NIM). See 6.6, “Firmware”, for further information.
8.2 Preparing AIX installation from CD/DVD
To prepare an AIX installation from CD/DVD, assign the media tray to the blade
server that you want, as described in 6.5.3, “Assigning the media tray” on
page 94. Mount the first volume of the AIX 5L installation CDs/DVDs in the media
tray’s optical drive. Continue with the procedure as described in “Booting from
CD/DVD” on page 145, and in 8.4, “Installing AIX on the client” on page 270.
260
8.3 Preparing AIX network installation using NIM
In this section, we describe how to set up the NIM master on a machine that is
already running AIX. This can be a machine outside the BladeCenter
environment or JS20 or JS21.
Note: If a NIM master already exists, define your client installation on the
existing NIM master and proceed with 8.4, “Installing AIX on the client” on
page 270. Be aware that the NIM master’s AIX version and technology level
(maintenance level) must be at least the same level as the level of the AIX it
offers to the client for installation. You can check this using one of the
following commands:
oslevel -rf
instfix -i | grep AIX_ML
Use the following command to see the service pack level:
/usr/bin/oslevel -s
8.3.1 Configuring AIX environment
We recommend that you create at least one extra file system for the NIM data.
By default NIM uses the /export directory. You can use the following command or
smitty manfs to create this file system.
# crfs -v jfs2 -g rootvg -a size=2G -m /export -A yes
Note: You can change rootvg and size, if necessary.
If you want to create a separate file system for the different NIM resources, refer
to 8.3.3, “Configuring the NIM master” on page 263. The NIM setup automatically
creates these file systems, if necessary. See the options Create new filesystem
for LPP_SOURCE and Create new filesystem for Shared Product Object Tree
(SPOT) in Figure 8-6 on page 267.
Chapter 8. Installing AIX
261
8.3.2 Installing the NIM filesets
While the fileset bos.sysmgt.nim.client is automatically installed with AIX, you
have to install the following filesets:
bos.sysmgt.nim.spot
bos.sysmgt.nim.master
You can do this by using smitty install_latest and selecting the highlighted
filesets in Figure 8-1.
bos.sysmgt
5.3.0.40 Filesystem Quota Commands
5.3.0.0 License Management
5.3.0.40 Network Install Manager - Client Tools
+ 5.3.0.40 Network Install Manager - Master Tools
+ 5.3.0.30 Network Install Manager - SPOT
5.3.0.40 Software Error Logging and Dump Service Aids
5.3.0.40 Software Trace Service Aids
@
@
@
>
>
@
@
Figure 8-1 Selecting the NIM fileset
262
ALL
8.3.3 Configuring the NIM master
There are several ways to configure the NIM master. The examples in this
section show the text-based System Management Interface Tool (SMIT)
interface. Refer to The IBM eServer BladeCenter JS20, SG24-6342, for
information about how to use Web-based Systems Manager (WebSM).
Note: You can also perform the commands that we issue using SMIT from the
command line. Press the F8 key in SMIT to display the used command.
However, most of the NIM functions are performed by a more complex script
in SMIT. In this example, it is more convenient to copy the appropriate entry
from $HOME/smit.script to another file, for example, /tmp/smit.test. The entry
might start with:
#
#
[May 30 2006, 20:51:47]
#
mkres()
At the end, you can find the following entry:
nim -o ...
exit $?
}
mkres ...
Insert an echo statement in the beginning of the line with the nim -o ...
command:
echo nim -o ...
Save the file and issue the script:
ksh /tmp/smit.test
This shows the used command.
263
You can access all of the tasks using smitty nim. Figure 8-2 shows the SMIT
screen.
Network Installation Management
Configure the NIM Environment
Perform NIM Software Installation and Maintenance Tasks
Perform NIM Administration Tasks
Create IPL ROM Emulation Media
NIM POWER5 Tools
F1=Help
F9=Shell
F2=Refresh
F10=Exit
F3=Cancel
Enter=Do
F8=Image
Figure 8-2 NIM: Main SMIT menu
Select the option Configure the NIM Environment. This opens the Configure a
Basic NIM Environment (Easy Startup) screen shown in Figure 8-3.
264
Configure a Basic NIM Environment (Easy Startup)
[TOP]
Initialize the NIM Master:
* Primary Network Interface for the NIM Master
Basic Installation Resources:
* Input device for installation images
* LPP_SOURCE Name
* LPP_SOURCE Directory
Create new filesystem for LPP_SOURCE?
Filesystem SIZE (MB)
VOLUME GROUP for new filesystem
* SPOT Name
* SPOT Directory
Create new filesystem for SPOT?
[Entry Fields]
[en0] +
[cd0] +
[lpp_source_aix_534]
[/export/lpp_source] +
[no] +
[650] #
[rootvg] +
[spot_aix_534]
[/export/spot] +
[no] +
[350] #
[rootvg] +
Create Diskless/Dataless Machine Resources?
Specify Resource Name to Define:
ROOT
(required for diskless and dataless)
DUMP
(required for diskless and dataless)
PAGING (required for diskless)
HOME
(optional)
SHARED_HOME (optional)
TMP
(optional)
Diskless/Dataless resource directory
Create new filesystem for resources?
[no] +
Define NIM System Bundles?
Define NIM bosinst_data?
Prepend level to resource name
[yes] +
[yes] +
[no] +
Add Machines from a Definition File?
Specify Filename
[no] +
[]
* Remove all newly added NIM definitions
and filesystems if any part of this
operation fails?
[BOTTOM]
[no] +
[root1]
[dump1]
[paging1]
[home1]
[shared_home1]
[tmp1]
[/export/dd_resource]
[yes] +
[150] #
[rootvg] +
Figure 8-3 NIM: Easy startup screen
265
8.3.4 Defining the NIM client
From the NIM main SMIT menu (Figure 8-2 on page 264), select Perform NIM
Administration Tasks → Manage Machines → Define a Machine. This opens
the screen shown in Figure 8-4. On this screen, you have to enter the host name
of the client machine.
Define a Machine
Type or select a value for the entry field.
[Entry
Fields]
* Host Name of Machine
(Primary Network Install Interface)
[js21client]
Figure 8-4 NIM: Selecting the client’s host name
Attention: The NIM master must be able to resolve the client’s Internet
Protocol (IP) address from the host name. In our example, the host name is
js21client. This can be from a name server, /etc/hosts, or another supported
resolver method. When you use /etc/hosts, list the short name in addition to
the fully qualified host name:
10.1.1.10
js21client.ibm.com
js21client
When you press Enter, and if the NIM master cannot resolve the js21client
name, you see the screen shown in Figure 8-5 in the SMIT.
Type of Network Attached to Primary Network Install Interface
tok
ent
fddi
generic
atm
=
=
=
=
=
token ring network
ethernet network
FDDI network
generic network (no network boot capability)
ATM network
Figure 8-5 NIM SMIT screen when the client’s host name cannot be resolved
266
If the host name resolution is successful, you see the screen shown in
Figure 8-6.
Define a Machine
[Entry
Fields]
* NIM Machine Name
* Machine Type
* Hardware Platform Type
Kernel to use for Network Boot
Communication Protocol used by client
Primary Network Install Interface
*
Cable Type
Network Speed Setting
Network Duplex Setting
*
NIM Network
*
Host Name
Network Adapter Hardware Address
Network Adapter Logical Device Name
IPL ROM Emulation Device
CPU Id
Machine Group
Comments
[js21client]
[standalone] +
[chrp] +
[mp] +
[] +
tp +
[] +
[] +
network1
js21nim
[001125c9113c]
[]
[] +/
[]
[] +
[]
Figure 8-6 NIM: Defining a machine
Tip: It is optional to set the “Network Adapter Hardware Address”. Leaving it
as “0” requires that you enter the client’s IP address and the NIM master’s IP
address at the client’s firmware boot menu (directed Bootstrap Protocol
(BOOTP)). When the client’s hardware Medium Access Control (MAC)
address is specified in this screen, no input is necessary at the client
(broadcast BOOTP). See 6.9.1, “Activating the Open Firmware interface” for
further information.
267
8.3.5 Allocating resources to a client
After you define the client, you have to allocate the NIM resources to prepare the
installation. From the NIM main SMIT menu (Figure 8-2 on page 264), select
Perform NIM Administration Tasks → Manage Machines → Manage
Network Install Resource Allocation → Allocate Network Install Resources.
This opens the screen shown in Figure 8-7.
Available Network Install Resources
Move cursor to desired item and press F7.
Press Enter AFTER making all selections.
[TOP]
> lpp_source_aix_534
> spot_aix_534
lpp_source
spot
Figure 8-7 NIM: Selecting the resources
Tip: For a basic installation, you have to select at least a SPOT and an
lpp_resource.
8.3.6 Activating the client installation
Now you have to activate the client installation at the NIM master. From the NIM
main SMIT menu (Figure 8-2 on page 264), select Perform NIM Administration
Tasks → Manage Machines → Perform Operations on Machines. This opens
the screen shown in Figure 8-8. On this screen, select perform a BOS
Installation.
268
Operation to Perform
diag
= enable a machine to boot a diagnostic
image
cust
= perform software customization
bos_inst
= perform a BOS installation
maint
= perform software maintenance
reset
= reset an object's NIM state
fix_query
= perform queries on installed fixes
check
= check the status of a NIM object
reboot
= reboot specified machines
maint_boot
= enable a machine to boot in maintenance
mode
showlog
= display a log in the NIM environment
lppchk
= verify installed filesets
restvg
= perform a restvg operation
update_all = update all currently installed filesets
Figure 8-8 NIM: Selecting the client operation
This opens the screen shown in Figure 8-9.
Perform a Network Install
[Entry
Fields]
Target Name
Source for BOS Runtime Files
installp Flags
Fileset Names
Remain NIM client after install?
Initiate Boot Operation on Client?
Set Boot List if Boot not Initiated on Client?
Force Unattended Installation Enablement?
ACCEPT new license agreements?
js21client
rte
[-agX]
[]
yes
no
no
no
[no]
Figure 8-9 NIM: Performing the client installation operation
269
The NIM master is now configured and waiting for the client’s BOOTP request.
Attention: If an operation is already defined at the client, you see the
following error:
0042-302 m_bos_inst: the state of "js21a2" prevents this operation
In this case, use the reset operation as shown in Figure 8-8 on page 269.
8.4 Installing AIX on the client
You have to boot the BladeCenter JS21 from an installation media as described
in 6.8.2, “Setting up remote initial program load” on page 135, because this is
independent of a specific operating system. When the boot is successful, you
see a message similar to Figure 8-10.
Tip: The following description shows the manual setup of the client’s
installation dialog. To avoid manual input on every client, NIM offers a
non-prompted installation method also. See the following Web site:
http://publib.boulder.ibm.com/infocenter/pseries
Select AIX documentation → Installation and migration → Installing with
Network Installation Management → Configuring NIM and other basic
operations → Performing a nonprompted BOS installation.
Alternatively, refer to NIM: From A to Z in AIX 4.3, SG24-5524.
--------------------------------------------------------------------------Welcome to AIX.
boot image timestamp: 03:00 05/15
The current time and date: 16:07:33 05/16/2006
number of processors: 4
size of memory: 3968MB
boot device:
/pci@8000000f8000000/pci@2/ethernet@4,1:9.3.5.228,,9.3.5.231,0.0.0.0,00,00
kernel size: 11000928; 32 bit kernel
--------------------------------------------------------------------------Figure 8-10 AIX welcome screen
270
1. After a short time, the installation dialog opens as shown in Figure 8-11.
Select the console by entering 1 followed by the Enter key.
******* Please define the System Console. *******
Type a 1 and press Enter to use this terminal as the
system console.
Pour definir ce terminal comme console systeme, appuyez
sur 1 puis sur Entree.
Taste 1 und anschliessend die Eingabetaste druecken, um
diese Datenstation als Systemkonsole zu verwenden.
Premere il tasto 1 ed Invio per usare questo terminal
come console.
Escriba 1 y pulse Intro para utilizar esta terminal como
consola del sistema.
Escriviu 1 1 i premeu Intro per utilitzar aquest
terminal com a consola del sistema.
Digite um 1 e pressione Enter para utilizar este terminal
como console do sistema.
Figure 8-11 Installation language panel
2. On the second panel, select the necessary language, as shown in
Figure 8-12.
>>> 1 Type 1 and press Enter to have English during install.
88 Help ?
>>> Choice [1]:
Figure 8-12 Selecting the language
Tip: If the correct number for the selection that you want is already
displayed between the square brackets (default), press Enter without
entering the number.
271
3. In the panel shown in Figure 8-13, select option 2, review the default settings,
and make the necessary changes.
Welcome to Base Operating System
Installation and Maintenance
Type the number of your choice and press Enter. Choice is indicated
by >>>.
>>> 1 Start Install Now with Default Settings
2 Change/Show Installation Settings and Install
3 Start Maintenance Mode for System Recovery
88 Help ?
99 Previous Menu
>>> Choice [1]: 2
Figure 8-13 Selecting the installation and maintenance
272
4. In the Installation and Settings window shown in Figure 8-14, it is important to
check the Method of Installation and the Disk Where You Want to Install
(option 1). If there is an AIX system on the target hard disk already, the
default Method of Installation might be Preservation instead of New and
Complete Overwrite. Be sure to make the appropriate selection.
Installation and Settings
Either type 0 and press Enter to install with current settings, or type the
number of the setting you want to change and press Enter.
1
System Settings:
Method of Installation.............New and Complete Overwrite
Disk Where You Want to Install.....hdisk0
2
Primary Language Environment Settings (AFTER
Cultural Convention................English
Language...........................English
Keyboard...........................English
Keyboard Type......................Default
3
More Options
>>> 0
Install):
(United States)
(United States)
(United States)
(Desktop, Security, Kernel, Software, ...)
Install with the settings listed above.
88 Help ?
99 Previous Menu
+----------------------------------------------------| WARNING: Base Operating System Installation will
|
destroy or impair recovery of ALL data on the
|
destination disk hdisk0.
>>> Choice [0]:1
Figure 8-14 Installation and language options
Attention: If the target hard disk already contains valuable data, this data is
overwritten or modified.
273
5. After you select 1 to choose the System Settings in the previous menu, the
screen shown in Figure 8-15 opens. You can select the installation method
that you want.
Change Method of Installation
Type the number of the installation method and press Enter.
>>> 1 New and Complete Overwrite
Overwrites EVERYTHING on the disk selected for installation.
Warning: Only use this method if the disk is totally empty or
if there is nothing on the disk you want to preserve.
2 Preservation Install
Preserves SOME of the existing data on the disk selected for
installation. Warning: This method overwrites the usr (/usr),
variable (/var), temporary (/tmp), and root (/) file systems.
Other product (applications) files and configuration data will
be destroyed.
88 Help ?
99 Previous Menu
>>> Choice [1]:
Figure 8-15 Changing the method of installation
274
6. In Figure 8-16, you can select the target hard disk. If you enter the number of
a disk, this toggles the selection state. You can enter more than one number
at a time separated by blanks.
Change Disk(s) Where You Want to Install
Type one or more numbers for the disk(s) to be used for installation
and press
Enter. To cancel a choice, type the corresponding number and Press
Enter.
At least one bootable disk must be selected. The current choice is
indicated
by >>>.
Name
>>>
>>>
1
2
hdisk0
hdisk1
Location Code
01-08-00-1,0
01-08-01-1,0
Size(MB)
34715
34715
VG Status
rootvg
none
Bootable
Yes
Yes
No
No
0
Continue with choices indicated above
55 More Disk Options
66 Disks not known to Base Operating System Installation
77 Display Alternative Disk Attributes
88 Help ?
99 Previous Menu
>>> Choice [0]:
Figure 8-16 Changing the target hard disks
Important: You have to verify the actual location code (for example,
01-08-00-1,0) of the hard disk. You have to do this because the logical name
for the hard disks (for example, hdisk0) that is displayed in this menu can be
different from the logical name for the same hard disk that is listed within the
AIX operating system (for example, from the lspv command) that runs on the
same machine. This can happen when disks are added after AIX is installed.
275
8.5 Changing client from rsh to nimsh
If you decide that the client is manageable by the NIM server even after the
successful installation is complete and AIX is running, choose the option Remain
NIM client after install? = yes when activating the client installation on the NIM
server as shown in Figure 8-9 on page 269.
When you define the client, you can select the Communication Protocol used
by client. Figure 8-17 shows the corresponding SMIT screen and the selection
box that you get when you press F4 on this item. You can select shell, which
uses the insecure remote shell (rsh) protocol, and nimsh, which uses the more
secure Secure Sockets Layer (SSL) protocol.
Define a Machine
[Entry Fields]
* NIM Machine Name
[js21client]
* Machine Type
[standalone]
+
* Hardware Platform Type
[chrp]
+
Kernel to use for Network Boot
[mp]
+
[]
+
Primary Network Install Interface
* ????????????????????????????????????????????????????????????????????
?
?
?
?
?
?
?
?
shell = RSH Protocol is used by client
?
?
nimsh = NIM Service Handler is used by client
?
?
?
? F1=Help
F2=Refresh
F3=Cancel
?
F1? F8=Image
F10=Exit
Enter=Do
?
F5? /=Find
n=Find Next
?
F9????????????????????????????????????????????????????????????????????
Figure 8-17 NIM communication protocol used by the client
When you choose nimsh, a daemon named nimsh runs on the client. The
communication with the NIM server uses SSL. For this, you must install SSL on
the server and the client. To do this, use:
# rpm -i openssl-0.9.7g-1.aix5.1.ppc.rpm
276
The OpenSSL Red Hat Package Manager (RPM) package exists on the AIX 5L
Linux Toolbox CD-ROM. It might have a different version than our example. You
can download the RPM from the following Web site:
http://www-1.ibm.com/servers/aix/products/aixos/linux/download.html
You have to activate SSL on the NIM server. To do this:
1. Select smitty nim → Perform NIM Administration Tasks → Configure NIM
Environment Options → Enable Cryptographic Authentication.
2. Change the option Enable Cryptographic Authentication for client
communication to enable as highlighted in Figure 8-18.
Enable Cryptographic Authentication
[Entry
Fields]
*
for client communication?
*
[enable]
Install Secure Socket Layer Software (SSLv3)?
Absolute path location for RPM package
-ORlpp_source which contains RPM package
[no]
[/dev/cd0]
DISPLAY verbose output?
[no]
F1=Help
F5=Reset
F9=Shell
F2=Refresh
F6=Command
F10=Exit
[]
F3=Cancel
F7=Edit
Enter=Do
F4=List
F8=Image
Figure 8-18 NIM: Enabling cryptographic authentication on the server
3. On the client, select smitty nim → Configure Client Communication
Services → Configure Client Communication Services.
277
4. Change Communication Protocol used by client to nimsh and Enable
Cryptographic Authentication for client communication to enable as
shown by the highlighted values in Figure 8-19.
Configure Client Communication Services
[Entry
Fields]
* Communication Protocol used by client
*
[nimsh]
NIM Service Handler Options
for client communication?
[enable]
Install Secure Socket Layer Software (SSLv3)?
Absolute path location for RPM package
-ORlpp_source which contains RPM package
Alternate Port Range for Secondary Connections
(reserved values will be used if left blank)
Secondary Port Number
Port Increment Range
F1=Help
F5=Reset
F9=Shell
F2=Refresh
F6=Command
[no]
[/dev/cd0]
[] +
[]
[]
F3=Cancel
F7=Edit
F10=Exit
Enter=Do
Figure 8-19 NIM: Enabling cryptographic authentication on the client
278
F4=List
F8=Image
5. To verify that it works, log in to the NIM server and issue the lsnim command
as shown in Example 8-1.
Example 8-1 lsnim command
# lsnim –l js21client
js21client:
...
connect = nimsh (secure)
...
The line containing connect = nimsh must have the word secure at the end.
Attention: It is possible that on a JS21 the CPUID is displayed with all zeroes.
You can test this with the following command:
uname -m
In this case, you have to update the BIOS of the JS21 to at least Version
240.470.014. If you get the following error from NIM on the client, this might be
the reason.
0042-172 NIMkid: This machine's CPU ID does not match the CPU ID
stored in the NIM database.
279
280
9
Chapter 9.
Installing Linux
This chapter describes the procedures to install Linux on a BladeCenter JS21.
The focus is on network installation, but we also provide some hints to install
Linux using a CD/DVD. One part of this chapter covers basic generic
descriptions about how to set up the infrastructure for a network installation. This
information is independent of the operating system and the hardware
architecture.
In general, it is much easier to install and maintain computer systems if an
appropriate network installation infrastructure is configured. This is especially
true for BladeCenter JS21, because BladeCenter JS21 is designed to be used in
scenarios with a high server density and different operating systems. The
BladeCenter JS21 and the related distributions are prepared to meet these
concerns.
AIX uses similar services and protocols, and the Network Installation Manager
(NIM) utility provided by any AIX installation uses these services and protocols to
enable AIX installations using a network. Hence it might be possible to use a
running AIX server to install Linux using the network. However, this book does
not cover this topic.
281
One part covers the network installation of the SUSE Linux Enterprise Server 9
(SLES9) Service Pack 3 (SP3). This chapter also provides a description of how
to perform a network installation of a Red Hat Enterprise Linux (RHEL) AS 4
Update 3 (U3).
Important: Before installing any operating system, update all firmware to the
latest level. See 6.6, “Firmware” on page 94 for instructions about how to
complete this task.
Note regarding VIOS: All procedures described in this chapter are also true
for Linux installations on logical partitions (LPARs) on BladeCenter JS21, but
there might be some minor changes in some actions or procedures. Especially
the soft reboot function does not always work if Linux is running on an LPAR.
In all cases, use the System Management Services (SMS) menu instead of
the BladeCenter management module. For more information about VIOS and
LPARs, see Chapter 7, “Installing and managing the Virtual I/O Server” on
page 193.
9.1 Installing Linux from a CD/DVD
For testing or as a single application server, it might be adequate to use the
CD/DVD device of a BladeCenter to load the operating system. The general
preparation steps and instructions about using the Serial over LAN (SoL) console
to install an operation system using CD/DVD on a BladeCenter JS21 are
described in 6.7, “Providing a console for the BladeCenter JS21” on page 121.
You can transfer some of the information provided in the following sections to a
CD/DVD installation, but most information such as preparing an infrastructure or
configuring the boot image are only helpful for a network installation.
282
Important for Red Hat Enterprise Linux installation: At the time of
publication, the installation of the Red Hat Enterprise Linux4 U3 using local
CD/DVD device on both the BladeCenter JS20 and JS21 requires that you
insert a non-bootable diskette in the diskette drive of the media tray. This is
posted on the Red Hat Knowledge base, which is available on the Web at:
http://www.redhat.com/search/ui.jsp
Also note that during the initial startup of the installation, Red Hat provides the
opportunity to run a media check on the CD media before using the CD for the
installation. If you select this option, it runs the check and then ejects the
media at the completion of the check. You have to reinsert the CD tray before
starting the installation.
Tip for Microsoft Windows Telnet users: The Serial over LAN Setup Guide
provides instructions about how to use the Telnet command to begin your
console sessions. You can find this guide at the following Web site:
http://www-307.ibm.com/pc/support/site.wss/document.do?sitestyle=ibm
&lndocid=MIGR-54666&velxr-layout=print
An alternative to using the Telnet command is the putty.exe, which you can
download from multiple Internet sources. Keyboard usage is much easier to
follow and alter than with Windows Telnet, and monitoring the screen output is
easier.
9.2 Basic preparations for a Linux network installation
This section provides all the basic information to set up services for a Linux
network installation. In principle, this is not bound to the operation system or
distribution, which runs on the infrastructure server to provide the necessary
services. Nevertheless, all descriptions in this section are based on general
Linux services, commands, and parameters. We presume that the required files
for all the services are already installed and that all the commands are issued
with superuser rights.
Chapter 9. Installing Linux
283
9.2.1 Installing Linux using the network: General remarks
You always require the following services to perform a network installation:
A running Bootstrap Protocol (BOOTP) service or a Dynamic Host
Configuration Protocol (DHCP) service that includes BOOTP support to
configure the network interface of a BladeCenter JS21
A running Trivial File Transfer Protocol (TFTP) service to serve the boot
image to a BladeCenter JS21
It is necessary to set up one the following services to provide the installation
packages for a network installation after the boot image is loaded:
– File Transfer Protocol (FTP): This is the protocol that causes the fewest
problems during the installation. It has good performance and is supported
by all operating systems. If you use this protocol as a read-only source,
the lack of security features might not be a problem.
– Hypertext Markup Language (HTML): If a Web server is already running,
this protocol might even be better than FTP. It is beyond the scope of this
book to cover the special features of an HTML installation.
– Network File System (NFS): Even with its special features, this protocol
is not a good choice, because of some incompatibilities between different
versions of NFS servers and clients.
– Server Message Block (SMB)/Common Internet File System (CIFS): If
Microsoft Windows is already running or the Samba Server is used as a
file server, you can use this server to serve the necessary data to a
BladeCenter JS21. But this protocol generates a lot of overhead and
supports starts with SLES10.
Note: If a firewall is running on your installation server, ensure that you update
the settings to allow traffic for your installation protocol.
284
9.2.2 Configuring a BOOTP or DHCP service
DHCP is an extension to the original BOOTP specification. As a result, you can
use DHCP to provide the BOOTP information for booting using the network. The
standard DHCP daemon is called dhcpd, but there are other DHCP daemons.
Note: The directory you use for the configuration files depends on the
distribution. The following directories are possible examples:
/etc/
/etc/sysconfig/
/etc/default/
/etc/xinet.d/ (eXtended InterNET daemon configuration files)
The examples in this chapter use the most common directories. In general,
the name of a configuration or script file is related to the name of the installed
package. For example, if a DHCP daemon is called dhcpd3-server, you can
find the configuration in /etc/dhcpd3-server.conf and
/etc/sysconfig/dhcpd3-server, and the start/stop script is in
/etc/init.d/dhcp3-server.
The standard DHCP daemon is configured through two files: The
/etc/sysconfig/dhcpd file, which stores the basic configuration, and the
/etc/dhcpd.conf file, which contains the configuration information for each
registered client. For a running service, the actual used configuration in most
cases is copied in a subdirectory of /var/. See Example 9-1 for a simple client
configuration stored in dhcpd.conf.
Attention: Keep in mind that Example 9-1 contains environmental-specific
Internet Protocol (IP) and Media Access Control (MAC) address information.
One way to learn the MAC address of a BladeCenter JS21 is to use the
BladeCenter management module. Select Monitors → Hardware VPD and
scroll down to the bottom of the Web page (see Figure 6-10 on page 81). For
more information about how to customize dhcpd.conf, see the man pages of
the dhcpd service and use man dhcpd on the command prompt.
Example 9-1 dhcpd.conf
always-reply-rfc1048 true;
allow bootp;
deny unknown-clients;
not authoritative;
default-lease-time 600;
285
max-lease-time 7200;
subnet 192.168.1.0 netmask 255.255.255.0 {
group {
next-server 192.168.1.254;
host js21-eth0 {
hardware ethernet 00:11:25:C9:0B:A6;
}
host js21-eth1 {
hardware ethernet 00:11:25:C9:0B:A7;
filename "/tftpboot/sles9_sp3_basic";
}
}
}
You can find the start and stop scripts of Linux services in the /etc/init.d/
directory. To start the standard DHCP daemon, use the /etc/init.d/dhcpd
start command. To restart the DHCP daemon, use the /etc/init.d/dhcpd
restart command.
Tip for Linux beginners: The following tasks help you to double-check or
troubleshoot a configuration in general.
To trace messages of running services, type tail -f -n 10
/var/log/messages to get the last 10 messages and auto update if there
are new messages.
Connect to a running service with a local client, remote client, or both these
clients and try to receive the data that you want.
Make sure a changed configuration is activated by restarting a service
directly after editing, for example:
a. vi /etc/dhcpd.conf
b. /etc/init.d/dhcpd restart
286
9.2.3 Configuring a Trivial File Transfer Protocol service
You can use the TFTP to provide a bootable image during a network installation.
There are several implementations of TFTP daemons available. The standard
TFTP daemon is called tftpd. In general, the xinetd or inetd super daemon is
used to create a TFTP daemon. You can also run a TFTP daemon without one of
the super daemons.
A TFTP daemon configuration for inetd is stored in inetd.conf or /etc/default/tftpd.
An example is shown in Example 9-2.
Example 9-2 Configuring a TFTP daemon in the /etc/inetd.conf file
tftp dgram udp wait nobody /usr/sbin/tcpd /usr/sbin/in.tftpd
--tftpd-timeout 300 --retry-timeout 5 --mcast-port 1758 --mcast-addr
239.239.239.0-255 --mcast-ttl 1 --maxthread 100 --verbose=5 /srv/tftp
Example 9-3 shows a TFTP daemon configuration for xinetd stored in
/etc/xinet.d/tftpd.
Example 9-3 Configuring a TFTP daemon in the /etc/xinet.d/tftp file
service tftp {
protocol = udp
port = 69
socket_type = dgram
wait = yes
user = nobody
group = nobody
server = /usr/sbin/in.tftpd
server_args = -vvv -s /srv/tftp
only_from = 192.168.1.0
disable = no
}
Independent of the used operating system or TFTP daemon, it is important to
configure the public TFTP directory containing the bootable image. In
Example 9-2 and Example 9-3, the setting of the public TFTP directory is marked
in bold. If you start the TFTP daemon without one of the super daemons, the
location of the configuration file depends on the distribution.
287
Red Hat Enterprise Linux and SLES create TFTP from the xinetd daemon. To
obtain a list of services started by xinet run the chkconfig --list command. By
default, TFTP is disabled on both distributions. To enable both xinetd and TFTP,
enter:
1. chkconfig tftp on
2. /etc/init.d/xinetd restart
Note: The chkconfig command changes the disable parameter in the
configuration file of a service. The configuration file is stored in the /etc/xinet.d/
directory. To check the status of a service controlled by xinetd, issue
chkconfig without any parameters.
In the /srv/tftp/tftpboot directory, we copy the bootable installation kernel later in
this chapter. The bootable kernel must match the Linux distribution that you
install. It must also match the file name listed in the dhcpd.conf, as shown in
Example 9-1 on page 285.
9.2.4 Configuring a File Transfer Protocol service
In this book, we use FTP to serve installation data to the BladeCenter JS21.
SLES includes a simple and basic FTP service setup if it is configured as an
installation server using Yet Another Setup Tool (YaST1). See 9.3.2, “Preparing
the installation source directory and service” on page 293. If you prefer to
configure the service manually, follow the presented instruction.
The standard FTP daemon is called ftpd, but there are other FTP daemons
available. In this book, we use vsftpd. Similar to the TFTP daemon, there are
several ways to start and configure an FTP service. In this case, we use the
xinetd super daemon to start the FTP daemon. Therefore, the network
configuration is stored in /etc/xinet.d/vsftpd. See Example 9-4.
Example 9-4 Configuring an FTP daemon in the /etc/xinet.d/vsftpd file
# description:
#
The vsftpd FTP server serves FTP connections. It uses
#
normal, unencrypted usernames and passwords for authentication.
# vsftpd is designed to be secure.
service ftp
{
#
server_args
=
#
log_on_success
+= DURATION USERID
#
log_on_failure
+= USERID
#
nice
= 10
1
288
YaST provides an interface to install and configure a SUSE Linux Enterprise Server
socket_type
protocol
wait
user
server
=
=
=
=
=
stream
tcp
no
root
/usr/sbin/vsftpd
The main configuration file is /etc/vsftpd.conf or /etc/vsftpd/vsftpd.conf. A cutout
is presented in Example 9-5. In most cases, for security reasons, an FTP service
must provide data read-only for anonymous users. The most important
customization in the public ftp directory2, which later stores the installation
source files or other public accessible data, is defined by the following setting:
anon_root=/srv/ftp/
This is highlighted in bold in Example 9-5. To enable the FTP service and restart
xinetd use these commands:
1. chkconfig vsftpd on
2. /etc/init.d/xinetd restart
Example 9-5 Configuring an FTP daemon in the /etc/vsftpd.conf file
.
.
# Anonymous FTP user Settings
#
# Allow anonymous FTP?
#
anonymous_enable=YES
#
# Anonymous users will only be allowed to download files which are
# world readable.
#
anon_root=/srv/ftp/
#
# Public directory for anonymous user
#
anon_world_readable_only=YES
#
2
This is the directory for anonymous login
289
# Uncomment this to allow the anonymous FTP user to upload files. This
only
# has an effect if the above global write enable is activated. Also,
you will
# obviously need to create a directory writable by the FTP user.
#
.
.
Tip: To test if an FTP or TFTP service is running, use the following command
on the FTP or TFTP server and look for LISTEN connections:
netstat -a |grep ftp
If you can connect to the FTP server with an FTP client or even download
data, but later the installation stops right after booting from the boot image, in
most cases, the installation directory is not set up correctly. See 9.4.2,
“Preparing the installation source for Red Hat Enterprise Linux” on page 324.
The next step is the preparation of the installation source directory and the
corresponding service. Here the preparation depends on the distribution to be
installed. Therefore, we document this in separate sections: 9.3, “Installing SLES
using the network”, and 9.4, “Installing Red Hat Enterprise Linux AS 4 Update 3
using the network” on page 321.
9.3 Installing SLES using the network
This section focuses on the SLES9 SP3. We emphasize special issues regarding
SLES10. The prerequisites to perform an SLES installation using network are:
A configured Linux installation infrastructure (see 9.2, “Basic preparations for
a Linux network installation” on page 283)
Configuring the boot sequence and the boot options of a BladeCenter JS21,
especially the Ethernet interface configuration, appropriately (see Chapter 6,
“Hardware setup” on page 63)
Note: SLES switches the boot sequence of the boot devices automatically
from Network - BOOTP to Hard drive X after the installation. Here X can be in
the range from 0 to 3.
290
9.3.1 Overview of SLES network installation
The basic preparations for the SLES network installation are mentioned in 9.3,
“Installing SLES using the network” on page 290. Define the installation server
as described in 9.3.2, “Preparing the installation source directory and service” on
page 293.
After this final step, the basic preparation for the network installation is complete.
There are different ways (shown as decision symbols in Figure 9-1 on page 292)
to install SLES:
Decide whether you want to define the boot parameter such as installation
server, installation source directory, or transfer protocol using the Open
Firmware prompt, also called Open Firmware interface. To learn more about
the Open Firmware prompt in general, see 6.9, “Open Firmware interface” on
page 152. Information is also available in 9.3.5, “Unattended installation with
SLES” on page 311. In most cases, the Open Firmware method is not used to
install SLES. Therefore, for a more sophisticated method, see the following
section.
You can predefine the installation server, installation source directory, and
transfer protocol. You can do this if the installation is attended or unattended.
To predefine the parameters by saving the necessary information in a boot
image, see 9.3.4, “Configuring the boot image file with mkzimage_cmdline”
on page 306.
This basic installation procedure is described in 9.3.3, “Basic attended SLES
network installation” on page 299. The tasks to enable a fully unattended
installation are explained in 9.3.5, “Unattended installation with SLES” on
page 311.
291
Basic setup of
services to
prepare the
network install
infrastructure
Set
Network – BOOTP
as first boot device
via MM or AMM
Power on
BladeCenter JS21
Define Install Server,
protocol and
source path via Open
firmware prompt?
No
No
Yes
Open firmware
prompt, set Install
Server, protocol
and source path
BOOTP request
send by
BladeCenter JS21
BOOTP request
send by
BladeCenter JS21
Receive IPAddress and boot
image path from
BOOTP server
image path from
BOOTP server
Load boot image
and start install
process
Load boot image
and start install
process
Install Server,
protocol and source
path predefinied?
Yes
Initial
installation menu,
set Install Server,
protocol and
source
Attended or
Unattended
install?
Attended
Main installation
menu, configure
provided options
Data transfer/
install from Install
Server to
BladeCentre JS21
Figure 9-1 SLES network installation overview
292
Unattended
9.3.2 Preparing the installation source directory and service
In this book, we use the FTP protocol as a service to serve the installation data of
a distribution after the network boot. If the running operating system on the
server is SLES, it is possible to configure the system with YaST. See “Preparing
SLES as an installation server using YaST” on page 294.
Creating and working with CD/DVD ISO images
You can use the following ways to name ISO3 files:
SLES CD no. X: SLES-9-ppc-SP3-CDX.iso
SLES DVD no. X: SLES-10-ppc-RC1-DVDX.iso
As an example, the following command creates a file in the /srv/data/iso-images/
directory directly from the second SLES CD using dd. You might want to have all
the Linux distribution CD-ROMs saved on your hard disk as ISO files.
dd if=/dev/cdrom of=/srv/data/iso-images/SLES-9-ppc-SP3-CD2.iso bs=32M
Repeat this step for all the CDs/DVDs, changing the file name as appropriate.
You can also increase the bs parameter as appropriate. This parameter controls
block size. The larger the block size is, the more RAM is taken for the dd
process, but the faster the process works.
Important: Ensure that the CD/DVD is not mounted before beginning the dd
process. Also ensure that the destination of the ISO has enough space to
store all the data. One CD ISO image file typically requires 650 MB and a
single layer DVD ISO image file requires up to 4.7 GB of hard drive space.
Note: We created our ISO files on a remote server and then transferred them
to our installation server within the BladeCenter to take advantage of its fast
network interfaces.
Though working with downloaded CD ISO images makes no difference, a DVD
ISO image is sometimes split up in different parts. For example, you can merge
the following two parts:
SLES-10-ppc-RC1-DVDX.iso#a
SLES-10-ppc-RC1-DVDX.iso#b
To fuse these parts, use the following command on a Linux operating system:
cat SLES-10-ppc-RC1-DVDX.iso#[ab] > SLES-10-ppc-RC1-DVDX.iso
3
An ISO image is a full CD or DVD image of an ISO 9660 file system
293
Preparing SLES as an installation server using YaST
Within YaST, there is an applet called Installation Server to configure installation
sources on the server. Use this applet to enable a service to serve the data and
to prepare the installation source.
1. Access this applet from the Misc section in the left pane, as shown in
Figure 9-2.
Figure 9-2 YaST installation server configuration (SLES9)
Important: The SLES version that you use as an installation server must
be equal to or a later version than the SLES version which is prepared as
an installation source to avoid problems.
294
2. The applet opens the window shown in Figure 9-3. At this point, there are no
configured sources. Select Settings to do the initial setup of the installation
server.
Figure 9-3 Initially no configured installation sources (SLES9) seen
295
3. In the Initial Setup -- Servers menu, choose a service. If you decide not to
configure any service at all, just set the destination directory for the
installation source directory tree. As an example, in Figure 9-4, we enable
FTP and define /srv/ftp as the destination directory for the installation
sources and as the public FTP directory. To save the settings, click Next.
Figure 9-4 Initial Setup -- Servers menu (SLES9)
4. In the Source Configuration pane, click Configure.
296
5. In the Source Configuration pane shown in Figure 9-5, choose a significant
Source Name.
This name is used as the directory name under the previously defined
installation source directory. Use ISO images instead of CDs/DVDs to create
the installation server quicker. You can also do the preparation remotely
without changing medias. For information about how to create an ISO image,
see “Creating and working with CD/DVD ISO images” on page 293. Proceed
past this window to build the source directory tree, for example, in
/srv/ftp/sles9_sp3/. If you are not building from a DVD ISO image, you are
prompted for additional CD-ROMs. You can see the number of the requested
CD-ROM in the title of the window.
Figure 9-5 Source configuration pane (SLES9)
297
6. Figure 9-6 shows SLES9 in the Configured Sources section. The Source to
Configure section shows all installation sources listed that are not active. The
active sources are listed in the Configured Sources section. To install the
service packs, click the Change button.
Figure 9-6 SLES9 configured as an installation source
7. Choose the configured installation source and proceed with Edit.
a. Select Prompt for additional CDs.
b. Click Next to select the additional ISO images that make up the service
pack.
Now the SLES installation server is ready.
8. Copy the bootable installation kernel to the TFTP directory. With SLES, you
can program this installation kernel with parameters that make installations
easier and more automated.
A bootable image contains both the kernel and initial RAM disk configuration
in a single file. The actual file name is only important for the configuration of
the DHCP daemon and TFTP daemon configuration. For SUSE, this file is
called install and you can find it in the root directory of the first CD/DVD.
298
Note: Adjust some of the boot parameter of the bootable image with the
mkzimage_cmdline tool to reduce the effort during a network installation.
For more details about this tool, see 9.3.4, “Configuring the boot image file
with mkzimage_cmdline” on page 306.
Copy and rename the bootable image to the public TFTP boot directory of the
TFTP service:
cp /mnt/loop/sles9_sp3/SUSE-SLES-Version-9/CD1/install
/srv/tftp/tftpboot/sles9_sp3_basic
Note: In general, use the bootable image that is shipped with the
distribution. This supports many different hardware drivers or software
versions. You can also build a new bootable image.
The explicit location during the boot process of the boot image file,
/srv/tftp/tftpboot/sles9_sp3_basic is defined by two entries:
– The setting of the public TFTP boot directory (for example, /srv/tftp/)
defined in the TFTP server configuration file (for example,
/etc/xinet.d/tftpd). See the bold line in Figure 9-2 on page 287 or
Figure 9-3 on page 287.
– The file name parameter in the /etc/dhcpd.conf. See the bold line in
If there are any changes in the configuration files of a service, restart the
services to activate the changes (see 9.2, “Basic preparations for a Linux
network installation” on page 283).
9.3.3 Basic attended SLES network installation
At this point, you must have prepared all the network services. If this is not the
case, see 9.2.1, “Installing Linux using the network: General remarks” on
page 284, and 9.3.2, “Preparing the installation source directory and service” on
page 293. If you have fulfilled the prerequisites, the integrated TFTP client of the
BladeCenter JS21 requests a bootable image4 during the boot process.
Tip: You can avoid the manual setup shown in this part by using the
mkzimage_cmdline tool from SUSE (see 9.3.4, “Configuring the boot image file
with mkzimage_cmdline” on page 306).
4
This bootable image is also called zimage or bzimage
299
If you have finished all the preparations, use an SoL session to follow the boot
process. If everything is set up correctly, the output shown in Figure 9-7 opens
after the initial boot phase.
Note: Ensure that the boot sequence is set to enable network boot using
Network - BOOTP. Additionally, do not set the BOOTP configuration of the
BladeCenter JS21 to a directed BOOTP request. Set the server IP and client
IP to 0.0.0.0, as shown in Figure 9-7.
.
.
BOOTP:
BOOTP:
BOOTP:
BOOTP:
BOOTP:
BOOTP:
BOOTP:
BOOTP:
server
IP =
0.0.0.0
client
IP =
0.0.0.0
client
HW addr =
0 11 25 c9 b a7
gateway IP =
0.0.0.0
device
loc-code U788D.001.23A1137-P1-T8
BOOTP R = 1 BOOTP S = 2
FILE: /tftpboot/sles9_sp3_basic
FINAL Packet Count = 10401
FINAL File Size = 5324986 bytes.
load-base=0x4000
real-base=0xc00000
zImage starting: loaded at 0x400000
initial ramdisk moving 0x3d7d000 <- 0x680000 (282925 bytes)
trying: 0x01400000
trying: 0x01500000
.
.
Figure 9-7 SoL of the SUSE network installation during startup using BOOTP of the
BladeCenter JS21
300
Figure 9-8 shows the screen from SoL after SLES9 has booted from the original
bootable image. The following message is seen because in this case the
installation source is not defined yet:
*** Could not find the SUSE Linux Enterprise Server 9 Installation CD.
We did not use mkzimage_cmdline in this example to program in an installation
source parameter into the installation kernel. You can still do the installation by
choosing it manually.
1. Select the language that you want to determine the language for the following
menus. We type 4 to choose English. Press Enter.
.
.
>>> SUSE Linux Enterprise Server 9 installation program v1.6.36 (c)
1996-2004 SUSE Linux AG <<<
Starting hardware detection...
Activating usb devices... done
Searching for info file...
*** Could not find the SUSE Linux Enterprise Server 9 Installation CD.
Activating manual setup program.
Select the language.
1)
2)
3)
4)
5)
6)
7)
8)
9)
10)
11)
12)
13)
14)
15)
16)
Bosnia
Cestina
Deutsch
English
Español
Français
Hellenic
Italiano
Japanese
Magyar
Nederlands
Polski
Português
Português Brasileiro
Russian
Slovencina
> _
Figure 9-8 SoL after operating system booted from the original SLES9 boot image
301
1. After you select the language, the main menu opens with a number of options
as shown in Figure 9-9.
Note: If you use hardware that is available during the installation process
but is not supported by the SLES install kernel version, you can load
additional kernel modules by typing 3 before starting the installation.
In this example, type 4 to choose Start Installation or System and proceed
with the installation process.
.
>>> Linuxrc v1.6 (Kernel 2.6.5-7.69-pseries64) (c) 1996-2004 SUSE
Linux AG <<<
Main Menu
1)
2)
3)
4)
5)
6)
7)
Settings
System Information
Kernel Modules (Hardware Drivers)
Eject CD
Exit or Reboot
Power off
> _
Figure 9-9 SLES9 main menu
2. The screen shown in Figure 9-10 opens. Type 1 to start the installation or
update.
1) Start Installation or Update
2) Boot Installed System
3) Start Rescue System
> _
Figure 9-10 SLES9 installation menu
302
3. You are prompted for the source of the installation medium as shown in
Figure 9-11. To perform a network installation, type 2 to choose Network as
the installation source.
Choose the source medium.
1) CD-ROM
2) Network
3) Hard Disk
> _
Figure 9-11 SLES9 source menu
4. You are prompted for the network protocol as shown in Figure 9-12. In our lab,
we set up our installation source to use FTP. In this screen, we specify option
1.
Choose the network protocol.
1)
2)
3)
4)
FTP
HTTP
NFS
TFTP
> _
Figure 9-12 SLES9 protocol menu
5. You are now prompted for the network options as shown in Figure 9-13.
a. The menu provides options to choose an Ethernet device, for example,
eth1. (We choose eth1 because eth0 is used by SoL.)
b. If a DHCP server is running in the subnet, see 9.2, “Basic preparations for
a Linux network installation” on page 283. You can type 1 to configure
using DHCP. In our case anonymous login is suitable, therefore, we type 2.
c. If no DHCP proxy is running, again type 2.
d. If the DHCP service serves data about the installation server and
directory, it is presented as default, but you can also choose a different
configuration.
303
Choose the network device.
1) eth0
2) eth1
: ethernet network card
: ethernet network card
> 2
Automatic configuration via DHCP?
1) Yes
2) No
> 1
Sending DHCP request...
Enter the IP address of the FTP server [192.168.1.254]>
Usually an FTP installation is performed via anonymous FTP. It is
possible to
specify a user name and password for the FTP server access.
Use a user name and password?
1) Yes
2) No
> 2
Use a HTTP proxy?
1) Yes
2) No
> 2
Trying to connect to the FTP server...
Enter the directory on the server
[/sles9]>
Figure 9-13 SLES9 initial network setup for the installation source
304
At this point, the initial setup of the SLES installation is complete and you can
install SLES as usual. But there are some particularities for a network installation
on a BladeCenter JS21:
Package evaluation: This process takes several minutes
Partitioning: Remember that SLES configures no logical volumes like Red Hat
Enterprise Linux as default. A simple configuration tested with single-path
storage area network (SAN) boot is shown in Example 9-6 and Example 9-7.
Example 9-6 SLES9 partition table with logical volumes
Disk /dev/sda: 146.7 GB, 146772852736 bytes
255 heads, 63 sectors/track, 17844 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot
/dev/sda1
*
/dev/sda2
/dev/sda3
Start
1
4
135
End
3
134
17843
Blocks
24066
1052257+
142247542+
Id
6
82
8e
System
FAT16
Linux swap
Linux LVM
Example 9-7 SLES9 logical volume configuration
--- Logical volume --LV Name
VG Name
LV UUID
LV Write Access
LV Status
# open
LV Size
Current LE
Segments
Allocation
Read ahead sectors
Block device
/dev/system/all
system
Fhbu32-LwuK-z9MQ-zqut-MYMp-rzW0-Kio7KT
read/write
available
2
135.60 GB
34714
1
inherit
0
253:0
You can avoid the manual setup shown in this section by using the
mkzimage_cmdline tool from SUSE (see 9.3.4, “Configuring the boot image file
with mkzimage_cmdline” on page 306).
305
9.3.4 Configuring the boot image file with mkzimage_cmdline
Use the mkzimage_cmdline tool to adjust the bootable image. You can find it in the
/ppc/netboot/ directory on the first CD/DVD of SLES.
Note for experts: You can create the mkzimage_cmdline binary for other
architectures using a C or C++ Compiler and the source code file called
mkzimage_cmdline.c. The binary that is on the SLES9 installation media only
runs on a Performance Optimization With Enhanced RISC (POWER) system
running Linux. This binary does not ship on SLES9 installation CD-ROMs for
x86 architecture.
The following examples demonstrate some possibilities and outputs of the
mkzimage_cmdline tool. Example 9-8 shows an example command to get the
boot parameter information stored in the bootable image file called install. The
output indicates that no command line options are stored in the image.
Example 9-8 Using mkzimage_cmdline to see boot parameters in boot image
# ./mkzimage_cmdline /srv/tftp/tftpboot/install
cmd_line size:512
cmd_line:
active: 0
To add new options, use -s “STRING”, where STRING is a variable which can
contain several options as shown in Table 9-1. To activate the options saved in
the boot image file use -a 1, and to deactivate use -a 0.
./mkzimage_cmdline -a 1 -s “STRING” /srv/tftp/tftpboot/install
Table 9-1 Some options to adjust a bootable image
Optiona
Function
insmod=MODULE
Adds special device driver to the kernel, where MODULE is a
placeholder for the filename of the module
dhcp=0
Disables IP address configuration using DHCP. The default is
dhcp=1.
netdevice=ethX
Defines the physical network interface, which is used as boot
device. X is the number of the device, where X ≥ 0. For a basic
blade installation without Ethernet daughter cards, we
recommend that you use eth1.
hostip=X.X.X.X
Defines the IP address of the Ethernet boot device to X.X.X.X
netmask=Y.Y.Y.Y
Defines the subnet mask of the Ethernet boot device to Y.Y.Y.Y
306
Optiona
Function
gateway=G.G.G.G
Defines the IP address of the gateway
nameserver=N.N.N.N
Defines the IP address of the name server
install=ftp://192.168.1.254/sles9_sp3
Defines the protocol (ftp), the IP address of the server
(192.168.1.254) and the path (/sles9_sp3) where the installation
source of the operating system is stored. The supported
protocols are NFS, Hypertext Transfer Protocol (HTTP), FTP,
and TFTP.
autoyast=ftp://192.168.1.254/conf/
sles9_sp3_basic.xml
Defines the protocol (ftp), the IP address of the server
(192.168.1.254) and the file (/conf/sles9_sp3_basic.xml) where
the AutoYaST configuration for an unattended installation is
stored
vnc=1 vncpassword=asyoulike
Enables a Virtual Network Computing (VNC) service on the
BladeCenter JS21 to get an insecure but graphical user interface
during the installation
usessh=1 sshpassword=asyoulike
Enables a Secure Shell (SSH) service on the BladeCenter JS21
to get a secure user interface during the installation. If an
X Server is running on the client connected to the BladeCenter
JS21, use a graphical interface in combination with a secure data
communication.
a. We did not verify all the options.
Note: When you install multiple blades, avoid reconfiguring the hostip network
parameters using parameters within a bootable image. This action overwrites
the configuration using a DHCP server and causes problems. When you
install multiple blades, we recommend that you use the DHCP server to
assign IP addresses.
The following examples enable attended installation without using SoL. SoL can
be helpful when starting the installation. When you use VNC as part of the
installation, the SoL screen shows you the IP address to connect to.
307
The command without a line break shown in Example 9-9 produces a
configuration that enables SSH connections to the system during the installation
process.
Example 9-9 Configuring the boot image to enable installation using SSH through eth1
./mkzimage_cmdline -a 1 -s “install=ftp://192.168.1.254/sles9_sp3
usessh=1 sshpassword=mysshpassword netdevice=eth1”
/srv/tftp/tftpboot/install
The command shown in Example 9-10 is similar, but enables VNC connections
to the system during the installation process.
Example 9-10 Configuring the boot image to enable installation using VNC through eth1
./mkzimage_cmdline -a 1 -s “install=ftp://192.168.1.254/sles9_sp3 vnc=1
vncpassword=myvncpassword netdevice=eth1” /srv/tftp/tftpboot/install
Note: It is not mandatory to use the netdevice=eth1 parameter, but in general,
eth1 is used for a Linux installation on a BladeCenter JS21 because eth0 is
used for SoL. Keep in mind that this setting is independent from the boot
sequencer definition. The defined interface is only used to install the operating
system.
To check whether a boot image is configured as required, use the following
command:
./mkzimage_cmdline /srv/tftp/tftpboot/install
Example 9-11 shows an example of this command usage after preparing the
boot image (as shown in Example 9-9).
Example 9-11 mkzimage_cmdline
# ./mkzimage_cmdline /srv/tftp/tftpboot/install
cmd_line size:512
cmd_line: install=ftp://192.168.1.254/sles9_sp3 usessh=1
sshpassword=mypassword netdevice=eth1
active: 1
Start the BladeCenter JS21 and connect using an SSH client or VNC client to
complete the installation. After a short time, the boot process is finished. You can
use an SSH client or VNC client to log on to the installation screen of YaST
without SoL. To confirm if a BladeCenter JS21 is ready for login, use ping or
follow the process by tracking the log file on the installation server. If the
connection is established, follow the instructions on the screen. For the
308
SSH-enabled installation, this means type yast and press Enter to start the
installation. For the VNC-enabled installation, you see the installation screen
when you connect.
The installation process is split up in two parts. The client connection is
terminated after completion of the first part. It is necessary to reconnect.
Figure 9-14 shows the last lines of the SoL output of the first installation part.
Starting SSH daemon
...
/sbin/ifconfig eth0 eth1
eth1
Link encap:Ethernet HWaddr 00:11:25:C9:0B:A7
inet addr:192.168.1.101 Bcast:192.168.1.255
Mask:255.255.255.0
***
sshd has been started
***
***
***
login using 'ssh -X [email protected]' ***
run 'yast' to start the installation ***
Figure 9-14 Last SoL output using SSH during the first part of a SLES9 installation
Figure 9-15 shows the last lines of the SoL output of the second installation part.
Starting SSH daemon
done
***
sshd has been started
***
you can login now and proceed with the installation
run the command '/usr/lib/YaST2/bin/YaST2.sshinstall'
active interfaces:
lo
Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
NET: Registered protocol family 10
IPv6 over IPv4 tunneling driver
Figure 9-15 Last SoL output using SSH during the second part of a SLES9 installation
309
For the second part, the BladeCenter JS21 already boots from the hard drive. To
continue the installation, run the following command:
/usr/lib/YaST2/bin/YaST2.sshinstall
When the second part is finished, the installation is complete.
It might happen that an error message is shown during the second connection
attempt as shown in Figure 9-16. To resolve this problem, delete the
corresponding entry in ~/.ssh/known_hosts5.
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@
WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
@
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle
attack)!
It is also possible that the RSA host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
43:9e:49:6e:84:b8:e8:cf:8c:e1:e2:8e:52:64:4f:79.
Please contact your system administrator.
Add correct host key in /root/.ssh/known_hosts to get rid of this
message.
Offending key in /root/.ssh/known_hosts:2
RSA host key for 192.168.1.101 has changed and you have requested
strict checking.
Host key verification failed.
Figure 9-16 SSH error message during the second connection attempt
5
310
“~” is a placeholder for the actual home directory and will be automatically resolved by Linux
Figure 9-17 shows another SoL example with VNC and SLES10.
starting VNC server...
A log file will be written to: /tmp/vncserver.log ...
***
***
You can connect to 192.168.1.101, display :1 now with
vncviewer
***
Or use a Java capable browser on
http://192.168.1.101:5801/
***
(When YaST2 is finished, close your VNC viewer and return to this
window.)
*** Starting YaST2 ***
Xlib: extension "XInputExtension" missing on display ":0.0".
Figure 9-17 Last SoL output using VNC during SLES10 installation
To learn more about how to set up a bootable image file to prepare an
unattended installation of SLES, see “Preparing an unattended installation using
AutoYaST”.
9.3.5 Unattended installation with SLES
To run an unattended installation, there are two main tasks:
1. All the configuration options that are usually chosen by a human during the
installation must be predefined. Therefore, SLES provides a tool called
AutoYaST to create an Extensible Markup Language (XML) file, which holds
all the predefined configurations options (see the following section).
2. After you create the XML file, you have to perform other steps to enable an
unattended installation.
Preparing an unattended installation using AutoYaST
There are two steps to create a working configuration file, for example,
sles9_sp3_basic.xml:
1. Use the AutoYaST configuration tool provided by the SUSE administration
tool called YaST to create a basic XML file. It provides a graphical-based or
text-based interface.
2. Adjust the entries in the XML file manually.
311
In this book, we use the AutoYaST configuration tool with the graphical interface
to demonstrate how to create a basic XML file. There are other options available.
However, it is not possible to describe all the features of the tool.
Apart from general explanations, this section describes how to build the XML file
shown in Appendix A, “SUSE Linux Enterprise Server 9 AutoYaST XML file” on
page 419. There are a lot of optional settings, but some are mandatory settings
or dependencies. For example, if you configure the firewall for eth0 and eth1, it is
also necessary to specify the Ethernet interfaces eth0 and eth1.
1. Start the YaST application, which opens a window as shown in Figure 9-18.
Launch the Autoinstallation applet from the Misc section of YaST.
Figure 9-18 AutoYaST module within YaST (SLES9)
312
2. After the selection, the main AutoYaST configuration window opens as shown
in Figure 9-19.
Figure 9-19 Main AutoYaST menu (SLES9)
There are five menu options: File, View, Classes, Tools, Preferences, and
three ways to create an AutoYaST file apart from loading an existing XML file
using File → Open.
Restriction: It is not possible to use a once created XML file from a
different SLES version, because the XML format differs.
You can import configuration from Red Hat or Alice6 by selecting
File → Import. We do not use this option in this book.
6
A predecessor to AutoYaST
313
Clone the configuration of the installation server by selecting Tools → Create
Reference Profile, as shown in Figure 9-20.
Figure 9-20 Creating AutoYaST file from the installation server configuration
314
3. A second window opens as shown in Figure 9-21. In this window, you can
clone parts of the installation server configuration to create the AutoYaST file.
In this case, select all three options in the Basic Resources section and click
Create to generate a minimal XML file.
Figure 9-21 Selecting the basic resources
After this step, continue with some of the changes using the AutoYaST tool as
described in the following section.
Note: In other cases, it might be helpful to import more settings. There
might be error messages, but you can correct potential errors during the
manual adjustment of the XML file later.
315
4. After you create the create a basic configuration, adjust some of the options
provided in the following sections in the menu and configure the provided
options:
– Software
Select the packages to install, and configure the automatic update client.
Figure 9-22 shows the window that opens when you select
Software → Package Selection. In this case, we choose the minimum
software configuration to save time during test installations.
Figure 9-22 Manual configuration of the AutoYaST file using YaST
– Hardware
Configure Partitioning, Audio, Printing, and Graphics Card and Monitor, if
necessary. Only the Partitioning settings are critical, but you have to
change them manually later. Therefore, leave all options as they are or
use an alternative partitioning configuration as a placeholder.
– System
Set the general system information such as language configuration, time
zone, other locale-related settings, logging, and run-level information in
this option. The most important configuration is the Boot Loader
316
Configuration. This configuration is cloned from the /etc/lilo.conf file on
the installation server. You can add a parameter, for example,
max_loop=255.
– Network Devices
Set the network adapter information. You can set kernel module
information and IP details here but avoid MAC-related configurations. In
the beginning, no adapter is configured.
i. To open the menu, click Configure.
ii. The Network cards configuration main menu opens, select Configure
to add interfaces.
iii. Some selections are already configured, such as Device Type:
Ethernet. Type bcm5700 as module name for the adapter and click
Next.
iv. In the Host name and name server section, choose DHCP for the
Hostname and Domain Name (Global) and also choose DHCP for
Name servers and domain search list.
v. Click OK and click Next. Interface eth0 is ready now.
To create interface eth1, repeat the steps. However, the DHCP settings
are automatically taken from the eth0 configuration and the interface name
automatically changes to eth1.
– Network Services
Configure network clients and daemons using this option. There are more
than 15 daemons to choose from. The settings are all optional. For this
example, do not change anything.
– Security and Users
You can create users, and configure security policies and virtual private
network (VPN). The firewall configuration is straightforward, but avoid
MAC-related configurations:
i. In the External Interface field, type eth0.
ii. In the Internal Interface field, type eth1.
iii. In the next menu, select Secure Shell to enable Port 22 connections
through the firewall.
iv. In the following menu, disable Protect from Internal Network.
v. Adapt the Security settings. The default is seven characters for the
maximum password length.
To log in as a root user using SSH, it is mandatory to define the root
user password. This password is saved encrypted in the XML file.
317
Select the Edit and create users menu. Change the Filter to System
Users by clicking the Set Filter button. Edit the root user and set the
password.
– Misc
This option allows you to add complete configuration files, or to add
special scripts to run before and after the installation.
5. To save the configuration, select File → Save. For example, save it as
sles9_sp3_basic.xml and store the file in the /srv/ftp/conf/ directory on the
installation server.
6. Manually configure the XML file. The most important changes apply to the
partitioning. Example 9-12 shows a possible configuration of the partitioning
section of the XML file that is not working immediately after the creation.
Example 9-12 A partitioning configuration not working in the newly created XML file
.
<partitioning config:type="list">
<drive>
<device>/dev/sda</device>
<partitions config:type="list">
<partition>
<partition_id config:type="integer">65</partition_id>
<partition_nr config:type="integer">1</partition_nr>
<region config:type="list">
<region_entry config:type="integer">0</region_entry>
</region>
<size>-2097151</size>
</partition>
<partition>
<filesystem config:type="symbol">swap</filesystem>
<format config:type="boolean">true</format>
<mount>swap</mount>
</region>
<size>1071644673</size>
</partition>
<partition>
<filesystem config:type="symbol">reiser</filesystem>
<format config:type="boolean">true</format>
318
<mount>/</mount>
</region>
<size>35307651073</size>
</partition>
</partitions>
<use>all</use>
</drive>
<drive>
<device>/dev/sdb</device>
<partitions config:type="list"/>
</drive>
</partitioning>
Example 9-13 shows a changed configuration with two drives that are working
fine. A complete functional XML file including partition information is shown in
Appendix A, “SUSE Linux Enterprise Server 9 AutoYaST XML file” on
page 419. After the changes in the partitioning section, the XML file is ready
to use.
Example 9-13 A simple working partitioning configuration in the XML file
.
<drive>
<use>all</use>
</drive>
<drive>
<device>/dev/sdb</device>
</drive>
</partitioning>
.
319
Tip: For information about AutoYaST, see the guide: Automatic Linux
Installation and Configuration with YaST2. You can find this guide on the Web
at:
http://www.suse.com/~ug/autoyast_doc/
Keep in mind that some of the options depend on the architecture or SLES
version.
Performing an unattended SLES installation
You can perform an unattended SLES installation by making an AutoYaST file
available when you boot the boot image. The AutoYaST file is an XML file that
can exist on a network server, such as an FTP server. One example is to add
the following line to your boot parameters of a boot image using
mkzimage_cmdline:
ftp://192.168.1.254/conf/sles9_sp3_basic.xml
With the mkzimage_cmdline tool, prepare a fully unattended installation of
BladeCenter JS21. Configure the bootable image file with the mkzimage_cmdline
tool. To learn more about the usage of this tool, refer to 9.3.4, “Configuring the
boot image file with mkzimage_cmdline” on page 306.
Issue the command shown in Example 9-14 without new lines to write the
configuration to the bootable image sles9_sp3_basic_auto. The file
/srv/tftp/tftpboot/sles9_sp3_basic_auto must exist to perform the operation.
Example 9-14 Configuring boot image for unattended SLES installation
./mkzimage_cmdline -a 1 -s “install=ftp://192.168.1.254/sles9_sp3
autoyast=ftp://192.168.1.254/conf/sles9_sp3_basic.xml netdevice=eth1”
/srv/tftp/tftpboot/sles9_sp3_basic_auto
Important: The unattended installation requires an empty partition table on
the target hard drive to avoid problems during the partitioning process.
Restriction: You cannot use SSH or VNC in combination with an AutoYaST
installation to follow the progress. Even if it is configured as an option, it does
not work. To follow the progress, use SoL or use standard tools such as ping
or logging.
320
Note: Use the Open Firmware prompt and set all the necessary options
instead of using the mkzimage_cmdline tool. Issue the following command at
the open firmware prompt:
boot net install=ftp://192.168.1.254/srv/ftp/sles9_sp3/
autoyast=ftp://192.168.1.254/conf/sles9_sp3_basic.xml netdevice=eth1
See “Performing an unattended Red Hat Enterprise Linux installation” on
page 343, for more information about this type of boot parameter definition.
After the installation, which might take some time, the newly installed server must
be reachable using SSH at the IP address configured using the DHCP server.
9.4 Installing Red Hat Enterprise Linux AS 4 Update 3
using the network
This section focuses on the Red Hat Enterprise Linux AS 4 (U3). The
prerequisites to perform a Red Hat Enterprise Linux installation using the
network are:
A configured Linux installation infrastructure (see 9.2, “Basic preparations for
a Linux network installation” on page 283)
The boot device sequence and the boot options of a BladeCenter JS21,
especially the Ethernet interface configuration, which you must configure
appropriately (see 6.1, “BladeCenter chassis” on page 64)
Attention: Red Hat Enterprise Linux does not switch the boot sequence of
boot devices after the installation. Perform this using the BladeCenter
management module, as described in 6.5, “Blade server configuration” on
page 90.
321
9.4.1 Overview of Red Hat Enterprise Linux network installation
Figure 9-23 on page 323 shows the steps during the network installation
process. Some necessary and basic preparations are already mentioned in the
preface of 9.4, “Installing Red Hat Enterprise Linux AS 4 Update 3 using the
network” on page 321. However, we describe some more tasks in 9.4.2,
“Preparing the installation source for Red Hat Enterprise Linux” on page 324
After this final step, the basic preparation of the network installation is complete,
but there are different ways, shown as decision symbols in Figure 9-23 on
page 323.
One possibility is installation without a defined installation server, installation
source directory, or transfer protocol, which in the most cases also means
attended installation. This basic installation procedure is described in 9.4.3,
“Basic attended Red Hat Enterprise Linux network installation” on page 326.
The definition of boot parameters such as installation server, installation
source directory, or transfer protocol using the Open Firmware prompt is used
in most cases in combination with an unattended installation. To learn more
about the Open Firmware prompt in general, see 6.9, “Open Firmware
interface” on page 152. More specific informations about unattended
installation are provided in 9.4.4, “Unattended installation with Red Hat
Enterprise Linux” on page 336.
Restriction: You cannot use the mkzimage_cmdline tool provided by SUSE
to configure a bootable image shipped with the Red Hat Enterprise Linux.
322
Basic setup of
services to
prepare the
network install
infrastructure
Set
Network – BOOTP
as first boot device
via MM or AMM
Power on
BladeCenter JS21
No
Define Install Server,
protocol and
source path via Open
firmware prompt?
Yes
Open firmware
prompt, set Install
Server, protocol
and source path
BOOTP request
send by
BladeCenter JS21
BOOTP request
send by
BladeCenter JS21
image path from
BOOTP server
image path from
BOOTP server
Load boot image
and start install
process
Load boot image
and start install
process
RHEL initial
installation menu,
set Install Server,
protocol and
source path
Attended
Attended or
Unattended
install?
Unattended
Main installation
menu, configure
provided options
Data transfer/
install from Install
Server to
BladeCentre JS21
Figure 9-23 Red Hat Enterprise Linux network installation overview
323
9.4.2 Preparing the installation source for Red Hat Enterprise Linux
Two steps are necessary to prepare the installation server:
1. It is necessary to have a service that provides installation data after the boot
image is loaded. In this part, as an example, we use FTP for this function.
Therefore, the first step is the preparation of an FTP service on the
installation server to enable access to the installation sources. The general
approach is already described in 9.2.4, “Configuring a File Transfer Protocol
service” on page 288.
2. When the FTP server is running, copy the Red Hat Enterprise Linux sources
from CDs or ISO images to the public FTP directory of the previously
prepared FTP server. To build an installation source directory for Red Hat
Enterprise Linux AS 4, follow the given instructions in this section.
Because it is much easier to work with ISO images, the first step is to create ISO
images. You can name the ISO files as follows:
Red Hat Enterprise Linux CD no. X: RHEL4-U3-ppc-AS-CDX.iso
As an example, the following command creates a file in the /srv/data/iso-images/
directory directly from the third Red Hat Enterprise Linux CD using dd:
dd if=/dev/cdrom of=/srv/data/iso-images/RHEL4-U3-ppc-AS-CD3.iso bs=32M
Repeat the previous step for all the CDs/DVDs, changing the file name as
appropriate. You can also increase the bs parameter as appropriate. This
parameter controls the block size. The larger the block size is, the more RAM is
taken for the dd process, but the faster the process works.
Important: Ensure that you do not mount the CD/DVD before beginning the
dd process. Also ensure that the destination of the ISO has enough space to
store all the data. One CD ISO image file typically requires 650 MB and a
single layer DVD ISO image file requires up to 4.7 GB of hard drive space.
Note: We created our ISO files on a remote server and then transferred them
to our installation server within the BladeCenter to take advantage of its fast
network interfaces.
Now copy the installation data to the FTP directory. Because we are handling
five CD-ROMs, we show how to process all the five CD-ROMs with single
commands.
324
1. Create some directories to mount the ISOs:
for X in ‘seq 1 5‘; do mkdir /mnt/loop/rhel4/CD$X -p; done
2. Mount the ISO files using a loop device. You can write the following command
in one command line:
for X in ‘seq 1 5‘; do mount -t iso9660 -o loop
/srv/data/iso-images/RHEL4-U3-ppc-AS-CD$X.iso /mnt/loop/rhel4/CD$X;
done
3. Create a new directory in the public FTP directory:
mkdir /srv/ftp/rhel4
4. Copy the now accessible files to this directory including hidden files. The
slash (“/”) at the end of the source directory is important for the following
command:
rsync -auv /mnt/loop/rhel4/CD[12345]/ /srv/ftp/rhel4
5. Copy the installation bootable image into the TFTP server. A bootable image
contains both the kernel and initial RAM disk configuration in a single file. The
actual file name is only important for the configuration of the DHCP-daemon
and TFTP-daemon configuration. For the Red Hat Enterprise Linux, this file is
called netboot.img and is in the directory /images/pseries/ of the first CD.
Copy and rename the bootable image to the public TFTP boot directory of the
TFTP service:
cp /mnt/loop/rhel4/CD1/images/pseries/netboot.img
/srv/tftp/tftpboot/rhel_as_4_u3.img
Note: In general, use the bootable image that is shipped with the
distribution to avoid trouble with different hardware drivers or software
versions.
6. The explicit location during the boot process of the boot image file,
/srv/tftp/tftpboot/rhel_as_4_u3.img is defined by two entries:
– The setting of the public TFTP boot directory (for example, /srv/tftp/)
defined in the TFTP server configuration file (for example,
/etc/xinet.d/tftpd). See the bold text in Figure 9-2 on page 287 or
– Adjust the file name parameter in /etc/dhcpd.conf to:
filenname “/tftpboot/rhel_as_4_u3.img
For more information, see 9.2.2, “Configuring a BOOTP or DHCP service”
on page 285.
325
7. If there are any changes in the configuration files of a service, restart the
services to activate the changes. See 9.2, “Basic preparations for a Linux
network installation” on page 283.
After this step, all basic preparations to install Red Hat Enterprise Linux using
the network are finished.
9.4.3 Basic attended Red Hat Enterprise Linux network installation
At this point, you must have prepared all the network services. If this is not the
case, see 9.2.1, “Installing Linux using the network: General remarks” on
page 284, and 9.4.2, “Preparing the installation source for Red Hat Enterprise
Linux” on page 324.
If you have fulfilled the prerequisites, the integrated TFTP client of the
BladeCenter JS21 requests a bootable image7 during the boot process. The
preparations to provide this bootable image using the prepared TFTP server are
described in this section.
Attention: The Red Hat Enterprise Linux installation does not set the boot
sequence to a hard drive after the installation using the network. You must
manually set this with the BladeCenter management module or one of the
boot menus. You can also configure the BladeCenter JS21 to boot using the
hard drive, but select the boot menu and select boot using the network
manually before the installation.
7
326
This bootable image is also called zimage or bzimage
1. After the reboot, the SoL output is similar to Figure 9-24. It is almost similar to
SLES.
.
.
BOOTP:
BOOTP:
BOOTP:
BOOTP:
BOOTP:
BOOTP:
BOOTP:
BOOTP:
server
IP =
0.0.0.0
client
IP =
0.0.0.0
client
HW addr =
0 11 25 c9 b a7
gateway IP =
0.0.0.0
device
loc-code U788D.001.23A1137-P1-T8
BOOTP R = 1 BOOTP S = 2
FILE: /tftpboot/netboot.img
FINAL Packet Count = 12149
FINAL File Size = 6220088 bytes.
load-base=0x4000
real-base=0xc00000
zImage starting: loaded at 0x400000
Allocating 0x66a000 bytes for kernel ...
trying: 0x01400000
trying: 0x01500000
.
.
Figure 9-24 SoL output of RHEL network installation during startup using BOOTP
function of the BladeCenter JS21
327
2. The SoL output after Red Hat Enterprise Linux has booted from the bootable
image is shown in Figure 9-25. Choose a language.
Welcome to Red Hat Enterprise Linux
+---------+ Choose a Language +---------+
|
|
| What language would you like to use
|
| during the installation process?
|
|
|
|
Catalan
#
|
|
Chinese(Simplified)
#
|
|
Chinese(Traditional)
#
|
|
Croatian
#
|
|
Czech
#
|
|
Danish
#
|
|
Dutch
#
|
|
English
#
|
|
|
|
+----+
|
|
| OK |
|
|
+----+
|
|
|
|
|
+---------------------------------------+
<Tab>/<Alt-Tab> between elements
| <Space> selects | <F12> next screen
Figure 9-25 SoL after the operating system boot from the RHEL bootable image
328
3. The screen shown in Figure 9-26 opens. Choose an installation method. We
choose FTP.
+------+ Installation Method +------+
|
|
| What type of media contains the
|
| packages to be installed?
|
|
|
|
Local CDROM
|
|
Hard drive
|
|
NFS image
|
| FTP
|
|
HTTP
|
|
|
|
+----+
+------+
|
|
| OK |
| Back |
|
|
+----+
+------+
|
|
|
|
|
+-----------------------------------+
Figure 9-26 Red Hat Enterprise Linux 4: Installation method menu
329
4. You see the Networking Device menu as shown in Figure 9-27. Choose eth1
because eth0 is used for SoL.
+----------------------+ Networking Device +----------------------+
|
|
| You have multiple network devices on this system. Which
|
| would you like to install through?
|
|
|
| eth0 - Broadcom Corporation NetXtreme BCM5780S Gigabit Ethernet |
| eth1 - Broadcom Corporation NetXtreme BCM5780S Gigabit Ethernet |
|
|
|
+----+
+------+
|
|
| OK |
| Back |
|
|
+----+
+------+
|
|
|
|
|
+-----------------------------------------------------------------+
Figure 9-27 Red Hat Enterprise Linux 4: Network device menu
330
5. The screen now prompts you to configure TCP/IP as shown in Figure 9-28. In
this example, the installation process uses DHCP. Press Enter.
+--------------------+ Configure TCP/IP +---------------------+
|
|
| Please enter the IP configuration for this machine. Each
|
| item should be entered as an IP address in dotted-decimal
|
| notation (for example, 1.2.3.4).
|
|
|
|
[*] Use dynamic IP configuration (BOOTP/DHCP)
|
|
|
| IP address:
________________
|
|
Netmask:
________________
|
| Default gateway (IP): ________________
|
|
Primary nameserver:
________________
|
|
|
|
+----+
+------+
|
|
| OK |
| Back |
|
|
+----+
+------+
|
|
|
|
|
+-------------------------------------------------------------+
Figure 9-28 Red Hat Enterprise Linux 4: Configure TCP/IP menu
331
6. You are prompted for a DNS as shown in Figure 9-29. If all the information is
not presented by the DHCP server, you see another message box.
+--------------------+ Configure TCP/IP +---------------------+
|
+-------------+ Nameserver +--------------+
|
| Please e|
| Each
|
| item sho| Your dynamic IP request returned IP
| cimal
|
| notation| configuration information, but it did
|
|
|
| not include a DNS nameserver. If you
|
|
|
[| know what your nameserver is, please
|
|
|
| enter it now. If you don't have this
|
|
|
| information, you can leave this field
|
|
|
| blank and the install will continue.
|
|
|
|
|
|
|
| Nameserver IP _________________________ |
|
|
|
|
|
|
|
+----+
+------+
|
|
|
|
| OK |
| Back |
|
|
|
|
+----+
+------+
|
|
|
|
|
|
|
|
|
|
+---------+-----------------------------------------+ --------+
Figure 9-29 RHEL4: Example for a missing name server definition in the DHCP server configuration file
332
7. Configure the FTP server information as shown in Figure 9-30.
+------------------------+ FTP Setup +-------------------------+
|
|
|
Please enter the following information:
|
|
|
|
o the name or IP number of your FTP server
|
|
o the directory on that server containing
|
| Red Hat Enterprise Linux for your
|
|
architecture
|
|
|
| FTP site name:
192.168.1.254___________ |
| Red Hat Enterprise Linux directory: rhel4___________________ |
|
|
| [x] Use non-anonymous ftp
|
|
|
|
+----+
+------+
|
|
| OK |
| Back |
|
|
+----+
+------+
|
|
|
|
|
+--------------------------------------------------------------+
Figure 9-30 Red Hat Enterprise Linux AS 4: FTP setup
333
After this step, the installation data is transferred through FTP and Anaconda.
The Red Hat Enterprise Linux installer Anaconda starts as shown in
Figure 9-31.
Red Hat Enterprise Linux AS (C) 2004 Red Hat, Inc.
+--------+ Red Hat Enterprise Linux AS +--------+
|
|
| Welcome to Red Hat Enterprise Linux AS!
|
|
|
|
|
|
+----+
+------+
|
|
| OK |
| Back |
|
|
+----+
+------+
|
|
|
|
|
+-----------------------------------------------+
Figure 9-31 Red Hat Enterprise Linux 4: Anaconda welcome screen
334
8. After some more menus to set up Red Hat Enterprise Linux 4, the last screen
before a reboot is shown in Figure 9-32. At this point, use the BladeCenter
management module to change the boot sequence from Network - BOOTP to
Hard drive X, if it is not prepared already.
Red Hat Enterprise Linux AS (C) 2004 Red Hat, Inc.
+----------------------+ Complete +-----------------------+
|
|
| Congratulations, your Red Hat Enterprise Linux AS
|
| installation is complete.
|
|
|
| Remove any installation media (diskettes or CD-ROMs)
|
| used during the installation process and press
|
| <Enter> to reboot your system.
|
|
|
|
|
|
+--------+
|
|
| Reboot |
|
|
+--------+
|
|
|
|
|
+---------------------------------------------------------+
<Enter> to reboot
Figure 9-32 SoL after Red Hat Enterprise Linux installation is complete
In the case of a Red Hat Enterprise Linux installation, it is not possible to reduce
the effort or avoid the SoL connection during a network installation because of
the lack of a tool such as mkzimage_cmdline, but you can prepare an unattended
installation.
335
9.4.4 Unattended installation with Red Hat Enterprise Linux
To run an unattended installation, there are two main tasks:
1. All the configuration options that are typically chosen by a human during the
installation must be defined in a Kickstart file. Therefore, Red Hat Enterprise
Linux provides a tool called Kickstart Configurator to create a text file, which
holds all predefined configurations options.
2. After you create the configuration file, you have to perform some more steps
to enable an unattended installation. However, there is no tool such as
mkzimage_cmdline to adjust the boot image file.
Preparing an unattended installation using Kickstart
Note for Red Hat Enterprise Linux 3 user: Be aware of the changes from
Red Hat Enterprise Linux 3 to Red Hat Enterprise Linux 4. For example, Red
Hat Enterprise Linux 3 uses redhat-config-xxx to start most of the
administration tools; Red Hat Enterprise Linux 4 uses system-config-xxx
instead.
Restriction: It is necessary to have a running X Server to use the
administration tools provided by Red Hat, because in most cases the tools use
a graphical user interface (GUI).
Red Hat provides a utility called system-config-kickstart to assist with the
creation of the configuration file for an unattended installation. This tool is a good
starting point to create an initial configuration file, but the provided options are
not sophisticated enough to create a working configuration. Therefore, after you
create the configuration file with Kickstart, you have to perform some manual
changes in this file.
Kickstart is not included in the default software installation. If the Red Hat
Enterprise Linux installation source is stored in the directory rhel4 on an FTP
server with the IP address 192.168.1.254, the first step is to start the package
management by issuing the following command:
system-config-packages -t ftp://192.168.1.254/rhel4
336
This connects to an FTP server with the IP address 192.168.1.254 with the
installation source directory rhel4. In the Package Management window, the
Kickstart tool is located under System → Administration Tools.
1. After you install Kickstart, issue the system-config-kickstart command to
launch this utility.
2. A window opens showing the Basic Configuration panel. The most important
configuration setting is highlighted with a rectangle in Figure 9-33. It is also
important to define a root password to enable SSH login after installation.
This password is saved encrypted in the configuration file.
Figure 9-33 Kickstart main window with Basic Configuration panel (RHEL4)
337
3. In the Installation Method panel (shown in Figure 9-34), all the basic
parameters for a network installation using FTP are shown.
Figure 9-34 Installation Method panel (RHEL4)
338
4. The next editable panel is the Partition Information panel, as shown in
Figure 9-35. The provided options are not specific enough to create a valid
partition configuration for a BladeCenter JS21. You can leave the
configuration as it is. After you create the configuration, adjust the settings
manually.
Figure 9-35 Partition Information panel (RHEL4)
5. The next panel is the Authentication panel. But in this case, we use the
default setting.
339
6. Figure 9-36 shows the Firewall Configuration panel. As an example, it is a
good idea to enable SSH using interface eth1 to access the system later
using network.
Figure 9-36 Firewall Configuration panel (RHEL4)
340
7. Figure 9-37 shows the Package Selection panel. We do not explain the other
panels in this context, because they are not important to create a basic
configuration file. After the package selection, save the configuration using
the Save option in the menu. When you save the file, this runs an automatic
basic check for any missing options.
Figure 9-37 Package Selection panel (RHEL4)
8. Manually adjust the Kickstart configuration file that you have created. The
basic Kickstart configuration file created with the Kickstart Configurator is
shown in Example 9-15.
Example 9-15 Basic Kickstart configuration file
#Generated by Kickstart Configurator
#platform=IBM pSeries
#System language
lang en_US
341
#Language modules to install
langsupport en_US
#System keyboard
keyboard us
#System mouse
mouse
#Sytem timezone
timezone America/New_York
#Root password
rootpw --iscrypted $1$/7IU0tSr$QmQtfRtKWulXPC8cmk2kf.
#Reboot after installation
reboot
#Use text mode install
text
#Install OS instead of upgrade
install
#Use Web installation
url --url ftp://192.168.1.254/rhel4
#System bootloader configuration
bootloader --location=mbr
#Clear the Master Boot Record
zerombr yes
#Partition clearing information
clearpart --all --initlabel
#System authorization infomation
auth --useshadow --enablemd5
#Network information
network --bootproto=dhcp --device=eth0
#Firewall configuration
firewall --enabled --trust=eth1 --ssh
#Do not configure XWindows
skipx
#Package install information
%packages --resolvedeps
Tip: Red Hat Enterprise Linux 4: System Administration Guide is a good
source of information regarding all Kickstart file options. It is also available at
the following Web site:
http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/sysadmin
-guide/
342
If you have not defined any partition options, the unattended installation process
stops. It is necessary to include the following new section after the #Disk
partitioning information section and before the #System authorization
information section:
#Disk partitioning information
autopart
Important: The order of the main sections in the Kickstart configuration file is
important for the functionality.
You can find the fully functional Kickstart configuration file with some additional
packages and partitioning information in Appendix B, “Red Hat Enterprise
Linux 4 Kickstart file” on page 427.
Performing an unattended Red Hat Enterprise Linux
installation
For an unattended installation, pass the location of the configuration file during
boot. BOOTP does not have the facility to provide anything more than the
location to the bootable image and the TFTP server IP address. To pass the
required parameters, use the Open Firmware prompt.
1. During the boot process, a screen output that is readable using SoL is
available. Type 8 to go to the Open Firmware prompt, as shown in
Figure 9-38.
1 = SMS Menu
Memory
Keyboard
5 = Default Boot List
Network
SCSI
Speaker
ok
0 > _
Figure 9-38 Open Firmware prompt
343
2. To start the unattended installation, type the following command at the Open
Firmware prompt if the configuration file is served using FTP:
boot net ks=ftp://192.168.1.254/conf/ks.cfg ksdevice=eth1
The nvsetenv command is provided by the ppc64-utils Red Hat Package
Manager (RPM) in Red Hat Enterprise Linux and the util-linux RPM in SLES.
The syntax to clear the boot-file variable is:
/sbin/nvsetenv boot-file ""
The unattended Red Hat Enterprise Linux installation is now complete.
344
10
Chapter 10.
System management
scenarios
This chapter primarily addresses two system management applications that
support BladeCenter environments: IBM Director and Cluster Systems
Management (CSM).
Cluster Systems Management and IBM Director positioning
CSM is recommended for managing UNIX-only clusters (Linux or AIX) when the
client prefers a scripting interface to perform cluster management tasks. CSM
provides full installation for Linux and a flexible command-line interface (CLI).
You can use IBM Director and CSM together to manage clustered systems. This
is especially true in the case of clustered blades.
345
10.1 BladeCenter Web interfaces
The BladeCenter Web interface allows system administrators to easily and
effectively manage up to 14 blades from an integrated interface. From trivial
tasks such as powering blades on or off, to more complex tasks such as firmware
management, the Web interface allows powerful control over all blades and
input/output (I/O) modules that are attached to the BladeCenter chassis.
You can use the Web interface for the management of other BladeCenter
resources, such as I/O modules, and the retrieval of system health information.
You can also configure BladeCenter-specific features such as the Serial over
LAN (SoL) from the Web interface.
For more information, see IBM eServer BladeCenter Systems Management,
REDP-3582.
10.2 IBM Director
IBM Director is an integrated suite of tools that provides you with comprehensive
system-management capabilities to maximize system availability and lower IT
costs. Its open, industry-standard design supports the management of a variety
of hardware and operating systems, including most Intel microprocessor-based
systems and certain IBM System p5, IBM eServer iSeries™, IBM eServer
pSeries, IBM System z9™, and IBM eServer zSeries® servers.
IBM Director automates many of the processes that are required to manage
systems proactively, including capacity planning, asset tracking, preventive
maintenance, diagnostic monitoring, troubleshooting, and more. It has a
graphical user interface (GUI) that provides easy access to both local and remote
systems.
At the time of writing this book, IBM Director V5.10.2 was the latest version
available and is the version we used for our testing for this topic. This latest
version includes:
Broader platform coverage for use in a heterogeneous environment that
includes IBM System p5, eServer p5, eServer i5, and eServer pSeries
A new streamlined interface to boost productivity
A new command-line interface in addition to the graphical interface
Lightweight agents for easy deployment
346
Important: Though you can use multiple versions of IBM Director in the same
environment, ensure that you use like versions of IBM Director Server and
IBM Director Agent. For example, use IBM Director Server V4.0 with IBM
Director Agent V4.0 and use IBM Director Server V5.10 to manage systems
with IBM Director Agent V5.10.
10.2.1 Installing the components for IBM Director
The primary components for IBM Director are:
Prerequisite applications
IBM Director Core Services
IBM Director Server
IBM Director Agent
IBM Director Console (Not to be confused with a Hardware Management
Console (HMC).)
IBM Director extensions (This includes extensions for HMCs)
Step-by-step instructions for installing the components on the various supported
hardware and software platforms (with the exception of HMCs) are provided in
detail on the following Web site:
http://publib.boulder.ibm.com/infocenter/eserver/v1r2/index.jsp
From the IBM Systems Software Information Center link, select Topic
overview → IBM Director.
The IBM Director CD is shipped with IBM BladeCenter chassis but is not shipped
with IBM blade servers. For the purposes of our review and testing, we
downloaded the necessary files and also the images as instructed on the Web
site.
IBM Director can gather some information from a blade server before the IBM
Director Agent or IBM Director Core Services is installed on the blade server.
The information is gathered from the blade server by way of the BladeCenter
management module. In the IBM Director Console, the blade server is
represented by a physical platform managed object. However, after you install
IBM Director Agent or IBM Director Core Services on the blade server, it is a
managed object, and the features and functions that you can use on the blade
server are comparable to those that you can use on any managed object.
Chapter 10. System management scenarios
347
The IBM Director tasks that you can use on your BladeCenter unit can vary,
depending on the features and options that you have installed. See Table 10-1 for
a list of the IBM Director tasks, and information about whether you can use a task
on the chassis, network device, or a blade server without IBM Director Agent or
IBM Director Core Services installed. Unless otherwise noted in this
documentation, a task behaves in the same way for blade servers as for any
managed system.
Note: When you install IBM Director Agent or IBM Director Core Services on a
blade server, the supported tasks depend on the operating system (OS) that
you install on the blade server.
Table 10-1 IBM Director task support for BladeCenter products
Tasks and subtasks
Chassis
Network
device
Blade server without IBM Director
Agent or IBM Director Core
Services installed
BladeCenter Configuration
Manager
Yes
No
Not applicable
Event action plans
Yes
Yes
Yes
Hardware status Y
Yes
No
Yesa
Inventory
Yes
Yes
Yes
Network Device Manager (formerly
Switch Management launch pad)
Not
applicable
Yes
Not applicable
Power Management™
No
No
Yes
Rack Manager
Yes
Yes
No
Remote session
Not
applicable
Yes
No
Remote monitors
No
Yes
No
Simple Network Management
Protocol (SNMP) browser
No
Yes
Yesb
a. You can obtain an inventory of the chassis, network device, and blade servers through the management module. Blade server inventory that is collected through the management module is a subset of the total inventory that is available if IBM Director Agent or IBM Director Core Services is
installed on the blade server.
b. To use the SNMP browser task, install the operating system SNMP agent on the blade server.
348
The following sections provide additional instructions that augment the details in
the Web site provided previously.
Installing IBM Director V5.10.2 on AIX
For this exercise, we used a desktop system running Windows 2000 for the IBM
Director Console. We installed the server and extensions code on a BladeCenter
JS20 running AIX V5.3 and installed the agent code on several BladeCenter
JS21s running AIX V5.3, Red Hat, and SUSE Linux in both symmetric
multiprocessor (SMP) and virtual input/output (VIO) settings (logical partitions
(LPARs)).
Prerequisite applications
There are two primary prerequisites that are described in the instructions
cited previously. The first is the IBM AIX Pegasus CIM Server. You can find
this on the AIX V5.3 Expansion Pack. Install this before the IBM Director
Server for AIX or the IBM Director Agent for AIX. The second application is
the OpenSSL code that is required by the CIM Server. You can install it from
the AIX Toolbox for Linux Applications.
Console code
We downloaded the console code directly to our desktop workstation running
Windows 2000. There is an IBM Director Console V5.10 compressed file and
a V5.10.2 patch compressed file. You can find the IBM Director Console
V5.10 for download at:
http://www.ibm.com/servers/eserver/xseries/systems_management/ibm_di
rector/
On this Web site, select Downloads in the left menu. Select IBM Director
5.10 for xSeries and BladeCenter and complete the registration as
requested and click Submit. The next screen provides a list of selectable
versions of the console code. Select IBM Director 5.10 Upward Integration
Modules (UIM) to install the initial version and follow the installation
instructions including selection of the Bladecenter Management Extension
and the Rack Manager. Install the IBM Director 5.10.2 patch for Windows
found at:
http://www-307.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR63788
Server and extensions code
You can find the IBM Director V5.10.2 Server installation code at the following
Web site. You have the option of downloading either a tar file or an ISO
image:
https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?source=dmp
349
We downloaded the ISO image, created a CD and proceeded to install the
server code and the BladeCenter and rack extensions using smitty as
instructed. Depending on your network setup, you might have to follow the
instructions to modify Secure Sockets Layer (SSL) port settings after
completing your installation. See the section “Configuring SSL settings for
IBM Director Console and IBM Director Server” in the IBM Systems Software
Information Center cited previously.
Agent code: The agent code is available at:
https://www14.software.ibm.com/webapp/iwm/web/reg/download.do?source
=dmp&S_PKG=diragent&cp=UTF-8
We downloaded the code and placed it on a CD. We extracted the
AIX-specific code from the tar file using the following command:
tar -xvf /mnt/*r director/agent/aix
We followed the instructions to run the proper file.
The instructions provided for downloading and installing from the ISO CD image
for the IBM Director Agent for AIX also worked as shown in the Web site
referenced previously.
We followed the installation instructions on the Web site provided at the
beginning of this section and started the console on our PC. The IBM Director
server discovered both the JS20 and JS21 blades and BladeCenter modules.
We also viewed vital product data and started a remote session on the one blade
with the IBM Director Server installed. Figure 10-1 shows the IBM Director
Console on our Windows system after the IBM Director automatically discovered
all systems in the network.
350
Figure 10-1 IBM Director BladeCenter discovery
Figure 10-1 reflects the selection of Chassis and Chassis Members group. The
only editing necessary for the discovered objects is to change the names for
convenience.
At this point the IBM Director Agent (Level 2) is not installed on any of the blades
except the one JS20 functioning as the IBM Director Server. As you can see, the
management module interface provides information for the management
module, assorted blades, and Ethernet switches. Basic functions such as power
management (on or off) for the blades is available. However, restarting the OS,
remote sessions, file transfer, and other options are not available without
installing the IBM Director Agent. Refer to Table 10-1 on page 348 for the
functions available without the agent on the blades to be managed.
351
Installing IBM Director Agent V5.10.2 on Linux
This version supports the JS21. The following Web site provides specific
instructions that are sufficient for a successful installation:
We downloaded the IBM Director V5.10.2 for Linux on Advanced Performance
Optimization with Enhanced RISC (POWER) ISO file and burned a CD for the
initial installation. We recommend that you use a Linux installation server where
you have multiple blades to install. Follow the preparation guidelines closely.
Prerequisite RPMs are identified.
For IBM Director Core Services
The prerequisites for IBM Director Core Services are:
Red Hat Enterprise Linux AS, Version 4.0, for IBM POWER only: Before you
install IBM Director on a system running Red Hat Enterprise Linux AS,
Version 4.0, for IBM POWER, ensure that the
compat-libstdc++-33-3.2.3-47.3.ppc.rpm Red Hat Package Manager (RPM)
file is installed.
Linux on POWER only: Ensure that the following RPM files (or later versions)
are installed:
– librtas-1.2-1.ppc64.rpm
– ppc64-utils-2.5-2.ppc64.rpm
– lsvpd-0.12.7-1.ppc.rpm
You can download these RPM files from the IBM Service and productivity
tools for Linux on POWER Web site at:
Select the appropriate tab for your Linux distribution.
Tip: SUSE Linux Enterprise Server 9 (SLES9) for IBM POWER only: Disable
the Service Location Protocol daemon (SLPD) before you install IBM Director
Core Services. IBM Director Server does not discover Level 1 managed
objects that are running SLPD.
IBM Director Agent
Red Hat Enterprise Linux AS, Version 4.0, for IBM POWER only: Before you
install IBM Director on a system running Red Hat Enterprise Linux AS, Version
4.0, for IBM POWER, ensure that the compat-libstdc++-33-3.2.3-47.3.ppc.rpm
RPM file is installed.
352
After you fulfill the prerequisites, use the IBM Director V5.10.2 for Linux on
POWER CD to install both SLES9 Linux and Redhat Linux blades and LPARs.
1. To start the installation from the CD, perform the following steps:
a. Insert the IBM Director Version 5.10.2 for Linux on POWER or IBM
Director Agents for AIX 5L, IBM i5/OS®, Windows, Linux on xSeries,
IBM System z™ and POWER CD into the drive.
b. If the CD automounts, go to step c. If the CD does not automount, type the
following command and press Enter:
mount /dev/cdrom /mnt/cdrom
Here dev/cdrom is the specific device file for the block device and
mnt/cdrom is the mount point of the drive. For our purposes, we used
/dev/sr0 as the device and /media/cdrom as a mount point.
c. Change to the directory in which the installation script is located. Type the
following command and press Enter:
cd /mnt/cdrom/director/agent/linux/ppc/FILES
Here mnt/cdrom is the mount point of the drive.
2. If you want to customize the installation, go to step 3. If you want to accept the
default settings for the installation, type the following command before moving
to step 7.
./dir5.10_agent_linppc.sh
Go to step 7.
3. To customize the installation, copy the response file (diragent.rsp) to a local
directory by entering this command:
cp diragent.rsp /directory
Here directory is the local directory.
4. Open an American Standard Code for Information Interchange (ASCII) text
editor and modify the installation settings in the copy of the diragent.rsp file.
This file is fully commented. You can specify the location of the RPM files and
select log file options.
5. Save the modified response file with a new file name.
6. To install IBM Director Agent using the response file, type the following
command and press Enter:
./dir5.10_agent_linppc.sh -r /directory/response.rsp
Here directory is the local directory to which you copied the response file, and
response.rsp is the name of the response file saved in step 5.
353
7. By default, encryption using the Advanced Encryption Standard (AES)
algorithm is enabled during installation. An optional step is to configure
encryption to a different setting. To disable encryption or change security
settings, type the following command and press Enter:
/opt/ibm/director/bin/cfgsecurity
8. To start IBM Director Agent, type the following command and press Enter:
/opt/ibm/director/bin/twgstart
9. If you installed IBM Director Agent from the CD you can now unmount the
CD-ROM.
Note: After we installed all the prerequisite RPM files, we experienced the
following scenarios:
With Red Hat V4 Update 3, we experienced an error at the end of the
installation of the IBM Director Agent shell script:
Failed Dependencies compat-libstdc++-33.3.2.3-47.3
lsvpd v0.12.7
librtas 1.2
ppc64-utils 2.5
After ensuring that we have already installed these files at the correct level,
we issued the twgstart command. The IBM Director Agent processes
started and performed as expected.
With SLES9 Upgrade 3, we experienced no errors with the installation shell
script, but we received an error when we issued the twgstart command.
Failed dependencies: lsvpd>=0.12.7 needed by
pSeriesCoreServices-level1-5.10.2.1-SLES9
We confirmed that we had already installed a more recent lsvpd level and
reissued the twgstart command. The IBM Director Agent started up with
no other problems.
354
10.2.2 Using the IBM Director Console in a BladeCenter context
After we installed the Agent on all blades and LPARs that we expected to work
with, we started the console on our Windows desktop system and began
managing the BladeCenter chassis, blades, and network devices. The
environment shown in Figure 10-1 on page 351 is a single chassis with only six
blades powered up. You can see from the following section that managing
multiple chassis with full complements of blades is much easier than attempting
to monitor and manage them solely with the BladeCenter Web Interface to each
individual chassis.
While the initial labeling of the BladeCenter elements is fairly intuitive after
discovery, we re-labeled some of the blades for easier tracking as shown in
Table 10-2.
Table 10-2 Blade location and names
Bay
Name
Blade type - machine
1
xSeries Bay1
HS20 8678-21X
2
JS21 VIOS Bay 2
JS21 8844-51X
3
JS21 Bay 3
JS21 8844-5CZ
4
JS20 Director Bay 4
JS20 8842-41X
5
JS20 NIM Bay 5
JS20 8842-21X
6
JS20 SLES Bay 6
JS20 8842-41X
355
An example of the inventory information available on a blade is shown in
Figure 10-2. This window is generated when we double-click the entry for JS21
VIOS Bay 2.
Figure 10-2 Inventory data
Blade management
When you install IBM Director Agent on a blade or LPAR, there are additional
functions that you can access. They include remote sessions (consoles), process
management, power management (restart), and others.
The IBM Director documentation available on the Web provides excellent
instructions on implementation. The objective of this section is to provide a brief
view of the basic features that are useful for the BladeCenter environment.
356
Discovering IBM Director Level 2 Agents
Figure 10-3 shows the display for systems discovered with Level 2 Director
Agents installed. You can see that one item shows a lock (linux.site). This is how
a newly discovered agent appears before any access to the agent is requested.
Also notice the right panel that displays the various options for selection. You can
initiate discovery by selecting Tasks → Discover → Level 2: IBM Director
Agents from the top menu bar. Alternatively, you can click the Flashlight icon to
perform the same task.
Figure 10-3 Level 2 Agents
The systems that we used for Agent installation and management as shown are:
linux.site (a SLES9 Linux LPAR on a JS21 blade)
js21a1.itsc.austin.ibm.com (an AIX LPAR on a JS21 blade)
js20ibmdirector2 (the IBM Director Server on a JS20 blade)
js20:linuxinstall1 (Red Hat Linux on a JS20 blade)
357
Power management
Figure 10-4 shows the selections available for actions on Level 2 Agents. You can
see this menu when you right-click any one of the entries in the center pane. In
this particular example, you see the power management selection which only
offers restart. This is only for rebooting the operating system and does not power
on or power off the blade itself.
Figure 10-4 Level 2 Agent: Menu selections
358
Remote console
Figure 10-5 displays the remote console that is provided when you select the
remote session option. IBM Director tries to access the system first through
Secure Shell (SSH) and then using Telnet if SSH fails. This screen reflects the
successful use of SSH on a SLES9 Linux LPAR.
Figure 10-5 Remote session
359
Resource monitors
Another useful task is the resource monitor. Select the system (agent) and then
the task either in the right task pane or right-click a particular system. This opens
a set of resources to select as shown in Figure 10-6. You can specify any number
of resources to monitor. In this case, we selected CPU Utilization and Disk
Monitors.
Figure 10-6 Resource monitors
360
If you determine that you have critical threshold for a given resource that is
important for your application environment, you can identify it as a selected
resource (for example, Space Available), and right-click the metric (for example,
30). This shows the data sheet to identify the threshold and notification data as
Figure 10-7 Resource threshold
BladeCenter management becomes much more efficient when you enlist the aid
of IBM Director.
361
For further information for the effective use of IBM Director, refer to Implementing
IBM Director 5.10, SG24-6188, and the documentation provided on the IBM
Director CD shipped with your IBM BladeCenter.
10.3 Cluster Systems Management
This section provides you with the CSM setup and installation tasks that are
relevant to the BladeCenter and BladeCenter JS21 environment. Cluster
implementation tasks are varied and dependent largely on your network and
switch usage and application parameters. We do not cover cluster
implementation in this section other than to show the successful establishment of
a CSM cluster using JS20 and JS21 blades over Ethernet.
The BladeCenter JS21 is now supported by CSM V1.5.1. There are specific
program temporary fixes (PTFs) for AIX V5.3 that you have to install along with
this version of CSM. Refer to the following Web site and ensure that you have
the authorized program analysis reports (APARs) or the referenced filesets
installed:
https://www14.software.ibm.com/webapp/set2/sas/f/csm/download/csmaix_1.
5.1.1down.html
For this exercise, we created a directory on our selected management server
/usr/sys/inst.images/csm and placed all the filesets in this directory and then
installed all the necessary filesets. We closely followed the updated instructions
in CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344,
and applied the V1.5.1 updates available on:
With the updates applied, you can follow the steps as outlined in CSM 1.5.0 for
AIX 5L and Linux: Planning and Installation Guide, SA23-1344. The basic steps
are as follows:
1.
2.
3.
4.
5.
6.
Set up the management server.
Set up one or more installation servers (optional).
Define the nodes in the cluster.
Define non-node devices to the cluster (optional).
Install the nodes of the cluster (optional).
Add the nodes to the cluster. You can add AIX, Linux, or both AIX and Linux
nodes.
Note that for this exercise, we did not run steps 2 and 4.
362
10.3.1 Installing and setting up the CSM management server
This section describes the steps to install and set up the CSM management
server.
Step 1: Start CSM installation
After you identify and install all the prerequisites noted previously, download the
full CSM V1.5.1 Update ISO image (60-day try-and-buy) and burn a CD. Use the
CD to install the CSM management server on a BladeCenter JS21 running AIX
V5.3.
After you assign the CD drive to the JS21 using the management module
interface, use the usual AIX commands to assign and mount /dev/cd0. Use the
smitty csm command and you see the screen shown in Figure 10-8.
Figure 10-8 SMIT CSM installation
363
When you select CSM Install, this generates a screen with multiple CSM
installation and configuration options as shown in Figure 10-9.
When you select Install the Management Server, this provides the options for
selecting your installation source (in our case, the CD), the filesets you want to
include, and the normal System Management Interface Tool (SMIT) software
installation options.
Figure 10-9 CSM installation options
Step 2: Register the host name
Confirm the registration of the server, client, and management module host
names and addresses with your name server. This is an important task that is
easily overlooked. Refer to “Installing the management server on AIX” of CSM
1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344.
For our exercise, we used a JS21 as the management server and set up an AIX
LPAR and SLES9 LPAR on a JS21 and another JS21 running AIX as compute
nodes.
364
Step 3: Update the $PATH and $MANPATH variables
Add /opt/csm/bin to the root user’s $PATH variable on the management server:
export PATH=$PATH:/opt/csm/bin
To access the CSM main pages, add /opt/csm/man to the root user’s $MANPATH
variable on the management server:
export MANPATH=$MANPATH:/opt/csm/man
To verify that this step is completed successfully, issue the following commands:
echo $PATH
echo $MANPATH
Note: The examples provided previously only show how to change the $PATH
and $MANPATH variables in the current login session. To permanently change
them, edit your login environment.
Step 4: Create the /csminstall file system
This is an optional step but we chose to implement a separate file system of
5 GB to anticipate AIX and Linux operating system resources. We also increased
the /VAR file system by 250 MB.
Step 5: Download CSM 1.5.1.1 updates
Download the csm-aix-1.5.1.1.power.tar.gz file from the following Web site to a
local PC:
http://www14.software.ibm.com/webapp/set2/sas/f/csm/home.html
Use File Transfer Protocol (FTP) to copy the file to the management server. You
can extract the file into the /tmp/csm directory and use the tar -xvf command to
extract the files. Use smit install with /tmp/csm as the source directory to
complete the update installation.
Step 6: Downloading open source software
Cluster Systems Manager uses several types of open source software to
manage various hardware devices.
Downloading openCIMOM for hardware control
Tip: This is only necessary for HMC hardware control. It was not necessary
for our purposes in a BladeCenter environment.
365
Downloading autoupdate
The autoupdate program is required to support the CSM Software Maintenance
System (SMS) feature that is available for updating software on Linux nodes.
You only require the software if you add Linux nodes to your CSM cluster.
You can download the autoupdate software from the following Web site:
To download the software, select the link under RPM package, then download
autoupdaterelease.noarch.rpm (for example, autoupdate-5.2.5-1.noarch.rpm).
Copy the RPM to a temporary directory, for example /tmp/csm/RPMS/ppc.
You do not have to install the RPM on the management server. The autoupdate
RPM is required only when you add Linux nodes to the cluster. You can postpone
downloading the autoupdate RPM until you are ready to follow the procedure to
add the Linux node.
In this exercise, because we have a Linux LPAR node we downloaded the
package to our PC and then used FTP to copy it to the suggested directory.
Step 7: Verify that the prerequisite software is installed
When AIX is installed, the following CSM base filesets are also automatically
installed:
csm.core
csm.client
csm.dsh
csm.diagnostics
If the filesets are not available, use AIX CD-ROM #1 to reinstall them.
You also have to check for the latest Reliable Scalable Cluster Technology
(RSCT) filesets. You can download the latest available from the following Web
site and install them:
https://techsupport.services.ibm.com/server/aix.fdc
366
Figure 10-10 shows a sample of the current output from the lslpp -L rsct*
command.
Fileset
Level State Type Description (Uninstaller)
---------------------------------------------------------------------------rsct.core.auditrm
2.4.5.0
C
F
RSCT Audit Log Resource
Manager
rsct.core.errm
2.4.5.0
C
F
RSCT Event Response Resource
Manager
rsct.core.fsrm
2.4.5.0
C
F
RSCT File System Resource
Manager
rsct.core.gui
2.4.5.0
C
F
RSCT Graphical User Interface
rsct.core.hostrm
2.4.5.0
C
F
RSCT Host Resource Manager
rsct.core.lprm
2.4.5.0
C
F
RSCT Least Privilege Resource
Manager
rsct.core.rmc
2.4.5.0
C
F
RSCT Resource Monitoring and
Control
rsct.core.sec
2.4.5.0
C
F
RSCT Security
rsct.core.sensorrm
2.4.5.0
C
F
RSCT Sensor Resource Manager
rsct.core.sr
2.4.5.0
C
F
RSCT Registry
rsct.core.utils
2.4.5.0
C
F
RSCT Utilities
Figure 10-10 Output from lslpp -L rsct* command
Step 8: Install CSM management server software
The following AIX filesets and RPM packages are available on AIX CD-ROM #1
product media.
CSM on AIX filesets:
–
–
–
–
csm.server
csm.gui.dcem
csm.gui.websm
csm.hpsnm
csm.gui.websm is a prerequisite for csm.hpsnm. If you have a High
Performance Switch (HPS) network in your cluster, csm.hpsnm is
required.
Open source software:
–
–
–
–
conserver
expect
tcl
tk
367
To install this software, insert CD-ROM #1 and use AIX Web-based system
manager, SMIT panels, or the AIX command-line interface. For example, you
can use the AIX geninstall command to install selected CSM software as
follows:
geninstall -IaX -d /dev/cd0 csm.server csm.gui.dcem \ csm.gui.websm
csm.hpsnm R:expect R:tcl R:tk R:conserver
Step 9: Apply CSM management server updates
This reconfirms that the updates we downloaded in “Step 5: Download CSM
1.5.1.1 updates” on page 365, have been applied. This has been done already.
Step 10: Check the remote shell attribute
Run the csmconfig command to check this attribute. Remember that your PATH
statement must include /opt/csm/bin. The output from this command on the
management server is as follows:
RemoteShell=/usr/bin/rsh
SetupRemoteShell = 1 (yes)
RemoteShell uses rsh as the default executable to run dsh for remote
commands. In our initial setup, we accepted the default and were not able to
communicate with our nodes. We first used a temporary override to OpenSSH by
using the command export DSH_REMOTE_CMD=/usr/bin/ssh. This allowed
immediate communication with those nodes where we had installed SSH and
had the daemon running. To make this the standard remote executable, we
issued the following command:
csmconfig RemoteShell=/usr/bin/ssh SetupRemoteShell=1
SetupRemoteShell Indicates to CSM that you want the remote shell to be
automatically configured (1 = yes, 0= no). Read more about this in CSM 1.5.0 for
AIX 5L and Linux: Planning and Installation Guide, SA23-1344.
Step 11: Install OpenSSH and OpenSSL (optional)
We installed CSM on a server that also functions as an IBM Director Server. Part
of this exercise is to install OpenSSH and OpenSSL. OpenSSH is available on
the AIX Expansion Pack CD. OpenSSL is available on the Linux Toolbox CD.
Use your standard Web-based Systems Manager (WebSM) or SMIT process to
install these from their respective CDs.
Step 12: Install Kerberos Version 5 for remote commands
(optional)
We chose not to implement this step initially.
368
Step 13: Accept the CSM license
See CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide,
SA23-1344, for the options if you have a full license.
We used a 60-day try-and-buy license. To accept the 60-day try-and-buy license
agreement, issue the csmconfig command:
csmconfig -L
You can check the success of the csmconfig command by running it with no
flags, and then checking the output as shown in Figure 10-11. You can also run
the license acceptance using the smit csm command.
AddUnrecognizedNodes = 0 (no)
BMCConsoleEncryptAuth = 1 (yes)
BMCConsoleKeepAlive = 0 (no)
BMCConsolePerMsgAuth = 0 (no)
ClusterSNum =
ClusterTM = 9078-160
DeviceStatusFrequency = 12
DeviceStatusSensitivity = 8
ExpDate = Sat Jul 22 18:59:59 2006
HAMode = 0
HeartbeatFrequency = 12
HeartbeatSensitivity = 8
MaxNumNodesInDomain = -1 (unlimited)
NetworkInstallProtocol = nfs
PowerPollingInterval = 300
PowerStatusMode = 1 (Events)
Properties =
RegSyncDelay = 1
RemoteCopyCmd = /usr/bin/rcp
RemoteShell = /usr/bin/rsh
SetupKRB5 = 0
SetupNetworkInstallProtocol = 1 (yes)
SetupRemoteShell = 1 (yes)
TFTPpackage = tftp-hpa
Figure 10-11 Output from the csmconfig command
Notice the ExpDate entry because we ran a 60-day license.
369
Step 14: Copy CSM files into the /csminstall subdirectories
Issue the following csmconfig command to copy the CSM files into the proper
/csminstall subdirectories:
csmconfig -c
These files are primarily used when CSM system management scripts are run on
the nodes of the cluster.
Note: You can combine the -c option with the -L option mentioned in the
previous step. Therefore, for example, you can just run csmconfig -c -L
instead of running the command twice.
Use the following command to set up the management server to place Linux
CSM installation files into their appropriate directories:
copycsmpkgs -p /csminstall/Linux InstallCSMVersion=1.5.1
InstallOSName=Linux InstallDistributionName=SLES
InstallDistributionVersion=9 InstallPkgArchitecture=ppc64
We already downloaded the Linux CSM software into the directory
/csminstall/Linux. You can download the file csm-linux-1.5.1.1.ppc64.tar.gz from
the following Web site using the 60-day try-and-buy license:
https://www14.software.ibm.com/webapp/set2/sas/f/csm/download/home.html
Extract the file in a temporary directory using the gunzip command and then
extract the files from the resulting tar file using the tar -xvf
csm-linux-1.5.1.1.ppc64.tar command. The tar file is 106 MB in size.
Therefore, you must ensure that your temporary directory or your /csminstall file
system has sufficient space to accommodate it.
Step 15: Set the cluster ID (optional)
This step is only appropriate for 1600 clusters that are assigned a serial number.
We did not use it in our context.
Step 16: Store hardware control point user IDs and passwords
This step covers several configuration options including HMC managed IBM
System p devices.
BladeCenter HS20 (other than HS20-8678), HS40, JS20, and JS21 servers
use the Serial over LAN (SoL) feature to provide remote console access. For
our BladeCenter environment, we followed these specific instructions from
the CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide,
SA23-1344.
370
BladeCenter 8677: To configure BladeCenter management modules, change
the default hardware control point login profile user ID and password using
the management module Web interface and documentation provided with the
hardware. For BladeCenter management modules, the default login profile
user ID is USERID, and the default password is PASSW0RD.
(P-A-S-S-W-zero-D).
BladeCenter blade servers, or IBM eServer xSeries 336, and xSeries 346
servers using SoL to access remote console: If the BladeCenter contains
HS20 (other than HS20-8678), HS40, JS20, or JS21 blade servers, or
xSeries 336 or xSeries 346 servers using SoL instead of a console server
device, run the systemid command with the -c option to store the login profile
user ID and password required to access the SoL feature.
For BladeCenter advanced management modules, the default console user
ID is RMTCON, and the default console password is RMTC0N
(R-M-T-C-zero-n). Change the default console login profile user ID and
password using the management module Web interface and documentation
provided with the hardware. You must not use a login profile with supervisor
privileges for console access. If the default console login profile user ID is not
used, the new login profile must only be granted Blade Console remote
access privileges.
Use the systemid -c command or the smit systemid command to complete this
step. We used the systemid -c command, as shown in Figure 10-12.
# systemid -c mmext.itsc.austin.ibm.com ausres08
New Password:
Verifying, please re-enter password:
systemid: following entries were created
9.3.5.220_console
Figure 10-12 systemid command
The command syntax in this case is systemid -c <management module external
ethernet hosthame> <userid>
Step 17: Install any user-defined power methods
We do not provide any details about this step. See CSM 1.5.0 for AIX 5L and
Linux: Planning and Installation Guide, SA23-1344, for further instructions, if
necessary.
371
Step 18: Verify the installation
To verify that the management server is installed correctly and is ready for use,
you can run the ibm.csm.ms probe, which is shipped with CSM. To run the probe
issue the following command
probemgr -p ibm.csm.ms -l 0
The output from the command provides a view of each of the checks performed
and a final summary statement. The output in Figure 10-13 shows the end of the
output from our exercise.
Probe ibm.csm.ms returned the following information.
ibm.csm.ms:trace:Checking that directory /opt/csm exists.
ibm.csm.ms:trace:Checking that directory /opt/csm/bin exists.
ibm.csm.ms:trace:Checking that directory /opt/csm/csmbin exists.
ibm.csm.ms:trace:Checking that directory /opt/csm/install exists.
ibm.csm.ms:trace:Checking that directory /usr/sbin/rsct/bin exists.
ibm.csm.ms:trace:Checking that directory /csminstall exists.
ibm.csm.ms:trace:Checking that directory /var/log/csm exists.
ibm.csm.ms:trace:Checking that directory /opt/csm/install/defs
exists.
ibm.csm.ms:trace:Checking that directory /opt/csm/install/pkgdefs
exists.
ibm.csm.ms:trace:Checking that directory /csminstall/csm/config/
exists.
ibm.csm.ms:trace:Checking that directory /csminstall/csm/status
exists.
ibm.csm.ms:trace:Checking for packages : csm.client*, csm.core*,
csm.deploy*, csm.diagnostics*, csm.dsh*, csm.server*.
ibm.csm.ms:trace:Checking for packages : conserver-8.1.7, expect*,
openCIMOM-0.8-1, tcl*, tk*.
ibm.csm.ms:trace:Check if the CFM cronjob is enabled.
Probe ibm.csm.ms was run successfully.
Figure 10-13 Sample output of the probemgr command
There are additional steps included in CSM 1.5.0 for AIX 5L and Linux: Planning
and Installation Guide, SA23-1344, such as installing Kerberos as an option,
which we did not use. It also provides a step-by-step instruction about how to
install CSM on a Linux Management Server. We did not implement this for our
test.
372
10.3.2 Creating a CSM installation server
CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344,
provides clear instructions on the various options for setting up installation
servers. With multiple blades, LPARs, or both to contend with, you might want to
give this serious attention. We did not set up an installation server for the few
nodes that we already installed with their respective operating systems.
10.3.3 Defining cluster nodes
Although CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide,
SA23-1344, supplies both general and specific attributes for the myriad platform
options available, we focus on those that apply to the BladeCenter context and
the JS20 or BladeCenter JS21 nodes that we tested.
Determining node attributes
As part of the CSM cluster installation process, gather information that is used to
define the cluster nodes. The pieces of information that make up a node
definition are referred to as node attributes. The complete list of node attributes
and descriptions are documented in the node attributes man page.
This section provides information about the attributes that are required when you
define nodes and how to determine which values to use. The information is
divided into three categories; general attributes, hardware control information,
and information about installation software.
General attributes
The general node attributes are:
Hostname
The resolvable host name or IP address of the node, as known by the
management server. It represents the network adapter host name or IP
address on the cluster virtual local area network (VLAN). Host name is always
required, and you must specify it when you define the node. In a pure Linux
cluster, this attribute can be an unresolved IP address.
ManagementServer
The host name of the CSM management server. ManagementServer is
always required. Set it to the host name of the management server as it is
known by the node. Because the management server can have multiple
interfaces, different nodes might use different interfaces to communicate with
the management server. If a route to the node exists at the time that the node
is defined, CSM attempts to set the value to the IP address of the
management server automatically. If a route to the node does not exist, CSM
373
cannot set the value automatically. Therefore, set the name explicitly to the
host name of the management server in the network that installs the node.
InstallServer
The host name or IP address of the installation server as known by the
management server, and an optional directory that serves the CSM
installation files to this node.
InstallAdapterHostname
This is the host name or IP address of the node as known by the installation
server. That is, it is the host name by which the node is connected to during
the installation and that is assigned to the installation adapter when it is
configured on the node during an installation. If this attribute is left blank, the
Hostname attribute value is used by default.
InstallServerAKBNode
This is the host name or IP address of the installation server as known by the
node. If there are multiple Linux installation servers on the same subnet, this
attribute is ignored, and the network connection on the installation server that
is used by the node is dynamically determined. If left blank, CSM determines
this value dynamically during run time. If there are multiple connections from
a Linux installation server to the installation subnet that the node is on, one
connection is chosen at random.
UserComment
This is any useful information concerning this node. Not required.
Mode
The node’s current phase in the installation process. Set this attribute to
MinManaged or run the definenode command with the minmanaged option
when you define a node to be minimally managed.
If you do not set the Mode attribute of the node, CSM sets it automatically
during the different phases of the installation.
Hardware control attributes
You must define the hardware-related attributes for the nodes if hardware control
is to be configured on the cluster. In some cases, default values are provided. If
these defaults are acceptable, you do not have to provide the attribute values
when you define the node. See CSM 1.5.0 for AIX 5L and Linux: Planning and
Installation Guide, SA23-1344. For a complete description of the meaning and
use of these attributes, see IBM CSM for AIX 5L and Linux: Administration
Guide, SA23-1343.
374
For BladeCenter blade servers, the following attributes apply to models HS20,
HS40, JS20, and JS21 blade servers:
PowerMethod
Set this value to blade.
HWControlPoint
Use the host name of the BladeCenter management module.
HWControlNodeId
Use the blade Advanced System Management (ASM) name.
The following attributes apply only to the JS20 and JS21 blade servers, and are
valid only if the SoL feature is enabled on the BladeCenter management module:
ConsoleMethod
Set this value to blade.
ConsoleServerName
Use the host name of the BladeCenter management module.
ConsolePortNum
Use the blade slot number within the BladeCenter chassis.
ConsoleSerialDevice
Leave this field blank. This field is not used for JS20 blade servers. If the SoL
feature is not enabled, set the ConsoleSerialDevice attribute for all blades to
NONE, and leave the ConsoleMethod, ConsoleServerName, and
ConsolePortNum attributes blank.
Using the definenode command
Use the definenode command to establish and set up individual nodes, one at a
time. With a large number of nodes, you might want to consider the multinode
options as explained in the CSM 1.5.0 for AIX 5L and Linux: Planning and
Installation Guide, SA23-1344, and in the CSM 1.5.0 for AIX 5L and Linux:
Command and Technical Reference, SA23-1345.
To establish one of the JS21 AIX LPARs as a node, issue the following
command:
definenode -n l2_aix_1 Hostname=l2_aix_1 CSMVersion=1.5.1
ConsoleMethod=blade ConsolePortNum=2 CondoleServerName=mmext
HWControlNodeId=JS21_VIO HWControlPoint=mmext
InstallDistributionVersion=5.3.0 InstallOSName=AIX
ManagementServer=ibmdirector2 PowerMethod=blade
375
The output of the lsnode -Al l2_aix_1 command (the command which lists the
node attributes of a defined note) is shown in Figure 10-14.
# lsnode -Al l2_aix_1
Hostname = l2_aix_1
AdapterStanzaFile =
AllowManageRequest = 0 (no)
CSMVersion = 1.5.1
ChangedAttributes = {}
ConfigChanged = 0 (no)
ConsoleMethod = blade
ConsolePortNum = 2
ConsoleSerialDevice =
ConsoleSerialSpeed = 9600
ConsoleServerName = mmext
ConsoleServerNumber =
FWSvcProc =
FWSysBIOS =
HWControlNodeId = JS21_VIOS
HWControlPoint = mmext
HWModel =
HWSerialNum =
InstallOSName = AIX
InstallStatus = PreManaged
InstallTemplate =
LICManagedSystemLevel =
LICPowerSubsystemLevel =
LParID =
LastCFMUpdateTime =
ManagementServer = 9.3.5.236
Mode = PreManaged
NFSServer =
Name = l2_aix_1
NodeNameList = {js20ibmdirector2}
PhysicalLocation =
PowerMethod = blade
PowerStatus = 127 (unknown)
Properties =
Status = 127 (unknown)
UUID =
UpdatenodeFailed = 0 (false)
UserComment =
Figure 10-14 Output of the lsnode command
376
As you can see from the preceding section, defining and managing more than a
few nodes individually is time consuming. If many nodes are similar in
construction, you can use more efficient methods such as host name mapping
files and node definition files to reduce the administrative work. This is described
in detail in CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide,
SA23-1344.
When you have an environment with multiple blades in multiple chassis, you can
use the installation server setup and appropriate InstallServer attributes to
enable you to install the operating system and updates to the CSM software as
necessary. In our test, we had a limited number of nodes and did not use an
installation server for our exercise.
10.3.4 Adding AIX nodes to the cluster
There are multiple preparatory steps that you must consider before adding a
node. These are spelled out in CSM 1.5.0 for AIX 5L and Linux: Planning and
Installation Guide, SA23-1344. See Chapter 11, “Adding AIX nodes to the
cluster” of this guide for information about this topic. For the purposes of this
exercise, we completed all the prerequisites and focus on the updatenode
command.
The updatenode command adds the AIX nodes to the cluster. Run the
updatenode command for the AIX nodes that you have defined. The updatenode
command does the following functions for an AIX node:
If remote shell authentication is not already set up, automatically sets up
remote shell authentication for OpenSSH or rsh
Distributes configuration files if the configuration file manager (CFM) is set up
Set up CFM before you install your nodes to avoid customizing the nodes
later. For information about how to configure CFM, see IBM CSM for AIX 5L
and Linux: Administration Guide, SA23-1343.
Runs any user customization scripts
Sets up the Kerberos Version 5 options for remote commands if requested on
the csmconfig command
377
Before you run updatenode, ensure the following prerequisites:
The management server is installed and the AIX nodes are defined
The AIX nodes have all the prerequisite AIX software. Ensure that the
following CSM software is installed on the AIX nodes (This software is
installed automatically with AIX. If for some reason it is uninstalled, you must
ensure that it is reinstalled):
– csm.core
– csm.client
If you plan to use Kerberos remote command processing, the necessary
software is installed
The AIX nodes are up and running
It is not necessary to update the cluster nodes with the latest available CSM
software updates unless the updates are specifically required. See the following
CSM support Web site for information about any required updates for the nodes:
http://techsupport.services.ibm.com/server/cluster/fixes
We updated the CSM node files csm.core and csm.client to V1.5.1 on all nodes.
For our exercise, we issued the updatenode command for each node
independently. In each case, we were prompted to supply the root password for
the node as shown in Figure 10-15.
# updatenode l2_aix_1
updatenode: 2653-206 dsh, using protocol /usr/bin/ssh, cannot
connect to nodes: l2_aix_1.itsc.austin.ibm.com.
Please enter the password for the current user (normally root) to
access the nodes l2_aix_1.itsc.austin.ibm.com:(Password entered)
Setup complete for remote shell: /usr/bin/ssh
Now running updatenode.client on the nodes.
l2_aix_1.itsc.austin.ibm.com: Setting Management Server to
9.3.5.236.
l2_aix_1.itsc.austin.ibm.com: Node Install - Successful.
l2_aix_1.itsc.austin.ibm.com: Output log is being written to
"/var/log/csm/install.log".
Now running CFM to push /cfmroot files the nodes.
There are no files in /cfmroot.
Figure 10-15 updatenode command applied to AIX node
378
To validate that the node is successfully added to the cluster, issue the lsnode
command with the node name and appropriate attributes to be shown.
Figure 10-16 shows the output for this command.
# lsnode -n js21nim -a Mode,UpdatenodeFailed
js21nim: Managed, 0
Figure 10-16 lsnode command
You can see that the Mode is now Managed and the UpdatenodeFailed is 0 (for
No).
The /var/log/csm/updatenode.log on the management server also provides clues
to recent updatenode activity.
10.3.5 Adding Linux nodes to the cluster
For this exercise, we established a SLES9 SP3 LPAR on a JS21 blade. We
defined it using the definenode command and ensured that all the attributes
followed the instructions in CSM 1.5.0 for AIX 5L and Linux: Planning and
Installation Guide, SA23-1344.
379
Figure 10-17 shows the output of lsnode -Al l4_sles9_1 command, where
l4_sles9_1 is the host name for the LPAR. We edited the output so that only the
essential attributes are shown.
# lsnode -Al l4_sles9_1
Hostname = l4_sles9_1.itsc.austin.ibm.com
CSMVersion = 1.5.1
ChangedAttributes = {AllowManageRequest}
ConsoleMethod = blade
ConsolePortNum = 2
ConsoleSerialDevice =
ConsoleSerialSpeed = 9600
ConsoleServerName = mmext
HWControlNodeId = JS21_VIOS
HWControlPoint = mmext
InstallCSMVersion=1.5.1
InstallDistributionName = SLES
InstallDistributionVersion = 9
InstallKernelVersion =
InstallMethod = autoyast
InstallOSName = Linux
InstallPkgArchitecture = ppc64
InstallServer =
InstallServerAKBNode =
InstallServiceLevel = SP3
InstallStatus = PreManaged
ManagementServer = 9.3.5.236
Mode = PreManaged
NFSServer =
Name = l4_sles9_1.itsc.austin.ibm.com
NodeNameList = {js20ibmdirector2}
PowerMethod = blade
PowerStatus = 127 (unknown)
Properties =
Status = 127 (unknown)
UUID =
UpdatenodeFailed = 1 (true)
UserComment =
Figure 10-17 lsnode -Al output for host l4_sles9_1
380
Using the copycsmpkgs command
The next step is to ensure that the management server is prepared to install
either the operating system or CSM or both. In this case, the OS is installed.
Ensure that the CSM software and open source software identified in CSM 1.5.0
for AIX 5L and Linux: Planning and Installation Guide, SA23-1344, is loaded in
the correct directories.
Download the CSM code for Linux and placed it in the /csminstall/Linux directory
on the management server. Keep the SLES9 Service Pack 3 (SP3) Linux CD
ready for use. They contain the required open source software.
Issue the copycsmpkgs -p /csminstall/Linux -n l4_sles9_1 command and
follow the prompts as it completes the task. The command process prompts you
for the distribution media.
Tip: Though you might want to mount the distribution media in the CD-ROM
tray, do not mount it. The copycsmpkgs command scripts mount it for you.
There will be confusion if you mount it before issuing the command.
Using the updatenode command
When the copycsmpkgs command is run successfully, you are ready to run the
updatenode command. The command runs with a default -I flag against the node
selected and begins to install the necessary CSM for Linux files and open source
files. If any files that are considered to be necessary are not found, the command
notifies you.
381
Figure 10-18 shows the output of the command run on our Linux LPAR.
# updatenode -n l4_sles9_1
updatenode: 2653-073 The Autoupdate RPM is missing from
/csminstall/Linux/SLES/csm/1.5.1/packages/. This means that
Autoupdate is probably not installed on the nodes. Please download
the Autoupdate RPM from http://freshmeat.net/projects/autoupdate and
place it in /csminstall/Linux/SLES/csm/1.5.1/packages/.
Now running updatenode.client on the nodes.
l4_sles9_1.itsc.austin.ibm.com: Installing
autoupdate-5.4.1-1.noarch.rpm.
l4_sles9_1.itsc.austin.ibm.com: The following OPTIONAL RPMs will not
be copied or installed (because they could not be found). This may
prevent the use of some CSM functionality or optional features.
Please consult the CSM Planning and Installation Guide for more
information:
l4_sles9_1.itsc.austin.ibm.com: perl-RPM2
l4_sles9_1.itsc.austin.ibm.com: Setting Management Server to
9.3.5.236.
l4_sles9_1.itsc.austin.ibm.com: Node Install - Successful.
l4_sles9_1.itsc.austin.ibm.com: Output log is being written to
"/var/log/csm/install.log".
Now running CFM to push /cfmroot files the nodes.
There are no files in /cfmroot.
Figure 10-18 Output from the updatenode command
You can see from Figure 10-18 that at least one fileset (perl_RPM2) was not
found. However, the command completed successfully and the node is now
shown as a managed node.
382
10.3.6 Testing cluster installation and basic command
Because the distributed shell is used for cluster commands, you can use it to test
your cluster. Issue the dsh -a date command. Figure 10-19 shows the output.
# dsh -as date
l4_sles9_1.itsc.austin.ibm.com: Tue Jun 6 06:17:52 PDT 2006
js21nim.itsc.austin.ibm.com: Tue Jun 6 08:16:11 CDT 2006
l2_aix_1.itsc.austin.ibm.com: Tue Jun 6 08:11:39 CDT 2006
#
Figure 10-19 Output from the dsh command
Node connectivity test
To ensure that the nodes are reachable, issue the lsnode -p command. You see
an output similar to Figure 10-20.
# lsnode -p
js21nim: 1 (alive)
l2_aix_1: 1 (alive)
l4_sles9_1: 1 (alive)
Figure 10-20 Output from the lsnode command
Your cluster is now active. You can begin normal cluster implementation tasks
that are appropriate for you application environment.
383
384
11
Chapter 11.
Performance tuning
This chapter discusses some of the useful tools and commands to maximize the
performance of your JS21. Some of the sections provide suggested configuration
changes with expected results to aid your implementation.
There are many factors that you have be address when looking at the
performance of a server. These include both software and hardware
configurations and how they interact with the workload.
385
11.1 Performance tuning for AIX
Performance tuning for AIX is discussed in detail in AIX 5L Version 5.3
Performance Management Guide, SC23-4905. You can find this on your AIX
documentation CD (see Infocenter) or online at:
http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp
On this Web site, on the left pane, select AIX documentation → AIX PDFs. On
the right pane, under the topic Performance management and tuning, select
Performance Management Guide.
Because there is another IBM Redbook that is written for AIX 5L, this chapter
concentrates on Linux. The AIX 5L performance tuning redbook is called: AIX 5L
Practical Performance Tools and Tuning Guide, SG24-6478.
11.2 Tuning options for Linux on POWER: Selected
examples
The Linux distributions that support the BladeCenter JS21 are built on the
2.6 kernel. The 2.6 kernel can take advantage of many Advanced Performance
Optimization with Enhanced RISC (POWER) processor features. We look at how
you can configure the Linux operating system (OS) to operate efficiently on this
platform.
11.2.1 Disabling unused processes
There are daemons (background services) running on every server that are
probably not required. Disabling these daemons frees memory, decreases
startup time, and decreases the number of processes that the central processing
unit (CPU) has to handle. A side benefit to this is increased security of the server
because fewer daemons mean fewer exploitable processes.
386
Table 11-1 shows some of the possible daemon processes that you can disable.
Table 11-1 Daemon processes that can be disabled
Daemons
Description
apmd
Advanced power management daemon
cups
Common UNIX printing system
hpoj
Hewlett-Packard (HP) OfficeJet Support
isdn
Integrated Services Digital Network (ISDN) modem support
netfs
Used in support of exporting Network File System (NFS) shares
nfslock
Used for file locking with NFS
pcmcia
Personal Computer Memory Card International Association (PCMCIA)
support on a server
portmap
Dynamic port assignment for Remote Procedure Call (RPC) services
(such as network information services (NIS) and NFS)
rhnsd
Red Hat Network update service for checking for updates and security
errata
sendmail
Mail transport agent
xfs
Font server for X Windows
Disabling daemons on Red Hat
In Red Hat, use the /sbin/chkconfig command to work with daemons. For
example, to stop the sendmail daemon immediately, enter the following
command as root:
/sbin/service sendmail stop
If you do not want the daemon to start the next time the machine boots, issue
either one of the following commands as root because they both accomplish the
same results:
/sbin/chkconfig --levels 2345 sendmail off
/sbin/chkconfig sendmail off
Similarly, there is a graphical user interface (GUI)-based program to choose
what daemons are started. To run the GUI, from the Red Hat desktop, select
Main Menu → System Settings → Server Settings → Services. Alternatively,
issue the following command:
/usr/bin/system-config-services
Chapter 11. Performance tuning
387
Disabling daemons on SUSE Linux
In SUSE Linux, you can temporarily stop most daemons with the stop parameter.
For example, to stop the sendmail daemon immediately, enter the following
command as root:
/etc/init.d/sendmail stop
In addition, SUSE Linux Enterprise Server (SLES) has three ways to work with
daemons:
A text-based UI: /sbin/yast runlevel
A GUI, Yet Another Setup Tool 2 (YaST2), which you can be start with the
following command:
/sbin/yast2 runlevel
Alternatively, you can open Yast2 by clicking Browse: YaST/ → YaST
modules → System → Runlevel editor.
The /sbin/chkconfig command
If you do not want the daemon to start the next time the machine boots, enter
the following command as root:
/sbin/chkconfig -s sendmail off
11.2.2 Disabling the graphical user interface
Whenever possible, do not run the GUI on a Linux server. When you do not run a
GUI, you have more system resources, including memory, for the applications.
Often, a GUI is not required on a Linux server, especially a JS21. You can
perform all administration tasks using the command line, redirecting the X
display, or through a Web browser interface. There are several useful
Web-based tools (for example, webmin, Linuxconf, and Samba Web
Administration Tool (SWAT)).
If a GUI is required, then start and stop it as necessary rather than running it all
the time. In most cases, the server must be running at run level 3, which does not
start the GUI when the machine boots. If you want to restart the X Server, use
startx from a command prompt. Run level 3 is multi-user mode without a GUI.
388
Setting the run level
To set the run level:
1. Determine the run level at which the machine is running with the following
command:
runlevel
This prints the previous and current run level (for example, N 5 means that
there was no previous run level (N) and that the current run level is 5).
2. To switch between run levels, use the init command. For example, to switch
to run level 3, enter the command:
init 3
The following short description provides the different run levels that are used
in Linux:
– 0: Halt (Do not set initdefault to this because the server immediately shuts
down after boot.)
– 1: Single user mode
– 2: Multi-user, without NFS (the same as 3, if you do not have networking)
– 3: Full multi-user mode
– 4: Unused
– 5: X11
– 6: Reboot (Do not set initdefault to this because the server machine
continuously reboots.)
3. To set the initial run level of a machine at boot, modify the /etc/inittab file with
the line as shown in Figure 11-1.
id:3:initdefault:
389
Figure 11-1 Run level and getty edits
11.2.3 Tuning the kernel parameters
You can make changes to the kernel by modifying parameters that control the
OS. Make these changes on the command line using the sysctl command. In
addition, Red Hat offers a graphical method of modifying these sysctl
parameters. To launch the tool, as shown in Figure 11-2, issue the following
command:
/usr/bin/system-config-proc
390
Tip: By default, the kernel includes the necessary module to enable you to
make changes using sysctl without having to reboot. However, If you choose
to remove this support (during the operating system installation), then you
have to reboot Linux before the change takes effect.
Figure 11-2 system-config-proc
The kernel parameters that control how the kernel behaves are stored in /proc
(and, in particular, /proc/sys), as shown in Table 11-2. Reading the files in the
/proc directory tree provides a simple way to view configuration parameters that
are related to the kernel, processes, memory, network, and other components.
Each process running in the system has a directory in /proc with the process ID
(PID) as name.
Table 11-2 Kernel parameters
File/directory
Purpose
/proc/sys/vm/*
Management of cache memory and buffer
/proc/stat
Kernel statistics as process, swap, and disk input/output (I/O)
/proc/cpuinfo
Information about the installed CPUs
391
File/directory
Purpose
/proc/meminfo
Information about memory usage. The free command uses this information.
/proc/sys/abi/*
Used to provide support for foreign binaries to Linux: Those compiled under other
UNIX variants such as SCO Unixware 7, SCO OpenServer, and SUN Solaris™ 2.
By default, this support is installed, although you can remove it during installation.
/proc/sys/fs/*
Used to increase the number of open files that the OS allows and to handle quota
/proc/sys/kernel/*
For tuning purposes, you can enable hotplug, manipulate shared memory, and
specify the maximum number of pid files and level of debug in syslog.
/proc/sys/net/*
Tuning of network in general, Internet Protocol Version 4 (IPV4) and IPV6
An example of using sysctl
The sysctl commands use the names of files in the /proc/sys directory tree as
parameters. For example, to modify the shmmax kernel parameter, you can
display (using cat) and change (using echo) the file /proc/sys/kernel/shmmax as
shown in Example 11-1.
Example 11-1 Manually updating /proc/sys/kernel/shmmax
#cat /proc/sys/kernel/shmmax
33554432
#echo 33554430 > /proc/sys/kernel/shmmax
#cat /proc/sys/kernel/shmmax
33554430
However, using these commands can easily introduce errors. Therefore, we
recommend that you use the sysctl command because it checks the
consistency of the data before it makes any change, as shown in Example 11-2.
Example 11-2 sysctl command
#sysctl kernel.shmmax
kernel.shmmax = 33554432
#sysctl -w kernel.shmmax=33554430
#sysctl kernel.shmmax
392
This change to the kernel stays in effect only until the next reboot. If you want to
make the change permanently, then you can edit the /etc/sysctl.conf file and add
the appropriate command, as shown in Example 11-3.
Example 11-3 /ec/sysctl.conf
The next time you reboot, the parameter file is read. You can obtain the same
result without rebooting by issuing the following command:
#sysctl -p
Table 11-3 shows a list of sysctl parameters that you can configure.
Table 11-3 sysctl parameters
Parameter
Description or example of use
net.ipv4.inet_peer_gc-maxtime
Sets how often the garbage collector (GC) must pass over the inet peer
storage memory pool during low or absent memory pressure. Default
is 120, measured in jiffies:
sysctl -w.ipv4.inet_peergc_maxtime=240
net.ipv4.inet_peer-gc-mintime
Sets the minimum time that the GC can pass cleaning memory. If your
server is heavily loaded, you might want to increase this value. Default
is 10, measured in jiffies:
sysctl -w net.ipv4.inet_peer_gc_mintime=80
net.ipv4.inet_peer_maxttl
The maximum time-to-live for the inet peer entries. New entries expire
after this period of time. Default is 600, measured in jiffies:
sysctl -w net.ipv4.inet_peer_maxttl=500
net.ipv4.inet_peer_minttl
The minimum time-to-live for inet peer entries. Set to a high enough
value to cover fragment time-to-live in the reassembling side of
fragmented packets. This minimum time must be smaller than
net.ipv4.inet_peer_threshold. Default is 120, measured in jiffies:
sysctl -w net.ipv4.inet_peer_minttl=80
net.ipv4.inet_peer_threshold
Sets the size of inet peer sotrage. When this limit is reached, peer
entries are thrown away, using the inet_peer_gc_mintime timeout.
Default is 65644:
sysctl -w net.ipv4.inet_peer_threshold=65644
net.core.rmem_max
Maximum receive window. Default is 131071:
sysctl -w net.core.rmem_max 16777216
net.core.wmem_max
Maximum Transmission Control Protocol (TCP) send window. Default
is 131071:
sysctl -w net.core.wmem_max 16777216
393
Parameter
Description or example of use
net.ipv4.tcp_rmem
Memory reserved for TCP receive buffers. Default is 4096 87380
174760:
sysctl -w net.ipv4.tcp_rmem 4096 87380 16777216
net.ipv4.tcp_wmem
Memory reserved for TCP send buffers. Default is 4096 65536 174760:
sysctl -w net.ipv4.tcp_wmem 4096 65536 16777216
vm.hugetlb_pool
The hugetlb feature works the same way as bigpages, but after hugetlb
allocates memory, the physical memory can only be accessed by
hugetlb or shm allocated with SHM_HUGETLB. It is normally used with
databases such as Oracle or IBM DB2®. Default is 0:
sysctl -w vm.hugetlb_pool=4608
vm.inactive_clean_percent
Designates the percent of inactive memory that must be cleaned.
Default is 5%:
sysctl -w vm.inactive_clean_percent=30
vm.pagecache
Designates how much memory must be used for page cache. This is
important for databases such as Oracle and DB2. Default is 1 15 100.
The three values of this parameter are:
Minimum percent of memory used for page cache. Default is 1%
The initial amount of memory for cache. Default is 15%
Maximum percent of memory used for page cache. Default is
100%
sysctl -w vm.pagecache=1 50 100
11.2.4 Disabling simultaneous multithreading
If an application's performance does not meet expectations and it is suspected
that the application may not be multithreading efficiently, it might be helpful to
disable the simultaneous multithreading (SMT) feature on a POWER system.
You can do this by modifying /etc/lilo.conf and adding the smt-enable=off
parameter to the list of kernel boot parameters.
11.2.5 Tuning memory
Improvements have been made in the 2.6 kernel. You do not have bdflush to
tune any more, which was a 2.4 kernel tuning option. Look at your application
installation procedure to determine whether there are recommended changes to
configure with the sysctl command.
394
11.2.6 Tuning the file system
Disk accesses are usually measured in milliseconds and are thousands of times
slower than other components (such as memory or Peripheral Component
Interconnect (PCI) operations, which are measured in nanoseconds or
microseconds). Different file systems are available for Linux that differ in
performance and scalability.
Besides storing and managing data on the disks, file systems are also
responsible for guaranteeing data integrity. The newer Linux distributions include
journaling file systems as part of their default installation. Journaling, or logging,
prevents data inconsistency in case of a system crash. All modifications to the
file system metadata have been maintained in a separate journal or log and can
be applied after a system crash to bring it back to its consistent state. Journaling
also improves recovery time, because there is no need to perform file system
checks at system reboot.
As with other aspects of computing, you find that there is a trade-off between
performance and integrity. However, as Linux servers make their way into
corporate data centers and enterprise environments, requirements such as high
availability can be addressed. A server’s disk subsystems can be a major
component of overall system performance. Understanding the function of the
server is key to determining whether the I/O subsystem has a direct impact on
performance.
Examples of servers where disk I/O is most important:
A file and print server must move data quickly between users and disk
subsystems. Because the purpose of a file server is to deliver files to the
client, the server must initially read all data from a disk.
A database server’s ultimate goal is to search and retrieve data from a
repository on the disk. Even with sufficient memory, most database servers
perform large amounts of disk I/O to bring data records into memory and flush
modified data to disk.
Examples of servers where disk I/O is not the most important subsystem:
An e-mail server acts as a repository and router for electronic mail and tends
to generate a heavy communication load. Networking is more important for
this type of server.
A Web server that is responsible for hosting Web pages (static, dynamic, or
both) benefits from a well-tuned network and memory subsystem.
395
11.2.7 Selecting the disk technology
Table 11-4 lists the disk technologies that are commonly used in desktops and
servers. The JS21 uses the Serial-Attached SCSI (SAS) interface for the optional
internal drives.
Table 11-4
Disk technologies
Technology
Cost
Function
Limitations and capabilities
Enhanced
Integrated
Drive
Electronics
(EIDE)
Lowest cost
Direct-attached
storage; for example,
low-end servers, local
storage (IBM eServer
x305)
An extension of Integrated Drive Electronics
(IDE) that is used for connecting internal
storage. Maximum: Two drives per EIDE
controller
Small
Computer
System
Interface
(SCSI)
Low cost
Direct-attached
mid-range to high-end
server with local
storage (x346, x365)
Although the standard for more than 10
years, current I/O demands on high-end
servers have stretched the capabilities of
SCSI. Limitations include cable lengths,
transfer speeds, maximum number of
attached drives, and limits on the number of
systems that can actively access devices on
one SCSI bus, affecting clustering
capabilities.
Serial ATA
Low cost
Midrange data-storage
applications
Generally available since late 2002, this new
standard in hard disk drive (HDD) or system
board interface is the follow-on technology to
EIDE. With its point-to-point protocol,
scalability improves as each drive has a
dedicated channel. Sequential disk access
is comparable to SCSI but random access is
less efficient. Redundant Array of
Independent Disks (RAID) functionality is
also available.
SerialAttached SCSI
Low cost
Direct-attached
mid-range to high-end
server with local
storage (JS21)
Generally available since late 2003, this new
standard in HDD or system board interface
provides performance and also reliability.
396
Technology
Cost
Function
Limitations and capabilities
iSCSI
Medium cost
Mid-end storage; for
example, File/Web
server
Became a Request For Comment (RFC)
recently. Currently being targeted toward
mid-end storage and remote booting.
Primary benefits are savings in infrastructure
cost and diskless servers. It also provides
the scalability and reliability associated with
TCP/IP/Ethernet. High latency of TCP/IP
limits performance.
Note: Red Hat Enterprise Linux currently
does not support iSCSI.
Fibre Channel
High cost
Enterprise storage; for
example, databases
Provides low latency and high throughput
capabilities and removes the limitations of
SCSI by providing cable distances of up to
10 km with fiber optic links; 2 Gbps transfer
rate, redundant paths to storage to improve
reliability. In theory, can connect up to
16 million devices; in loop topologies, up to
127 storage devices or servers can share
the same Fibre Channel connection allowing
implementation of large clusters.
The number of disk drives significantly affects performance because each drive
contributes to total system throughput. Capacity requirements are often the only
consideration that is used to determine the number of disk drives that are
configured in a server. Throughput requirements are usually not well understood
or are completely ignored. The key to a good performing disk subsystem
depends on maximizing the number of read-write heads that can service I/O
requests. With RAID technology, you can spread the I/O over multiple spindles.
Accessing time updates
The Linux file system keeps records of when files are created, updated, and
accessed. Default operations include updating the last-time-read attribute for
files during reads and writes to files.
Because writing is an expensive operation, eliminating unnecessary I/O can lead
to overall improved performance. Mounting file systems with the noatime option
eliminates the inode access times from being updated. If file update times are not
critical to your implementation, as in a Web-serving environment, a user can
choose to mount file systems with the noatime flag in the /etc/fstab file. Update
the /etc/fstab file with noatime option set:
/dev/sdb1 /mountlocation ext3 defaults,noatime 1 2
397
It is generally a good idea to have a separate /var partition and mount it with the
noatime option. This is applicable to both SUSE and Red Hat Linux.
Selecting the journaling mode of an ext3 file system
You can set three different journaling options in the ext3 file system with the data
option in the mount command:
data=journal
This journaling option provides the highest form of data consistency by
causing both file data and metadata to be journaled. It also has the highest
performance cost of the three options.
data=ordered (default)
In this mode only metadata is written. However, file data is guaranteed to be
written first. This is the default setting.
data=writeback
This journaling option provides the fastest access to the data at the expense
of data consistency. The data is guaranteed to be consistent as the metadata
is still being logged. However, no special handling of actual file data is done
and this might lead to old data appearing in files after a system crash.
There are two ways to change the journaling mode on a file system:
When issuing the mount command:
mount -o data=writeback /dev/sdb1 /mnt/mountpoint
Including it in the options section of the /etc/fstab file:
/dev/sdb1 /testfs ext3 defaults,journal=writeback 0 0
If you want to modify the default data=ordered option on the root partition, make
the change to the file listed previously, then issue the mkinitrd command to scan
the changes in the /etc/fstab file and create a new image. Update /etc/lilo.conf to
point to the new image.
For more information about ext3, refer to the following Web site:
http://www.redhat.com/support/wpapers/redhat/ext3/
398
ReiserFS: The default SUSE Linux file system
The default file system on a SUSE installation since SUSE Linux 7.1 has been
ResierFS, developed by Hans Reiser. From its initial design, key performance
aspects have included the following features:
Journaling that is designed into the file system from the beginning improves
reliability and recovery
Provides faster access through the use of balanced tree data structures that
allow the storing of both content data and security metadata
Efficient use of disk space because, unlike other file systems, this file system
does not rely on block sizes
Tuning ReiserFS
One of the strengths of the ReiserFS is its support for a large number of small
files. Instead of using the traditional block structure of other Linux file systems,
ReiserFS uses a tree structure that has the capability to store the actual contents
of small files or the tails of those that are larger in the access tree itself. This file
system does not use fixed block sizes, therefore only the space that is required
to store a file is consumed, leading to less wasted space.
There is an option when mounting a ReiserFS file system that improves
performance but at the expense of space. When mounting a ReiserFS, you can
disable this tail packing option by specifying notail so that the file system
performs a little faster but uses more disk space.
An example of mounting a ReiserFS file system with the notail option is:
/dev/sdb1 /testfs resierfs notail 0 0
11.2.8 Tuning in Transmission Control Protocol and
User Datagram Protocol
Use the following commands for tuning servers that support a large number of
multiple connections:
For servers that receive many connections at the same time, re-use the
TIME-WAIT sockets for new connections. This is useful in Web servers, for
example:
sysctl -w net.ipv4.tcp_tw_reuse=1
If you enable this command, you must also enable fast recycling of
TIME-WAIT sockets status:
sysctl -w net.ipv4.tcp_tw_recycle=1
399
With these parameters enabled, the number of connections is significantly
reduced. This is good for performance because each TCP transaction
maintains a cache of protocol information about each of the remote clients. In
this cache, information such as round-trip time, maximum segment size, and
congestion window are stored. For more details, review RFC 1644.
The parameter tcp_fin_timeout is the time to hold a socket in state
FIN-WAIT-2 when the socket is closed at the server.
A TCP connection begins with a three-segment synchronization SYN
sequence and ends with a three-segment FIN sequence, both of which do not
hold data. If you change the tcp_fin_timeout value, the time from the FIN
sequence to when the memory can be freed for new connections can be
reduced, thereby improving performance. However, change this value only
after careful monitoring, because there is a risk of overflowing memory due to
the number of dead sockets.
sysctl -w net.ipv4.tcp_fin_timeout=30
One of the problems found in servers with many simultaneous TCP
connections is the large number of connections that are open but unused.
TCP has a keepalive function that probes these connections and, by default,
drops them after 7200 seconds (2 hours). This length of time might be too
large for your server and can result in excess memory usage and a decrease
in server performance. Setting it to 1800 seconds (30 minutes), for example,
might be more appropriate:
sysctl -w net.ipv4.tcp_keepalive_time=1800
Set the maximum OS send buffer size (wmem) and receive buffer size
(rmem) to 8 MB for queues on all protocols:
– sysctl -w net.core.wmem_max=8388608
– sysctl -w net.core.rmem_max=8388608
These specify the amount of memory that is allocated for each TCP socket
when it is created. In addition, you must also use the following commands for
send and receive buffers. They specify three values: minimum size, initial
size, and maximum size:
– sysctl -w net.ipv4.tcp_rmem="4096 87380 8388608"
– sysctl -w net.ipv4.tcp_wmem="4096 87380 8388608"
The third value must be the same as or less than the value of wmem_max
and rmem_max.
400
11.2.9 Performance tuning tools
This section describes some of the performance tuning tools.
The uptime command
Use the uptime command to verify how long the server has been running and
how many users are logged on. You can also use it for a quick overview of the
average load of the server. The system load average is displayed for the last
1-minute, 5-minute, and 15-minute intervals. The load average is not a
percentage but the number of processes in queue waiting to be processed. If
processes that request CPU time are blocked (which means the CPU has no
time to process them), the load average increases. Alternatively, if each process
gets immediate access to CPU time and there are no CPU cycles lost, the load
decreases.
The optimal value of the load is 1, which means that each process has
immediate access to the CPU and there are no CPU cycles lost. The typical load
can vary from system to system: for a uniprocessor workstation, 1 or 2 might be
acceptable, but you might probably see values of 8 to 10 on multiprocessor
servers.
A sample output of the uptime command:
1:57am up 4 days 17:05, 2 users, load average: 0.00, 0.00, 0.00
The dmesg command
The main purpose of dmesg is to display kernel messages. It can provide helpful
information in case of hardware problems or problems with loading a module into
the kernel. In addition, with dmesg, you can determine what hardware is installed
in your server. During every boot, Linux checks your hardware and logs
information about it. You can view these logs using the /bin/dmesg command.
Example 11-4 shows a partial output from the dmesg command.
Example 11-4 Partial output from the dmesg command
EXT3 FS 2.4-0.9.19, 19 August 2002 on sd(8,1), internal journal
EXT3-fs: mounted filesystem with ordered data mode.
IA-32 Microcode Update Driver: v1.11 <[email protected]>
ip_tables: (C) 2000-2002 Netfilter core team
3c59x: Donald Becker and others. www.scyld.com/network/vortex.html
See Documentation/networking/vortex.txt
01:02.0: 3Com PCI 3c980C Python-T at 0x2080. Vers LK1.1.18-ac
00:01:02:75:99:60, IRQ 15
product code 4550 rev 00.14 date 07-23-00
Internal config register is 3800000, transceivers 0xa.
401
8K byte-wide RAM 5:3 Rx:Tx split, autoselect/Autonegotiate interface.
MII transceiver found at address 24, status 782d.
Enabling bus-master transmits and whole-frame receives.
01:02.0: scatter/gather enabled. h/w checksums enabled
divert: allocating divert_blk for eth0
ip_tables: (C) 2000-2002 Netfilter core team
Intel(R) PRO/100 Network Driver - version 2.3.30-k1
Copyright (c) 2003 Intel Corporation
divert: allocating divert_blk for eth1
e100: selftest OK.
e100: eth1: Intel(R) PRO/100 Network Connection
Hardware receive checksums enabled
cpu cycle saver enabled
ide-floppy driver 0.99.newide
hda: attached ide-cdrom driver.
hda: ATAPI 48X CD-ROM drive, 120kB Cache, (U)DMA
Uniform CD-ROM driver Revision: 3.12
Attached scsi generic sg4 at scsi1, channel 0, id 8, lun 0, type 3
The top command
The top command shows you actual processor activity. By default, it displays the
most CPU-intensive tasks running on the server and updates the list every
5 seconds. You can sort the processes by PID (numerically), age (newest first),
time (cumulative time), and resident memory usage and time (time the process
has occupied the CPU since startup).
Example 11-5 shows the output from the top command.
Example 11-5 Sample output from the top command
top - 02:06:59 up 4 days, 17:14, 2 users, load average: 0.00, 0.00,
0.00
Tasks: 62 total, 1 running, 61 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.2% us, 0.3% sy, 0.0% ni, 97.8% id, 1.7% wa, 0.0% hi, 0.0% si
Mem: 515144k total, 317624k used, 197520k free, 66068k buffers
Swap: 1048120k total, 12k used, 1048108k free, 179632k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
13737 root 17 0 1760 896 1540 R 0.7 0.2 0:00.05 top
238 root 5 -10 0 0 0 S 0.3 0.0 0:01.56 reiserfs/0
1 root 16 0 588 240 444 S 0.0 0.0 0:05.70 init
2 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
3 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0
4 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/1
5 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/1
6 root 5 -10 0 0 0 S 0.0 0.0 0:00.02 events/0
402
7 root 5 -10 0 0 0 S 0.0 0.0 0:00.00 events/1
8 root 5 -10 0 0 0 S 0.0 0.0 0:00.09 kblockd/0
9 root 5 -10 0 0 0 S 0.0 0.0 0:00.01 kblockd/1
10 root 15 0 0 0 0 S 0.0 0.0 0:00.00 kirqd
13 root 5 -10 0 0 0 S 0.0 0.0 0:00.02 khelper/0
14 root 16 0 0 0 0 S 0.0 0.0 0:00.45 pdflush
16 root 15 0 0 0 0 S 0.0 0.0 0:00.61 kswapd0
17 root 13 -10 0 0 0 S 0.0 0.0 0:00.00 aio/0
18 root 13 -10 0 0 0 S 0.0 0.0 0:00.00 aio/1
You can further modify the processes using renice to give a new priority to each
process. If a process stops or occupies too much CPU, you can end the process
(kill command). The columns in the output are as follows:
PID: Process identification
USER: Name of the user who owns (and perhaps started) the process
PRI: Priority of the process
NI: Niceness level (that is, whether the process tries to be nice by adjusting
the priority by the number given; see the following section for details)
SIZE: Amount of memory (code + data + stack), in KB, that is used by the
process
RSS: Amount of physical RAM used, in KB
SHARE: Amount of memory shared with other processes, in KB
STAT: State of the process: S=sleeping, R=running, T=traced, D=interruptible
sleep
Z=zombie
%CPU: Share of the CPU usage (since the last screen update)
%MEM: Share of physical memory
TIME: Total CPU time used by the process (since it started)
COMMAND: Command line used to start the task (including parameters)
Process priority and nice levels
Process priority is a number that determines the order in which the process is
handled by the CPU. The kernel adjusts this number to higher and lower as
necessary. The nice value is a limit on the priority. The priority number is not
allowed to go below the nice value (a lower nice value is a more favored priority).
It is not possible to change the priority of a process. This is only indirectly
possible through the use of the nice level of the process. Note that it may not
always be possible to change the priority of a process using the nice level. If a
403
process is running too slowly, you can assign more CPU to it by giving it a lower
nice level. This means that all other programs have fewer processor cycles and
run more slowly.
Linux supports nice levels from 19 (lowest priority) to -20 (highest priority). The
default value is 0. To change the nice level of a program to a negative number
(which makes it a high-priority process), it is necessary to log on or su to root.
To start the program xyz with a nice level of -5, issue the command:
nice -n -5 xyz
To change the nice level of a program already running, issue the command:
renice level pid
If you to change the priority of the xyz program that has a PID of 2500 to a nice
level of 10, issue the following command:
renice 10 2500
The iostat command
The iostat command is part of the Sysstat set of utilities, available at the
following Web site:
http://perso.wanadoo.fr/sebastien.godard/
The iostat command lets you see average CPU times since the system was
started, in a way that is similar to uptime. In addition, iostat creates a report
about the activities of the disk subsystem of the server. The report has two parts:
CPU utilization and device (disk) utilization.
Example 11-6 shows a sample output of the iostat command.
Example 11-6 Sample output from the iostat command
Linux 2.4.21-9.0.3.EL (x232) 05/11/2004
avg-cpu: %user %nice %sys %idle
0.03 0.00 0.02 99.95
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
dev2-0 0.00 0.00 0.04 203 2880
dev8-0 0.45 2.18 2.21 166464 168268
dev8-1 0.00 0.00 0.00 16 0
dev8-2 0.00 0.00 0.00 8 0
dev8-3 0.00 0.00 0.00 344 0
404
The CPU utilization report has four sections:
%user: Shows the percentage of CPU utilization that is taken up while
running at the user level (applications)
%nice: Shows the percentage of CPU utilization that is taken up while running
at the user level with a nice priority (For details about priority and nice levels,
see “Process priority and nice levels” on page 403.)
%sys: Shows the percentage of CPU utilization that is taken up while running
at the system level (kernel)
%idle: Shows the percentage of time the CPU is idle
The device utilization report is split into the following sections:
Device: This the name of the block device.
tps: The number of transfers per second (I/O requests per second) to the
device. Multiple single I/O requests can be combined in a transfer request,
because a transfer request can have different sizes.
Blk_read/s, Blk_wrtn/s: Blocks read and written per second indicate data that
is read and written from and to the device in seconds. Blocks might also have
different sizes. Typical sizes are 1024 bytes, 2048 bytes, or 4048 bytes,
depending on the partition size. For example, you can find the block size of
/dev/sda1:
dumpe2fs -h /dev/sda1 |grep -F "Block size"
This gives an output similar to:
dumpe2fs 1.34 (25-Jul-2003)
Block size: 1024
Blk_read, Blk_wrtn: This indicates the total number of blocks that are read
and written since the boot.
The vmstat command
The vmstat command provides information about processes, memory, paging,
block I/O, traps and CPU activity. Example 11-7 shows the output from the
vmstat command.
Example 11-7 Sample output from the vmstat command
procs -----------memory------- ---swap-- ---io--- ---system-- ----cpu-----r b
swpd free
buff cache
si so
bi bo
in cs
us sy id wa
2 0
0
154804 77328 910900 0 0
4 6
103 19
0 0 100 0
405
The columns in the output are as follows:
Process
– r: The number of processes waiting for run time
– b: The number of processes in uninterruptable sleep
Memory
–
–
–
–
swpd: The amount of virtual memory used (KB)
free: The amount of idle memory (KB)
buff: The amount of memory used as buffers (KB)
cache: The amount of memory used as cache (KB)
Swap
– si: Amount of memory swapped from the disk (KBps)
– so: Amount of memory swapped to the disk (KBps)
IO
– bi: Blocks sent to a block device (blocks per second)
– bo: Blocks received from a block device (blocks per second)
System
– in: The number of interrupts per second, including the clock
– cs: The number of context switches per second
CPU (percentages of total CPU time)
–
–
–
–
us: Time spent running non-kernel code (user time, including nice time)
sy: Time spent running kernel code (system time)
id: Time spent idle. Before Linux 2.5.41, this included I/O wait time
wa: Time spent waiting for I/O. Before Linux 2.5.41, this appeared as zero
The free command
The /bin/free command displays information about the total amounts of free
and used memory (including swap) on the system. It also includes information
about the buffers and cache used by the kernel. Example 11-8 shows the output
from the free command.
Example 11-8 Sample output from the free command
total used free shared buffers cached
Mem: 1291980 998940 293040 0 89356 772016
-/+ buffers/cache: 137568 1154412
Swap: 2040244 0 2040244
406
The pmap command
The pmap command reports the amount of memory that one or more processes
are using. You can use this tool to determine which processes on the server are
being allocated memory and whether this amount of memory is a cause of
memory bottlenecks:
pmap -x <pid>
Tip: Using pipes, it is possible to produce the output in one command. For
example, to generate a report in Hypertext Markup Language (HTML), run:
cat output_traffic-collector | traffic-sort -Hp | traffic-tohtml -o
output_traffic-tohtml.html
To generate a report as a GIF file, run:
cat output_traffic-collector | traffic-sort -Hp | traffic-togif -o
output_traffic-togif.gif -x 600 -y 600
Example 11-9 shows the total amount of memory that the cupsd process is
using.
Example 11-9 Total amount of memory used by the cupsd process
linux:~ # pmap -x 1796
1796: /usr/sbin/cupsd
Address Kbytes RSS Anon Locked Mode Mapping
08048000 244 - - - r-x-- cupsd
ffffe000 4 - - - ----- [ anon ]
-------- ------- ------- ------- ------total kB 6364 - - For the complete syntax of the pmap command, issue:
pmap -?
The strace command
The strace command intercepts and records the system calls that are called by
a process, and the signals that are received by a process. This is a useful
diagnostic, instructional, and debugging tool. System administrators might find it
valuable for solving problems with programs. To use the command, specify the
process ID to be monitored:
strace -p <pid>
407
Example 11-10 shows the output of the strace monitoring httpd process.
Example 11-10 Output from the strace monitoring httpd process
[root@x232 html]# strace -p 815
Process 815 attached - interrupt to quit
semop(360449, 0xb73146b8, 1) = 0
poll([{fd=4, events=POLLIN}, {fd=3, events=POLLIN, revents=POLLIN}], 2, -1) = 1
accept(3, {sa_family=AF_INET, sin_port=htons(52534),
sin_addr=inet_addr("9.42.171.197")},
[16]) = 13
semop(360449, 0xb73146be, 1) = 0
getsockname(13, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr
("9.42.171.198")}, [16]) = 0
fcntl64(13, F_GETFL) = 0x2 (flags O_RDWR)
fcntl64(13, F_SETFL, O_RDWR|O_NONBLOCK) = 0
read(13, 0x8259bc8, 8000) = -1 EAGAIN (Resource temporarily unavailable)
poll([{fd=13, events=POLLIN, revents=POLLIN}], 1, 300000) = 1
read(13, "GET /index.html HTTP/1.0\r\nUser-A"..., 8000) = 91
gettimeofday({1084564126, 750439}, NULL) = 0
stat64("/var/www/html/index.html", {st_mode=S_IFREG|0644, st_size=152, ...}) = 0
open("/var/www/html/index.html", O_RDONLY) = 14
mmap2(NULL, 152, PROT_READ, MAP_SHARED, 14, 0) = 0xb7052000
writev(13, [{"HTTP/1.1 200 OK\r\nDate: Fri, 14 M"..., 264}, {"<html>\n<title>\n
RedPaper
Per"..., 152}], 2) = 416
munmap(0xb7052000, 152) = 0
socket(PF_UNIX, SOCK_STREAM, 0) = 15
connect(15, {sa_family=AF_UNIX, path="/var/run/.nscd_socket"}, 110) = -1 ENOENT (No
such
file or directory)
close(15) = 0
11.2.10 Tuning for applications
Many applications specify prerequisite settings for various CPU, memory, and
I/O parameters to ensure the best performance. If you have followed these
instructions, or an application does not provide instructions on these settings and
you consider that your application performance is less than anticipated, check
some of the tools and commands that are explained in this chapter.
For issues relating to memory and file parameters, and also Web servers with
large numbers of network connections, look at the sysctl command. Database
applications rely heavily on efficient I/O read and write activities. If this is an area
of concern, see 11.2.6, “Tuning the file system” on page 395.
408
12
Chapter 12.
Storage area network
installation and booting
Installing the operating system (OS) to a storage area network (SAN) drive or
booting from a SAN drive in combination with a BladeCenter JS21 has some
specialities. This chapter discusses these topics.
409
12.1 Storage area network setup
We assume that the user has basic knowledge about SAN setup and
configuration. Several IBM Redbooks are available for more general information
about SAN, for example, IBM TotalStorage: SAN Product, Design, and
Optimization Guide, SG24-6384.
A minimal two-path SAN installation is shown in Figure 12-1. We used the IBM
two-port Fibre Channel (FC) switch module in this book.
BladeCenter Chassis
BladeCente JS21
FC expansion
card (HBA)
FC Switch Module
Bay 1
FC Switch Module
Bay 2
Controller 1
Controller 2
Storage Unit
Hard drives
Path 1
Path 2
Figure 12-1 Basic SAN layout with a two-path configuration for a BladeCenter JS21
Important: The JS21 supports only the Qlogic 4Gb Fibre Channel Small Form
Factor expansion card. We recommend that you use at least firmware Version
4.00.22, which includes IBM driver Version 1.14. Additionally, we recommend
that you update the firmware of all SAN components to the latest available or
possible firmware level.
410
To benefit from a two-path configuration, the most important setting is a multipath
or failover configuration. The functionality of such a configuration depends on the
hardware, firmware, the drivers, and the operating system. Because this is a very
complex task, it is beyond the scope if this book to describe a complete solution.
12.1.1 Assigning a disk
Follow the description to assign a disk from a storage unit to a BladeCenter JS21
QLogic 4Gb Fibre Channel Small Form Factor expansion card (host bus adapter
(HBA)):
1. Install the FC expansion card in a BladeCenter JS21 and connect the devices
as shown in Figure 12-1 on page 410.
2. Update the firmware. At the time of the creation of this book, it is only possible
to use AIX command diag or the stand-alone diagnostics CD (see 6.6.9,
“Updating the firmware using the stand-alone diagnostics CD” on page 111)
to update the firmware.
3. Configure a SAN name server. This function is included in any SAN switch. If
more than one switch is used in a fabric1, the entries in each name server are
auto-synchronized between all switches in this fabric. Hence you can use
each switch to configure the name server.
a. To do out-of-band management for Fibre Channel switch modules, use
the BladeCenter management module for a basic Ethernet setup, as
described in 6.4, “I/O module configuration” on page 85.
Important: Make sure that the external ports are enabled, because this
affects the Ethernet and Fibre Channel ports.
b. To configure McDATA or IBM or QLogic Fibre Channel switch modules,
download and install the BladeCenter SAN Utility or SANsurfer Switch
Management Software, which is available for different architectures and
operating systems.
1
Physical connected SAN
Chapter 12. Storage area network installation and booting
411
The management software shows a screen similar to Figure 12-2.
Figure 12-2 SAN switch module configuration panel using the BladeCenter SAN utility
i. To download the software, go to:
http://support.qlogic.com
ii. Select Drivers, Software and Manuals.
iii. In the OEM MODELS section, click IBM.
iv. In the IBM eServer BladeCenter Fibre Channel Switches section,
choose IBM / QLogic code or McData code.
Note: To configure Brocade Fibre Channel switch modules, use the
Web or command-line interface (CLI).
412
c. After you connect to one Fibre Channel switch module using the
management software, you can use the name server fabric overview of a
SAN switch to see all the connected SAN devices in fabric, see
Figure 12-3. This example does not correspond with the fabric layout
shown in Figure 12-1 on page 410.
Figure 12-3 Name server and fabric overview
Note: In fabrics with multiple switches, each SAN switch requires an
explicit identifier, called domain ID.
The BladeCenter JS21 HBA and especially the worldwide name (WWN)
does not show up in the name server of the SAN Switch without activation.
To activate the card, scan for all boot devices using the SMS menu (see
6.8, “System Management Services interface” on page 133). After this
procedure, the HBA is visible in the name server list. The scan activates
the card, but without any bootable partitions, it is not possible to assign the
Fibre Channel expansion card as a boot device. The next step is the
configuration of a zone.
413
Note: To activate the BladeCenter JS21 HBA, it is also possible to start
an operating system from a local disk and load the HBA driver in the
modules named qla2400 and qla2xxx. If the HBA is installed, this
occurs automatically.
Important: For a SAN high-availability solution, it is necessary to have
two fabrics with two different name servers. Relating to the example in
Figure 12-1 on page 410, this means that the configuration of path 1
and path 2 are done completely separated. For the configuration of
path 1, use the first switch, and configure path 2 with the second switch.
It is not allowed to have identical WWNs in one of the two name
servers. If this happens, the configuration is wrong.
d. The zone configuration is also done on a SAN switch of the fabric. One
zone is like a logical network inside the SAN. All components in one zone
are visible to each other. Typically there is a switch for global visibility for
all devices and ports which are not in a zone.
To create a functional zone for the first path of the example setup shown in
Figure 12-1 on page 410, use the Zone configuration panel to create:
i. ZoneSet, for example, zone_1
ii. Zone, for example, zone_1
iii. An alias2 for HBA Port 1 (for example, JS21_Bay3_Port1) and Storage
Unit Controller 1 (for example, Sage1_Controller 1)
e. Assign the alias to the zone.
2
414
An alias is a selectable name and might be used instead of WWNs
Figure 12-4 shows the result.
Figure 12-4 Zoning for path 1
f. Now activate the zone, which is usually done in the zone menu.
Afterwards the zone configuration must also emerge in the name server.
After activation, the zone configuration is distributed to all SAN switches in
a fabric. Even if the HBA is deactivated and not listed in the name server
anymore, the zone setting itself is persistent and works after a reactivation
of the HBA as required.
For a two-path configuration, perform the same steps on the other switch.
Attention: Zone activation is especially critical with running systems. A
misconfigured zone might lead to a locked up server.
415
4. The configuration of the storage unit is only drafted in the following section,
because the configuration can be very complex and depends on the storage
system:
a. Create a physical array of hard disk. In general, this must be a Redundant
Array of Independent Disks 10 (RAID 10) configuration for a fast and save
configuration or a RAID 5 for large amounts of data in combination with
redundancy.
b. Map one or more arrays and host ports to a host group or host.
i. To add a host port to a host or host group, the WWN is required. In
most cases, the WWN is determined automatically by the storage unit.
But this process does not always work with a BladeCenter JS21,
therefore, you must type in the WWN manually. You can find the WWN
of a BladeCenter JS21 HBA port in the name server. To enable a
two-path configuration, it is necessary to add both BladeCenter JS21
HBA ports.
ii. To add an array to a host group or host, specify a specific logical unit
number (LUN) and an operating system type.
After this step, at least one host port and one array is assigned to a single
host group or host. The SAN configuration is now complete.
Important for storage unit OS configuration: If more than one type of
operating system is used with a storage unit, make sure that the
storage unit supports multiple operating systems at the same time.
Restriction: SAN boot with a BladeCenter JS21 might be an issue if an
array is configured as LUN0. To be safe, we recommend that you use
LUN1 or later to avoid problems.
After the SAN configuration is complete, you can install the operating system.
The necessary single-path driver for the 4 GB FC expansion card is provided by
AIX, Virtual I/O Server (VIOS), SUSE Linux Enterprise Server (SLES), and Red
Hat Enterprise Linux by default. Almost all configurations presented in this book
were also tested in a single-path SAN environment.
416
12.1.2 SAN boot characteristics
The following list provides some of the characteristics of a SAN boot:
The boot device name hard drive 1 and hard drive 2 define the internal hard
drive of a BladeCenter JS21. The boot device name hard disk 3 points to
port 1 of the HBA and hard drive 4 points to port 2 of the HBA.
Even with a two-path configuration, there is no redundancy during the boot
process. Therefore, the corresponding multipath driver is loaded during the
boot phase of the operating system. If more than one path is configured, but
the special multipath driver is not running, in most cases, the same disk is
discovered as a second disk. It is not advisable to use more than one path if
the multipath driver is not used.
AIX always uses an explicit hard disk identifier. Linux uses as default device
names such as sda or sdb. These device names are not bound to a specific
hard disk. To enable an explicit identification, use Universal Unique Identifiers
(UUIDs) for Linux. Specify this during the partitioning of a hard drive.
For all operating systems, the boot process can be slow. It might happen that
the system is running fine, even if no output through Serial over LAN (SoL) is
provided for a long period.
During the HBA discovery period before the operating system starts, the
output shown in Example 12-1 can be seen using SoL with a two-path and
two-LUN configuration. The path and LUNs are highlighted in bold.
Example 12-1 SMS device scan with a two-path and two-LUN configuration
scan /pci@8000000f8000000/pci@1/fibre-channel@2/diskQLogic QMC2462SHost Adapter
Driver(IBM): 1.17 03/31/06
Firmware version 4.00.22
check
/pci@8000000f8000000/pci@1/fibre-channel@2/disk@200400a0b80ba0ed,0001000000000000
QLogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06
check
/pci@8000000f8000000/pci@1/fibre-channel@2/disk@200400a0b80ba0ed,0002000000000000
417
scan /pci@8000000f8000000/pci@1/fibre-channel@2,1/diskQLogic QMC2462SHost Adapter
Driver(IBM): 1.17 03/31/06
check
/pci@8000000f8000000/pci@1/fibre-channel@2,1/disk@200400a0b80ba0ed,0001000000000000QL
ogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06
check
/pci@8000000f8000000/pci@1/fibre-channel@2,1/disk@200400a0b80ba0ed,0002000000000000QL
ogic QMC2462SHost Adapter Driver(IBM): 1.17 03/31/06
Keep these specialties in mind and start the installation process using network as
usual. Double-check if the host type, which means the operating system, is
specified correctly for the LUN that is configured in the storage unit. The
installation procedure for AIX and SLES are similar. After the installation, the
boot sequence must automatically switch to hard disk 2. For Red Hat, it is
important to install the system only with one activated path. After the installation,
select hard disk 2 as boot device and reboot the system. After this step, it is not a
problem to activate the second path.
418
A
Appendix A.
SUSE Linux Enterprise
Server 9 AutoYaST XML file
The following settings are used in the Extensible Markup Language (XML) file
shown in Example A-1:
Install minimum amount of software packages
Enable the firewall, but allow Secure Shell (SSH) login after installation
Create a root user and define a password (encrypted)
Configure two Ethernet devices eth0 and eth1 using Dynamic Host
Configuration Protocol (DHCP)
Enable host name and Domain Name System (DNS) server assignment
using DHCP
PowerPC Bootloader (PREP) on partition sda1
Add the max_loop=255 parameter to the boot configuration
Use the <use>all</use> switch for the partitioning of sda. It is not a problem to
use another hard drive, for example, sdc, but all other partitioning
configurations will most likely result in a failed installation.
The created partition table is shown in Example A-2 on page 425.
419
Example: A-1 Basic tested AutoYaST XML file
<?xml version="1.0"?>
<!DOCTYPE profile SYSTEM "/usr/share/autoinstall/dtd/profile.dtd">
<profile xmlns="http://www.suse.com/1.0/yast2ns"
xmlns:config="http://www.suse.com/1.0/configns">
<configure>
<firewall>
<fw_allow_fw_traceroute>no</fw_allow_fw_traceroute>
<fw_autoprotect_services>no</fw_autoprotect_services>
<fw_dev_ext>eth1</fw_dev_ext>
<fw_dev_int>eth0</fw_dev_int>
<fw_ipsec_trust>no</fw_ipsec_trust>
<fw_log_accept_all>no</fw_log_accept_all>
<fw_log_accept_crit>yes</fw_log_accept_crit>
<fw_log_drop_all>no</fw_log_drop_all>
<fw_log_drop_crit>yes</fw_log_drop_crit>
<fw_masq_nets></fw_masq_nets>
<fw_masquerade>no</fw_masquerade>
<fw_protect_from_internal>no</fw_protect_from_internal>
<fw_route>no</fw_route>
<fw_services_ext_tcp>ssh</fw_services_ext_tcp>
<fw_services_ext_udp></fw_services_ext_udp>
<fw_services_int_ip></fw_services_int_ip>
<start_firewall config:type="boolean">true</start_firewall>
</firewall>
<networking>
<dhcp_options>
<dhclient_additional_options></dhclient_additional_options>
<dhclient_client_id></dhclient_client_id>
<dhclient_hostname_option>AUTO</dhclient_hostname_option>
</dhcp_options>
<dns>
<dhcp_hostname config:type="boolean">true</dhcp_hostname>
<dhcp_resolv config:type="boolean">true</dhcp_resolv>
<domain>site</domain>
<hostname>linux</hostname>
<nameservers config:type="list">
<nameserver>192.168.1.254</nameserver>
</nameservers>
<searchlist config:type="list">
<search>js21testnet</search>
</searchlist>
</dns>
<interfaces config:type="list">
420
<interface>
<bootproto>dhcp</bootproto>
<device>eth0</device>
<startmode>onboot</startmode>
</interface>
<interface>
<bootproto>dhcp</bootproto>
<device>eth1</device>
<startmode>onboot</startmode>
</interface>
</interfaces>
<modules config:type="list">
<module_entry>
<device>static-0</device>
<module>bcm5700</module>
<options></options>
</module_entry>
<module_entry>
<device>static-1</device>
<module>bcm5700</module>
<options></options>
</module_entry>
</modules>
<routing>
<ip_forward config:type="boolean">false</ip_forward>
</routing>
</networking>
<security>
<console_shutdown>reboot</console_shutdown>
<cracklib_dict_path>/usr/lib/cracklib_dict</cracklib_dict_path>
<cwd_in_root_path>yes</cwd_in_root_path>
<cwd_in_user_path>yes</cwd_in_user_path>
<displaymanager_remote_access>no</displaymanager_remote_access>
<enable_sysrq>no</enable_sysrq>
<fail_delay>3</fail_delay>
<faillog_enab>yes</faillog_enab>
<gid_max>60000</gid_max>
<gid_min>1000</gid_min>
<kdm_shutdown>all</kdm_shutdown>
<lastlog_enab>yes</lastlog_enab>
<obscure_checks_enab>yes</obscure_checks_enab>
<pass_max_days>99999</pass_max_days>
<pass_max_len>16</pass_max_len>
<pass_min_days>0</pass_min_days>
<pass_min_len>7</pass_min_len>
Appendix A. SUSE Linux Enterprise Server 9 AutoYaST XML file
421
<pass_warn_age>7</pass_warn_age>
<passwd_encryption>des</passwd_encryption>
<passwd_use_cracklib>yes</passwd_use_cracklib>
<permission_security>secure</permission_security>
<run_updatedb_as>nobody</run_updatedb_as>
<system_gid_max>499</system_gid_max>
<system_gid_min>100</system_gid_min>
<system_uid_max>499</system_uid_max>
<system_uid_min>100</system_uid_min>
<uid_max>60000</uid_max>
<uid_min>500</uid_min>
<useradd_cmd>/usr/sbin/useradd.local</useradd_cmd>
<userdel_postcmd>/usr/sbin/userdel-post.local</userdel_postcmd>
<userdel_precmd>/usr/sbin/userdel-pre.local</userdel_precmd>
</security>
<users config:type="list">
<user>
<encrypted config:type="boolean">true</encrypted>
<user_password>0jZJQH0JKZajo</user_password>
<username>root</username>
</user>
</users>
</configure>
<install>
<bootloader>
<activate config:type="boolean">true</activate>
<board_type>chrp</board_type>
<default>linux</default>
<global config:type="list">
<global_entry>
<key>timeout</key>
<value config:type="integer">100</value>
</global_entry>
<global_entry>
<key>boot</key>
<value>/dev/sda1</value>
</global_entry>
<global_entry>
<key>activate</key>
<value config:type="boolean">true</value>
</global_entry>
<global_entry>
<key>default</key>
<value>linux</value>
</global_entry>
422
</global>
<initrd_modules config:type="list">
<initrd_module>
<module>ipr</module>
</initrd_module>
<initrd_module>
<module>qla2400</module>
</initrd_module>
<initrd_module>
<module>ipr</module>
</initrd_module>
</initrd_modules>
<loader_device>/dev/sda1</loader_device>
<loader_type>ppc</loader_type>
<location>boot</location>
<of_defaultdevice config:type="boolean">true</of_defaultdevice>
<prep_boot_partition>/dev/sda1</prep_boot_partition>
<sections config:type="list">
<section config:type="list">
<section_entry>
<key>image</key>
<value>/boot/vmlinux</value>
</section_entry>
<section_entry>
<key>label</key>
<value>linux</value>
</section_entry>
<section_entry>
<key>root</key>
<value>/dev/sda3</value>
</section_entry>
<section_entry>
<key>initrd</key>
<value>/boot/initrd</value>
</section_entry>
<section_entry>
<key>append</key>
<value>selinux=0 elevator=cfq max_loop=255</value>
</section_entry>
</section>
</sections>
</bootloader>
<general>
<clock>
<hwclock>UTC</hwclock>
423
<timezone>US/Pacific</timezone>
</clock>
<keyboard>
<keymap>english-us</keymap>
</keyboard>
<language>en_US</language>
<mode>
<confirm config:type="boolean">false</confirm>
</mode>
<mouse>
<id>none</id>
</mouse>
</general>
<drive>
<use>all</use>
</drive>
</partitioning>
<report>
<errors>
<log config:type="boolean">true</log>
<show config:type="boolean">true</show>
<timeout config:type="integer">0</timeout>
</errors>
<messages>
</messages>
<warnings>
</warnings>
<yesno_messages>
</yesno_messages>
</report>
<software>
<addons config:type="list">
<addon>Base-System</addon>
<addon>YaST2</addon>
424
</addons>
<base>Minimal</base>
<packages config:type="list">
<package>dhcp</package>
<package>dhcp-server</package>
<package>expect</package>
<package>inetd</package>
<package>iprutils</package>
<package>libidl-64bit</package>
<package>ppc64-utils</package>
<package>rdist</package>
<package>rsh-server</package>
<package>telnet-server</package>
<package>librtas</package>
</packages>
</software>
</install>
</profile>
Example A-2 shows the created partition table.
Example: A-2 Default partition table created by the AutoYaST configuration file
Device Boot
Start
End
Blocks
/dev/sda1
*
1
1
172+
Partition 1 does not end on cylinder boundary.
/dev/sda2
4
134
1052257+
/dev/sda3
135
17843
142247542+
Id
41
System
PPC PReP Boot
82
83
Linux swap
Linux
425
426
B
Appendix B.
Red Hat Enterprise Linux 4
Kickstart file
The following settings are used in the Kickstart configuration file shown in
Example B-1:
Install basic amount of software packages
Enable the firewall, but allow Secure Shell (SSH) login after installation
Create a root user and define a password (encrypted)
Configure two Ethernet devices eth0 and eth1 using Dynamic Host
Configuration Protocol (DHCP)
Use the autopart switch for the partitioning. All other configurations will most
likely result in a failed installation.
The created partition table is shown in Example B-2. The created logical
volume group configuration is shown in Example B-3.
Example: B-1 Basic tested Kickstart configuration file
#Generated by Kickstart Configurator
#platform=IBM pSeries
#System language
lang en_US
#Language modules to install
427
langsupport en_US
#System keyboard
keyboard us
#System mouse
mouse
#Sytem timezone
timezone America/New_York
#Root password
rootpw --iscrypted $1$/7IU0tSr$QmQtfRtKWulXPC8cmk2kf.
#Reboot after installation
reboot
#Use text mode install
text
#Install OS instead of upgrade
install
#Use Web installation
url --url ftp://192.168.1.254/rhel4
#System bootloader configuration
bootloader --location=mbr
#Clear the Master Boot Record
zerombr yes
#Partition clearing information
clearpart --all --initlabel
#System authorization infomation
auth --useshadow --enablemd5
#Disk partitioning information
autopart
#Network information
#Firewall configuration
firewall --enabled --trust=eth1 --ssh
#Do not configure XWindows
skipx
#Package install information
%packages --resolvedeps
@ server-cfg
@ ftp-server
@ development-tools
@ compat-arch-development
@ admin-tools
@ system-tools
@ compat-arch-support
428
Example B-2 shows the created partition table.
Example: B-2 Default partition table created by the Kickstart configuration file
Device Boot
/dev/sda1
*
/dev/sda2
/dev/sda3
Start
1
2
15
End
1
14
17844
Blocks
8001
104422+
143219475
Id
41
83
8e
System
PPC PReP Boot
Linux
Linux LVM
Example B-3 shows the created logical volume group configuration.
Example: B-3 Default logical volumes created by the Kickstart configuration file
VG Name
LV UUID
LV Write Access
LV Status
# open
LV Size
Current LE
Segments
Allocation
Read ahead sectors
Block device
/dev/VolGroup00/LogVol00
VolGroup00
1Vzc3F-6ubR-bCrX-Iv4i-hAtZ-lXMl-AHMLwe
read/write
available
1
134.59 GB
4307
1
inherit
0
253:0
/dev/VolGroup00/LogVol01
VG Name
VolGroup00
LV UUID
AkDpUP-q6Wb-MdKl-xDLx-QYXn-SKRd-fhlF7J
LV Write Access
read/write
LV Status
available
# open
1
LV Size
1.94 GB
Current LE
62
Segments
1
Allocation
inherit
Read ahead sectors
0
Block device
253:1
Appendix B. Red Hat Enterprise Linux 4 Kickstart file
429
430
C
Appendix C.
SUSE Linux Installation
Server setup
Use Yet Another Setup Tool (YaST) to configure the SUSE Linux Installation
Server as described in “Preparing SLES as an installation server using YaST” on
page 294. However, if you have a system where you cannot use YaST to perform
this function, then you can use the following instructions.
The following steps describe the manual approach which the YaST installation
server applet does automatically. This might be helpful for troubleshooting and
for other operating systems or distributions that do not provide this applet.
We assume that a File Transfer Protocol (FTP) service is already configured.
See 9.2.4, “Configuring a File Transfer Protocol service” on page 288. The next
step is to build an installation directory.
1. Create directories to mount the basic SLES ISOs and service pack ISOs:
mkdir -p /mnt/loop/sles9_sp3/SUSE-SLES-Version-9/CD1
And
for X in ‘seq 1 5‘; do mkdir -p
/mnt/loop/sles9_sp3/SUSE-CORE-Version-9/CD$X; done
431
And
for X in ‘seq 1 3‘; do mkdir -p
/mnt/loop/sles9_sp3/SUSE-SLES-9-Service-Pack-Version-3/CD$X; done
Note on SLES10: Because SUSE Linux Enterprise Server 10 (SLES10) is
also available on a DVD it is enough to create a single directory, for
example, mkdir -p /mnt/loop/sles10/DVD1.
2. Mount the ISO files using a loop device. The following commands must be
written without a newline/return but issued with return at the end:
mount -t iso9660 -o loop,ro
/srv/data/iso-images/SLES-9-ppc-RC5-CD1.iso
/mnt/loop/sles9_sp3/SUSE-SLES-Version-9/CD1
And
for X in ‘seq 2 6‘; do mount -t iso9660 -o loop,ro
/srv/data/iso-images/SLES-9-ppc-RC5-CD$X.iso
/mnt/loop/sles9_sp3/SUSE-CORE-Version-9/CD$((X-1)); done
And
for X in ‘seq 1 3‘; do mount -t iso9660 -o loop,ro
/srv/data/iso-images/SLES-9-ppc-SP3-CD$X.iso
/mnt/loop/sles9_sp3/SUSE-SLES-9-Service-Pack-Version-3/CD$X; done
432
Tip: If enough loop devices are not available, there are two ways to add
loop devices, depending on your kernel configuration.
You can add the max_loop=255 parameter in the /etc/lilo file to enable
255 loop-devices:
# header section
timeout = 100
default = linux
# image section
image = /pci@8000000f8000000/ide@4,1/disk@0:3,/boot/vmlinux
label = linux
root = /dev/hda3
append = "selinux=0 elevator=cfq max_loop=255"
initrd = /pci@8000000f8000000/ide@4,1/disk@0:3,/boot/initrd
Issue the lilo command and restart the system to enable the added
parameter. The shown settings might be differ on other systems or
distributions.
If the loop-device driver is a kernel module, as in Red Hat Enterprise
Linux (RHEL), it is necessary to add the following line in
/etc/modeprobe.conf:
options loop max_loop=255
Note on SLES10 - Mount the DVD:
mount -t iso9660 -o loop,ro
/srv/data/iso-images/SLES-10-ppc-RC1-DVD1.iso
/mnt/loop/sles10/DVD1
3. Copy the now accessible files, create one directory and some soft links by
issuing the following commands. The copy process takes some time. The
final directory structure is shown in Example C-1:
rsync -auv /mnt/loop/sles9_sp3 /srv/ftp
And
mkdir /srv/ftp/sles9_sp3/yast
And
ln -s SUSE-SLES-Version-9/CD1/boot /srv/ftp/sles9_sp3/boot
And
ln -s SUSE-SLES-Version-9/CD1/content /srv/ftp/sles9_sp3/content
Appendix C. SUSE Linux Installation Server setup
433
And
ln -s SUSE-SLES-Version-9/CD1/control.xml
/srv/ftp/sles9_sp3/control.xml
And
ln -s SUSE-SLES-9-Service-Pack-Version-3/CD1/driverupdate
/srv/ftp/sles9_sp3/driverupdate
And
ln -s SUSE-SLES-9-Service-Pack-Version-3/CD1/linux
/srv/ftp/sles9_sp3/linux
And
ln -s SUSE-SLES-Version-9/CD1/media.1 /srv/ftp/sles9_sp3/media.1
Restriction: It is not possible to work directly with the loop-mounted ISO
files. You have to copy the files to the hard drive to create a functional
installation source.
Tip: On SLES10 it is sufficient to copy the content of the DVD using:
cp -dbr /mnt/loop/sles10/ /srv/ftp/sles10/
Example: C-1 First level of the installation source directory
/srv/ftp/sles9_sp3/
|-- SUSE-CORE-Version-9
|-- SUSE-SLES-9-Service-Pack-Version-3
|-- SUSE-SLES-Version-9
|-- boot -> SUSE-SLES-Version-9/CD1/boot
|-- content -> SUSE-SLES-Version-9/CD1/content
|-- control.xml -> SUSE-SLES-Version-9/CD1/control.xml
|-- driverupdate -> SUSE-SLES-9-Service-Pack-Version-3/CD1/driverupdate
|-- linux -> SUSE-SLES-9-Service-Pack-Version-3/CD1/linux
|-- media.1 -> SUSE-SLES-Version-9/CD1/media.1
`-- yast
434
4. Create two files instorder and order in /srv/ftp/sles9_sp3/yast with the content
shown in Example C-2 and Example C-3.
Example: C-2 The instorder file
/SUSE-SLES-9-Service-Pack-Version-3/CD1
/SUSE-SLES-Version-9/CD1
/SUSE-CORE-Version-9/CD1
Important: The path names in Example C-3 are TAB separated.
Example: C-3 The order file
/SUSE-SLES-Version-9/CD1 /SUSE-SLES-Version-9/CD1
/SUSE-CORE-Version-9/CD1 /SUSE-CORE-Version-9/CD1
All basic preparations to install SLES using the network are now finished.
Appendix C. SUSE Linux Installation Server setup
435
436
Abbreviations and acronyms
ac
alternating current
GUI
graphical user interface
AIX
Advanced Interactive
eXecutive
HTTP
Hypertext Transfer Protocol
HTTPS
BIU
bus interface unit
Hypertext Transfer
Protocol-Secure
BOOTP
Bootstrap Protocol
I/O
input/output
BOS
base operating system
IBM
BPU
branch processing unit
International Business
Machines Corporation
BSMP
blade system management
processor
IGESM
CISCO Intelligent Gigabit
CISC
Complex Instruction Set
Computer
IP
Internet Protocol
IPL
initial program load (boot)
CLI
command-line interface
ITSO
CRLF
carriage return-line feed
International Technical
Support Organization
CSM
Cluster Systems
Management
KVM
keyboard, video, and mouse
LAN
local area network
CTR
count register
LPAR
logical partition
dc
direct current
MAC
Media Access Control
DDR
double data rate
Mb
megabit
DHCP
Dynamic Host Configuration
Protocol
MB
megabyte
MFLOPS
DNS
Domain Name Server
Million Floating Point
Operations Per Second
ECC
error-checking and correction
MP
multiprocessor
ERAT
effective to real address
translation
NFS
Network File System
NIC
Network Interface Controller
ESM
Ethernet switch module
NIM
Network Installation Manager
FPR
floating-point register
PKT
packet
FPU
floating point unit
POSIX
FTP
File Transfer Protocol
Portable Operating System
for UNIX
FXU
fixed point unit
POST
Power On Self Test
Gb
gigabit
POWER
GB
gigabyte
Performance Optimization
With Enhanced RISC
gcc
GNU C Compiler
RAM
random access memory
GPR
general purpose register
RHEL
Red Hat Enterprise Linux
437
RISC
reduced instruction set
computer
ROM
read-only memory
RTC
Real-Time Clock
SAN
storage area network
SCSI
Small Computer System
Interface
SMP
symmetric multiprocessor
SoL
Serial over LAN
SPOT
Shared Product Object Tree
SSH
Secure Shell
TCP/IP
Transmission Control
Protocol/Internet Protocol
TFTP
Trivial File Transfer Protocol
TLB
Translation Lookaside Buffer
USB
Universal Serial Bus
VLAN
virtual local area network
VPN
virtual private network
VXU
vector processing unit
WAN
wide area network
XER
Fixed Point Exception
Register
XML
Extensible Markup Language
YaST
Yet Another Setup Tool
(SUSE)
438
Related publications
The publications listed in this section are considered particularly suitable for a
more detailed discussion of the topics covered in this redbook.
IBM Redbooks
For information on ordering these publications, see “How to get IBM Redbooks”
on page 444. Note that some of the documents referenced here may be available
in softcopy only.
NIM: From A to Z in AIX 4.3, SG24-5524
Implementing IBM Director 5.10, SG24-6188
The IBM eServer BladeCenter JS20, SG24-6342
IBM TotalStorage: SAN Product, Design, and Optimization Guide,
SG24-6384
AIX 5L Practical Performance Tools and Tuning Guide, SG24-6478
Advanced POWER Virtualization on IBM System p5, SG24-7940
The Cutting Edge: IBM eServer BladeCenter, REDP-3581
IBM eServer BladeCenter Systems Management, REDP-3582
Nortel Networks L2/3 Ethernet Switch Module for IBM eServer BladeCenter,
REDP-3586
IBM eServer BladeCenter Layer 2-7 Network Switching, REDP-3755
Cisco Systems Intelligent Gigabit Ethernet Switch Module for the
IBM eServer BladeCenter, REDP-3869
Virtual I/O Server Integrated Virtualization Manager, REDP-4061
439
Other publications
These publications are also relevant as further information sources:
IBM BladeCenter Solution Assurance Planning Review Guide, Document
SA749
IBM CSM for AIX 5L and Linux: Administration Guide, SA23-1343
CSM 1.5.0 for AIX 5L and Linux: Planning and Installation Guide, SA23-1344
CSM 1.5.0 for AIX 5L and Linux: Command and Technical Reference,
SA23-1345
AIX 5L Version 5.3 Performance Management, SC23-4905
Online resources
These Web sites and URLs are also relevant as further information sources:
Power Module Upgrade Guidelines (Technical Update) - IBM BladeCenter
(Type 8677)
http://www.ibm.com/pc/support/site.wss/document.do?sitestyle=ibm&lnd
ocid=MIGR-53353
ServerProven Web site
http://www.ibm.com/servers/eserver/serverproven/compat/us/eserver.ht
ml
IBM Middleware on Linux
http://www.ibm.com/software/os/linux/software
Information Center VIOS commands
http://publib.boulder.ibm.com/infocenter/eserver/v1r3s/index.jsp?top
ic=/iphb1/iphb1_vios_commandslist.htm
IBM Virtualization
http://www.ibm.com/servers/eserver/about/virtualization/
Advanced POWER Virtualization
http://www.ibm.com/servers/eserver/pseries/ondemand/ve/resources.htm
l
Virtual I/O Server
http://techsupport.services.ibm.com/server/vios/home.html
440
IBM personal computing support site
http://www.ibm.com/pc/support/site.wss
IBM Cluster information center
http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?top
ic=/com.ibm.cluster.csm.doc/clusterbooks.html
Dropped sessions with Serial over LAN (SOL) and BOOTP on the same port IBM eServer BladeCenter JS20
2
Download IBM Director 5.1
http://www.ibm.com/servers/eserver/xseries/systems_management/ibm_di
rector/
Download site for AutoUpdate V4.3.4 or later levels
Download sg3_utils-1.06-1.ppc64*
http://people.redhat.com/pknirsch
BladeCenter internal network diagram
BladeCenter networking
http://www.research.ibm.com/journal/rd/496/hunter.pdf
BladeCenter processor blades, I/O expansion adapters, and units
http://www.research.ibm.com/journal/rd/496/hughes.pdf
Software and device drivers - IBM BladeCenter
7
Support for BladeCenter products
http://www-03.ibm.com/servers/eserver/support/bladecenter/allproduct
s/installing.html
Expect scripting (to automate/repeat firmware updates)
http://expect.nist.gov
IBM Linux On POWER diagnostic software
441
Open Firmware
http://www.firmworks.com
Download site for OpenSSL Red Hat Package Manager (RPM)
http://www-1.ibm.com/servers/aix/products/aixos/linux/download.html
Red Hat Enterprise Linux installation tip
Automatic Linux Installation and Configuration with YaST2
http://www.suse.com/~ug/autoyast_doc/
Red Hat Enterprise Linux installation tip
Red Hat Enterprise Linux 4: System Administration Guide
http://www.redhat.com/docs/manuals/enterprise/RHEL-4-Manual/sysadmin
-guide/
IBM Director installation instructions
Cluster Systems Manager 1.5.1.1 for AIX Package Information
https://www14.software.ibm.com/webapp/set2/sas/f/csm/download/csmaix
_1.5.1.1down.html
Cluster Systems Manager Document Library
http://publib.boulder.ibm.com/infocenter/clresctr/vxrx/index.jsp?top
ic=/com.ibm.cluster.csm.doc/clusterbooks.html
Cluster Systems Manager Web site
http://www14.software.ibm.com/webapp/set2/sas/f/csm/home.html
Site for autoupdate RPM
IBM pSeries and AIX Information Center
http://publib.boulder.ibm.com/infocenter/pseries
AIX Information Center
http://publib.boulder.ibm.com/infocenter/pseries/v5r3/index.jsp
QLogic software download page
http://support.qlogic.com
442
Unrolling AltiVec, Part 1: Introducing the PowerPC SIMD unit
http://www.ibm.com/developerworks/power/library/pa-unrollav1/
High-Performance Processors: Altivec Technology
http://www.freescale.com/files/32bit/doc/fact_sheet/ALTIVECFACT.pdf
AltiVec - Wikipedia
http://en.wikipedia.org/wiki/AltiVec
PowerPC Microprocessor Family: AltiVec Technology Programming
Environments Manual
http://www-306.ibm.com/chips/techlib/techlib.nsf/techdocs/FBFA164F82
4370F987256D6A006F424D
IBM Personal computing support
http://www-307.ibm.com/pc/support/site.wss/document.do?lndocid=MIGR63788
IBM Director 5.10 Update 2
https://www14.software.ibm.com/webapp/iwm/web/preLogin.do?source=dmp
IBM Director 5.10 Update 2: Sign in
https://www14.software.ibm.com/webapp/iwm/web/reg/download.do?source
=dmp&S_PKG=diragent&cp=UTF-8
IBM Support: Fix Central
https://techsupport.services.ibm.com/server/aix.fdc
Whitepaper: Red Hat's New Journaling File System: ext3
http://www.redhat.com/support/wpapers/redhat/ext3/
SYSSTAT utilities
http://perso.wanadoo.fr/sebastien.godard/
Serial over LAN (SOL) Setup Guide - IBM BladeCenter, T
http://www-307.ibm.com/pc/support/site.wss/document.do?sitestyle=ibm
&lndocid=MIGR-54666&velxr-layout=print
443
How to get IBM Redbooks
You can search for, view, or download IBM Redbooks, Redpapers, Hints and
Tips, draft publications and Additional materials, as well as order hardcopy
Redbooks or CD-ROMs, at this Web site:
ibm.com/redbooks
Help from IBM
IBM Support and downloads
ibm.com/support
IBM Global Services
ibm.com/services
444
Index
Symbols
/etc/dhcpd.conf 299
/etc/init.d/dhcpd restart command 286
/etc/init.d/dhcpd start command 286
/etc/modeprobe.conf 433
/etc/vsftpd.con 289
/etc/vsftpd.conf file 289
/etc/xinet.d/vsftpd 288
Numerics
2-port FC switch module
firmware update 108
8677 19
8844-31X 11
8844-51X 11
8852 21
A
activating NIM master 268
active console, selecting 134
advanced management module 71, 77, 98
command-line interface 72
connectors 73
Ethernet port 74
firmware update 104
management network 77
Advanced Micro Devices (AMD) 5
advanced POWER Virtualization 5, 36, 53
Web site 38
AIX
performance tuning 386
AIX 5L 26
environment 244
installation CDs/DVDs 260
levels supported by JS21 26
mirroring with LVM 250
network installation 261
new virtual disks 239
RAID 182
RAID configuration 170
technology levels 53
virtual SCSI disk LUN 251
AltiVec 15
optimized applications 5
Web sites 17
AMD 3DNow! 15
application subnet 51
ARP function 49
array processing 14
AutoYaST 311, 419
B
baseboard management controller (BMC) 70
bkprofdata command 255
blade management 356
blade server 2
assigning media tray 94
assigning names 90
eth0 port 123
I/O expansion card 169
setting the boot sequence 91
Blade System Management Processor (BSMP) 97
BladeCenter 1, 5
advantages
high availability 3
lower cost 2
SAN optimization 3
server consolidation 2
switch technology 3
benefits 2
chassis 19, 64
device drivers 95
documentation 46
high-performance, low-latency interconnection
network 51
integrated switches 2
internal management network 69
internal network 66
management interface 32
management module 3, 29–30, 411
managing multiple chassis 33, 53
media tray 24
memory 17
multi-chassis interconnection 47
445
physical layout 64
planning resources 46
power modules 20, 23
power requirements 4
product documentation updates 46
schematic 66
using IBM Director Console 355
Web interface 32, 346
BladeCenter chassis T 23
BladeCenter H 5, 21
BladeCenter JS20 133
BladeCenter JS21
chassis 19
dedicated processor 39
device alias 157
directed BOOTP 160
hardware setup 63
IBM Director Agent support 58
IBM Director support 58
internal hard disk 18
introduction 4
L2 cache 9
large configuration 53
management module 24
memory features 17
micro-partitioning 29
minimal network requirement 47
minimum and optional features 11
models 11
network boot configuration 135
network installation 54–55
network installation environment 54
network interface selection 56
operating system support 26
operating systems 26
physical packaging 10
planning elements 45
power/thermal management architecture 23
processor 13
RAID 18
SLES supported versions 27
SMS interface 133
specification 8
specifications 9
storage 3
technology 1
viewing Open Firmware settings 155
446
VIOS installation 193
BladeCenter T
physical layout 65
boot command 161
boot device
CD/DVD 145
order 144
boot image 290
boot parameter 299
definition 298
location 325
mkzimage_cmdline 306
using SSH 308
using VNC 308
with AutoYaST 320
boot process
explicit location 299, 325
logical partition 231
boot sequence
BOOTP 300
current setting 91
default 135
device alias 157
multipath I/O 418
RHEL installation 321, 326, 335
setting 90–91
updated by SLES install 290
Bootstrap Protocol (BOOTP) 284
coexistence with DHCP 55
definition 55
directed versus broadcast 136
byte sector, RAID format 173
C
C2 security 26
capacity planning 33
CD/DVD drive 120
cfgmgr command 239
chsyscfg command 237, 240
chvg -g command 248
Cluster Systems Management (CSM) 32, 345, 362
AIX nodes 377
automating software installations 54
cluster node installation 373
hardware control attributes 374
hardware management subnet 49
installation server 363, 373
introduction 33
Linux nodes 379
network considerations 61
node attributes 373
planning 60
supported operating systems 59
verifying installation 383
command
/usr/lib/methods/define_rspc 166
/usr/sbin/installios 194
AIX
cfgmgr 167, 234
mkcd 256
mksysb 256
odmget 166
boot 130–131, 161
console 130
CSM
copycsmpkgs 381
definenode 375
dsh 383
lsnode 376
updatenode 377, 381
diag 99
diagmenu 252
env 130–131
env -T 130
info 96
Linux
bcmflashdiag 102
chkconfig 288–289
iprconfig 183
mkzimage_cmdline 301, 306, 320
nvsetenv 344
lscfg 251
lshwres 38
lsmap 250
mkzimage_cmdline 299, 322
netstat 290
Open Firmware
devalias 156
printenv 154
power 130, 133, 152, 183
reset 130
smitty nim_power5 194
sol 131
telnetcfg 132
update 106
update_flash 101
update_flash (AIX 5L) 100
VIOS
backupios 255
cfgdev 233
chdate 210
chsyscfg 231
chsysstate 231
lsdev 233
mkgencfg 209
mktcpip 210
mkvt 231–232, 257
rmvt 231
command-line interface (CLI) 31
Common Internet File System (CIFS) 284
configuration
AIX environment 261
NIM master 263
NIM using command line 201
NIM using SMIT 196
VIOS partition 221
configuration file
AutoYaST 311
disable parameter 288
Kickstart 341, 343
TFTP 325
xinet.d 288
csmconfig command 368–369
D
date and time setting 210
default user ID and password, management module
133
DHCP
AutoYaST configuration 317
coexistence with BOOTP 55
configuration 285
network planning 46
restarting 286
service 54, 325
TFTP file name 298
disk array 188–189
selecting disk units 191
disk space
adding to virtual disk 245
logical partition 217
logical volume 222
virtual disk 247
disk technology 396
diskette drive 24
Index
447
dmesg 401
double data rate 2 (DDR2) 4
DVD drive, VIOS installation media 194
Dynamic Host Configuration Protocol (DHCP) 284
dynamic logical partitioning (DLPAR) 29, 43
E
e-mail alert 84
Enhanced Integrated Drive Electronics (EIDE) 396
error-checking and correction (ECC) 4
Ethernet
expansion card 68, 103, 162, 169
firmware update process 99
port enumeration 162
changing in AIX 162
switch module 68, 89, 108, 122, 169
F
Fibre Channel (FC) 410–411, 416
field-replaceable unit (FRU) 24
field-replaceable units (FRU) 59P6629 24
file system
access time updates 397
ext3 journaling mode 398
tuning 395
File Transfer Protocol (FTP)
configuring service 288, 324
installation server 284, 296
installation service 293
firmware
committing update 101–102
copying permanent to temporary 101
identifying levels 96
rejecting update 101–102
update_flash (Linux command) 101
updating redundant MM 83
updating with AIX 99
floating-point registers (FPRs) 16
free command 406
fully qualified domain name (FQDN) 203
G
General Parallel File System (GPFS) 52
general purpose register (GPR) 16
geninstall command 368
GNU C Compiler (gcc) 27
graphical user interface (GUI) 31, 388
448
gunzip command 370
H
hard disk drives (HDD) 9
hardware
alerts 33
vital product data (VPD) 81
hardware management 27, 29, 41, 49, 61
subnet 47
Hardware Management Console (HMC) 27, 29–30,
41, 194
high availability 3
high-performance computing (HPC) 4
ideal blade server solution 4
vector processing 14
host bus adapter (HBA) 411, 413–416
discovery period 417
HS20 60
Hypertext Markup Language (HTML) 284
Hypervisor 29
I
I/O module 57
accessing management interface 68
advanced configuration 85, 88
basic setup 85
differences in bay location 68
Ethernet expansion card 51
Ethernet ports 68
expansion card options 169
external ports 89
firmware update 106
IP address 77
management 32
by management module 77
interface 47, 49, 69, 85–87
IP address 85
port 68
port enabling 88
setting IP address 85
SoL 89, 241
updating firmware 106
IBM diagnostic tools and utilities 96
IBM Director 57, 346
Agent 352
components 58
Core Services 352
disabling SLPD on SLES 352
installation on AIX 349
installing components 347
introduction 33
inventory 57
network
installation of OS 53
requirements 58
performance planning 33
planning 58
primary functions 57
remote management 57
resource monitoring 57
TCP/IP port 84
IBM General Parallel File System (GPFS) 52
IBM middleware 34
ibmvmc0 31
IEEE 802.1Q 37
input/output (I/O) 2
installation
AIX 5L
CD/DVD 260
general topics 260
AIX on client 270
CSM management server 363
IBM Director agent on Linux 352
IBM Director components 347
IBM Director on AIX 349
Integrated Virtualization Manager (IVM) 193
Linux from CD/DVD 282
Linux on LPAR 282
operating system on LPAR 232
RPM package 203
SUSE Linux Enterprise Server (SLES) 282
VIOS image from Linux server 202
VIOS in a JS21 194
Virtual I/O Server (VIOS) 194
installation source 294
configuring with YaST 294, 298
destination directory 296
directory 293, 297, 324
for RHEL 337
tree 296
undefined 301
integrated development environment (IDE) 3
Integrated Drive Electronics (IDE) 396
integrated switch technology 3
integrated systems management 3
Integrated systems management processor (ISMP)
70
architecture 30
command logs 257
definition 27
description 41
design 29
devices tab 233
dynamic logical partitioning 43
functions 29
installation 193
limitations 42
LPAR console 37–38
maintenance 253
navigation area 212
Partition Management menu 223
requirements 42
restrictions 28
starting 212
starting graphical interface 211
Storage Management menu 233
VIOS maintenance 253
VIOS management 211
work area
Devices tab 218, 222
Logical Volume tab 222, 238
Optical Devices tab 233
Start Wizard button 223
storage pool 218
Intel Multimedia Extensions (MMX) 15
Intel Streaming SIMD Extensions (SSE) 15
INTx port 68
iostat command 404
iSmall Computer System Interface (iSCSI) 13
J
JS21, see BladeCenter JS21
K
kernel parameter 390
kernel.shmm axe 392
keyboard, video, and mouse (KVM) 56, 94, 121
ports 71
Kickstart configuration file 427
Index
449
L
license -accept command 208
Lightweight Directory Access Protocol (LDAP) 26,
78
lilo command 433
Linux
basic preparation 283
IBM diagnostic tools and utilities 96
nodes 379
supported kernel 386
tuning options on POWER 386
logical partition (LPAR) 28, 245, 253–256
adding disks 239
assigning
memory 224
optical device 234
benefits 35
configuration change 235, 240
creation 218, 223
disk space 252
dynamic 43
Ethernet adapters in SMS 149
Integrated Virtualization Manager (IVM) 30, 211
Linux installation 282
logical volume 217, 222
maintenance 256
management 29
moving optical device 234
next reboot 240
non-dynamic operations 211
opening virtual terminal 231
operating system installation 232
optical device 229, 234
physical adapters 28
power on 231
processing mode 225
processing unit 240
rootvg 217
secure environment 32
security layer 36
SMS mode 232
storage assignment 228
virtual Ethernet 226
adapaters 240
virtual LAN channel 36
virtual SCSI adapater 42
virtual terminal 231–232, 257
virtualization feature 38
450
logical unit number (LUN) 216, 250, 416
Logical volume (LV)
creation 222
logical volume (LV)
advanced storage management 244
assigning 239
commands 215
data mirroring 250
default storage pool 217
RAID support 41
viewing 223
VIOS concepts 216
virtual disk extension 245
lpcfgop command 209
lsdev command 209
lsdev -virtual command 31
lshwres command 38
lspv command 275
lsrefcode command 231
M
man dhcpd command 285
management module 24
10/100BaseT Ethernet interface 49
CLI timeout 132
command-line interface 72, 123, 128, 130
configuration 71
connection drawing 73
controlling SoL using commands 130
default gateway 75
default IP address 74–75
default user ID and password 133
earlier firmware 124
external interface 49, 68, 74, 78
firmware 105
firmware update 104
hardware VPD 81
initial configuration 75
installation guide 71
interface 49
internal network interface 49
IP address 49, 74
(no DHCP) 75
login session limits 124
management interface 71, 73
network requirements 47
redundancy 82
resetting 84
restarting 78
setting IP configuration 75
setup task 78
SoL 50, 89
SSH 128
system management interface 97
TCP/IP port usage 83
user 75
Web interface 75, 126
management node 60
Management Processor Assistant (MPA) 58
maximum transmission unit (MTU) 37, 77
Media Access Control (MAC) 73–75, 104, 154
determining address 81
media tray 24, 94, 194, 233, 260
diskette drive 283
Medium Access Control (MAC) 69
memory dual inline memory modules (DIMMs) 11
message
Could not find the SUSE Linux Enterprise Server
9 Installation CD 301
SoL session is already active 130
This machine’s CPU ID does not match the CPU
ID stored in the NIM database 279
Virtual terminal is already connected 231
message passing interface (MPI) 52
metadata, see file system 395
Micro-Partitioning 36, 39–40
middleware 34
midplane 69
mirrorios command 250
mkgencfg command 209
mkinitrd command 398
mkvt command 216
mount command 398
Myrinet 51–52
expansion card 51
N
name server
SAN 411, 415
worldwide name 416
navigation area
Partition Management menu 230
Service Management menu 253, 257
Storage Management menu 218–219,
222–223, 238, 249
network
boot 97, 133, 135, 149, 151, 161, 293, 300
logical layout 48
planning 46
minimal requirements 47
Network Equipment Building Standards (NEBS) 23
Network File System (NFS) 54, 284
export and server daemon 205
network information services (NIS) 387
network installation
basic preparation 283
configuration file 285
Linux 281, 284
NIM 270
Open Firmware 322
options 53
planning 54
preparation 55
Red Hat Enterprise Linux (RHEL) 322
setting up 55
infrastructure 281
network boot 135
SUSE Linux Enterprise Server 291
Trivial File Transfer Protocol (TFTP) 287
Network Installation Manager (NIM) 40, 194, 281
master 202, 260–261, 263, 266–268, 270
state 269
network planning 46
network services 54
NIM client 266
allocating resources 268
changing from rsh to nimsh 276
Nortel Networks Layer 2-7 Gigabit Ethernet switch
module 89, 125
O
ODM 162–166
Open Firmware
activating interface 133, 152
aliases 93
boot parameters 343
boot sequence 92–93
defining an alias 156
device tree 161
directed BOOTP 136, 160
Index
451
IP parameters 160
ls command 161
setenv 321
operating system (OS)
boot phase 417
supported by JS21 26
optical device sharing 233
optical pass-thru I/O module 51
option blades 2
P
parallel network installation 33
PCI-X SCSI disk array 173, 177, 182
Performance Optimization with Enhanced RISC
(POWER) 4
performance tuning tool 401
Peripheral Component Interconnect (PCI) 3
physical disk 188, 216, 244
physical volume 216–218, 245, 250
commands 215
PKT file 104, 106
Pluggable Authentication Modules (PAM) 26
pmap command 407
POWER Hypervisor 36–37
power management 358
power modules 23
Power On Self Test (POST) 133, 135, 153
power requirements for BladeCenter 4
power/thermal management architecture 23
PowerPC
970MP 13
AltiVec extensions 16
processor
features 13
operating frequency 19
proxy ARP 49, 69–70
PUTTY 124
R
RAID 18
configuration tool 170
preparing the disks 170
Red Hat
disabling daemons 387
Red Hat Enterprise Linux
attended installation 330
452
AutoYaST with Open Firmware 321
boot sequence 335
configuration tool 336
CSM support 59
general information 26
importing configuration 313
installation 283
iSCSI support 397
Kickstart 342
maintenance contract 26
supported level 26
sysctl 390
Red Hat Enterprise Linux (RHEL)
Red Hat Package Manager (RPM) 59
Redundant Array of Independent Disks (RAID) 9
ReiserFS 399
remote console 359
remote shell (rsh) 276
Request For Comment (RFC) 397
resource monitor 360
resource monitoring 57
RETAIN tip H181655 56
RPM
AutoUpdate 59
bcmflashdiag-js20 102
conserver 60
fping-2.4b2-5 60
IBMJava2-JRE 60
openssl 277
sg3_utils-1.06-1.ppc64 59
tftp-HPA 60
rstprofdata command 255
Run-Time Abstraction Services (RTAS) 101
S
Samba Web Administration Tool (SWAT) 388
Secure Shell (SSH) 78, 359
during installation 308
Secure Sockets Layer (SSL) 78
selecting active console 134
Serial Attached SCSI (SAS) 3–4, 18, 396
Serial over LAN (SoL) 32
attaching to open console 130
boot sequence 92
configuring management module 125
description 121
Directed BOOTP 136
enabling 127
Ethernet I/O Switch 56
firmware level requirements 124
I/O module requirement 122
installation without SoL 307
interacting with VNC 311
networking 51
persistent target 131
prerequisites 124
range of IP addresses 124
Red Hat Enterprise Linux (RHEL) installation
335
setting up CISCO switch 90
starting session 123, 128
subnet 47, 50
terminate session 131
User Defined Keystroke Sequences 125
VLAN 89, 124
Serial Storage Architecture (SSA) 41
Server Message Block (SMB) 284
ServerProven 12
shared Ethernet adapter (SEA) 41, 242–244
Simple Mail Transfer Protocol (SMTP) 78
simultaneous multithreading (SMT) 394
single-active core 13
single-instruction multiple-data (SIMD) 13
small computer system interface (SCSI) 3
small-form-factor ((SFF) 12
smit csm command 369
smitty manfs command 261
software
distribution 57
remote management 33
Software Distribution Premium Edition 33
sshpassword 307
stand-alone diagnostics CD
updating firmware 99
storage area network (SAN) 3, 216, 409
boot specialities 417
optimization 3
setup 410
storage pool 215–220, 222, 227, 249–250, 252
advanced configuration 218
creation 218–219
subnet
hardware management 47, 49
logical layout 48
SUSE Linux Enterprise Server (SLES)
attended network installation 299
AutoYaST 311, 320
compatibility 313
clearing boot-file variable 344
configuring RAID 183
creating installation server with service pack
298
CSM support 59
disabling SLPD 352
importing Kickstart file 313
installation 282
after RAID creation 191
kernel modules 302
installation server 294
introduction 27
network installation 290–291
no hard drives message 188
preparing installation source 293
rescue mode 170
SMB as installation service 284
support for JS21 27
supported level 26
supported version 53
unattended installation 311, 320
with CSM 60
switch module 68, 410, 412
IP address 106
symmetric multiprocessor (SMP) 349
server 35
synchronous dynamic random access memory
(SDRAM)) 9
sysctl command 390
system backup image 256
System Management Interface Tool (SMIT) 196
System Management Service (SMS) 133
activating interface 133
options 152
System Management Services (SMS) 133
system management tools 32
systemid command 371
T
Telnet 132
telnetcfg command 132
top command 402
Index
453
Trivial File Transfer Protocol (TFTP) 54, 56
configuration 287, 299
Red Hat bootable image 325
testing service 290
tuning
file system 395
memory 394
ReiserFS 399
Transmission Control Protocol (TCP) 399
User Datagram Protocol (UDP) 399
twgstart command 354
U
Universal Manageability Initiative 32
Universal Serial Bus (USB) 24
uplink command 82
uptime command 401
USB CD-ROM 145
usessh 307
V
varyoff command 246
varyonvg command 248
vector length 13
vector processing 14
vector processor (VXU) 15
vector register file (VRF) 16
Vector/SIMD Multimedia eXtension 13
virtual disk 216, 227, 254
LPAR operation 238
virtual Ethernet
adapter 146, 226, 240
boot device 149
bridging 242
with IVM 241
design 37
dynamic operations 237
features 37
management 29
network traffic 41
VIOS setup 209
virtual LAN (VLAN) 42
virtual I/O adapters 37
Virtual I/O Server (VIOS) 5, 27, 39
backup and restore 255
command-line interface 214
454
creating logical partitions 223
data protection 244
default storage pool 219
default user 212
define NIM client 202
definition 39
device sharing 233
disk mirroring 249
installation 193–194
from NIM server 194
on RAID array 252
with NIM 202
without DVD drive 194
Integrated Virtualization Manager (IVM) 28, 42,
211
maintenance 253
micro-partitioning 39–40
moving memory 237
network configuration 242
opening a virtual terminal 231
partition configuration 221
RAID support 41
required version 38
rootvg mirroring 250
storage management 216
storage pool 250
rootvg 217
supported operating systems 38
VIOS partition 221
virtual disks 250
virtual Ethernet 37
virtual processor operation 237
virtual SCSI 37
virtualization 35–36, 209
virtual LAN (VLAN) 46
4095 for SoL 50
planning 46, 70
SoL 89
virtual Ethernet 37
virtualization 36
Virtual Management Channel (VMC) 29–30
Virtual Network Computing (VNC) 130, 307
virtual processor 39
virtual SCSI
adapter 42
client adapter 41
introduction 37, 41
server adapter 41
virtual terminal 231
virtualization 36
Virtualization Engine 27, 41
vital product data (VPD) 24
vmstat command 405
vncpassword 307
vsftpd 288
W
worldwide name (WWN) 413
X
xinetd 289
Y
Yet Another Setup Tool (YaST) 294
Index
455
456
IBM BladeCenter JS21: The
POWER of Blade Innovation
(1.0” spine)
0.875”<->1.498”
460 <-> 788 pages
Back cover
®
IBM BladeCenter JS21
The POWER of Blade Innovation
High-performance
blade server ideal
for extremely dense
HPC clusters
First IBM blade
server with native
virtualization for
server consolidation
Exceptional
price-performance
per watt for
WebSphere with AIX
or Linux OS
Blade servers have captured the industry focus because of
their efficient, powerful technology and their modular design,
which reduces cost with a more efficient use of valuable floor
space. They offer simplified management, which can help to
speed up tasks such as installing, provisioning, updating, and
troubleshooting hundreds of blade servers. You can do all of
this remotely using one graphical console with IBM Director
systems management tools or cluster management software
such as Cluster Systems Management (CSM).
This IBM Redbook takes an in-depth look at the IBM
BladeCenter JS21. This is a 2-core or 4-core blade server for
applications requiring 64-bit computing. It is ideal for
computer-intensive applications and transactional Internet
servers. This book helps you to install, tailor, and configure the
IBM BladeCenter JS21 running either IBM AIX 5L or the Linux
operating system (OS).
This document expands the current set of IBM BladeCenter
JS21 documentation by providing a desktop reference that
offers a detailed technical description of the JS21 system. This
publication does not replace the latest IBM BladeCenter JS21
marketing materials and tools. It is intended as an additional
source of information that you can use, together with existing
sources, to enhance your knowledge of IBM blade solutions.
INTERNATIONAL
TECHNICAL
SUPPORT
ORGANIZATION
BUILDING TECHNICAL
INFORMATION BASED ON
PRACTICAL EXPERIENCE
IBM Redbooks are developed by
the IBM International Technical
Support Organization. Experts
from IBM, Customers and
Partners from around the world
create timely technical
information based on realistic
scenarios. Specific
recommendations are provided
to help you implement IT
solutions more effectively in
your environment.
For more information:
ibm.com/redbooks
SG24-7273-00
ISBN 0738494968

PDF - IBM Redbooks

Transcription

Similar documents

Home Networking: Building an Ethernet LAN

Installing And Configuring Windows Deployment

MODEL LM 1 - Log Master Portable Saw Blade

FKN Depaneling Data Sheet

JOEWELL PRODUCT GUIDE 【Specification of Scissors】

Xactimate System Requirements

questmaster-manual - Museum of Computer Adventure Game History

Extreme curve cutting speed

Installing SolidWorks Network Licenses