Virtual Server Farms with Mainframe Linux

Transcription

Sine Nomine Associates
Virtual Server Farms
with Mainframe Linux
Scott D. Courtney, Senior Engineer
http://www.sinenomine.net/
[email protected]
© 2005 Sine Nomine Associates
Contents
 Mainframe 101
 Overview of Linux on zSeries
 Selected customer experiences
 Challenges of zSeries Linux deployment
© 2005 SNA
Mainframe 101
© 2005 SNA
Mainframe 101 -- History in brief
 Mainframe Hardware
– IBM 360 introduced c. 1963
– IBM 370 introduced c. 1971, virtual memory added 1973
– IBM 3090 introduced c. 1985
– In about 1990, IBM renamed the ES/9000 as "S/390" (31-bit*)
– IBM zSeries (current generation) introduced in 2000 (64-bit)
 Software Lineage
– VM (virtual machine) created in 1960s, officially released for 370
– VM is still in widespread use today as o.s. and as hypervisor
– First S/390 Linux port was "Bigfoot", done in 1998
– Port by IBM (contribs by others) created in "skunkworks" in 1999
– IBM's first official S/390 Linux announcement: May 17, 2000
* Older S/390 architecture was a hybrid 31/32-bit system.
IBM calls it "31 bit", but for our purposes it's 32-bit equivalent.
© 2005 SNA
Mainframe 101 -- Not your father's dinosaur!
 IBM's top-of-the-line z990 Model D32 ("T-Rex") has:
– 16GB ~ 256GB main memory
– Up to 16 dedicated crypto co-processors (for SSL, etc.)
– Up to 48 GigEthernet channels, up to 12 10-Gig Ethernets
– Up to 30 LPARs (plus thousands of VMs)
– Full 64-bit architecture, but can support 31-bit VM guests
– Processors are 12-way multi-chip modules (MCMs), each with
– 8 main processors for general use
– 2 supervisory processors
– 2 spares that hot-swap if any other processor faults
– Up to four MCMs per z990, so up to 32 general processors
 In addition, modern mainframes are not like in the old days:
– Direct connectivity to Fibre Channel SANs, SCSI storage
– Prices have dropped enormously in the past five years
© 2005 SNA
Mainframe 101 -- Not just a big PC
 Superior Industrial-spec or MIL-SPEC components, boards
 ECC memory, ECC backplanes
 Hipersockets are a concept simply
not found in PCs
 Extremely high levels of redundancy,
fault tolerance
– MTBF measured in decades, not years
– Self-diagnosis, extensive hot-swap
capability
 No such thing as "dumb" I/O --- ever!
 Very high-level (i.e., "very" CISC) instruction set
– Performance not directly comparable against x86 or RISC clock
– No timer-tick interrupts (delays handled in hardware, µcode)
 Optimized for continuous high utilization percentages, not burst
© 2005 SNA
Mainframe 101 -- Networking
 Hundreds or even thousands of "guest LANs", or "gLANs"
– Usually implemented with HiperSockets connectivity
– "Guest" used rather than "virtual" to distinguish from IEEE
802.1q VLAN (new zSeries also support that standard, though)
– Operate at backplane speeds, measured in gigabytes/second
– Separate, dedicated backplane channels, not just RAM bus
– Under VM operating system, the HiperSockets themselves can
be virtualized
– gLANs are fully isolated with security enforced by µcode
 External network adapters support either L2 or L3 or both
– L3 offloads much of the processing for IP-only networks
– L2 offers bridge-like flexibility for IPX, DECnet, OSI MAP....etc.
 Multi-port network adapters at GigE or 10GigE speeds
© 2005 SNA
Virtualizing the Virtual:
It’s Turtles All the Way Down!
VM can virtualize virtual hardware, n levels deep, and can run thousands
of images at once.
 Test Plan Charlie: 41,400 Linux images
in an LPAR on a G5
 Test Plan Omega: 97,943 Linux images
on a ZZ7, 12-way@160 MIPS each,
16G RAM
 Thornton: Linux/390 under Hercules under Linux/390 under VM …
whoda thunk it?
These specific demonstrations are “lab queens” but the practical value of
this capability is very, very real!
© 2005 SNA
Overview of zSeries Linux
© 2005 SNA
Limitations (real & perceived) of traditional mainframes
 New hires don’t know how to run it
 It’s….(gasp)…old!
 Expensive per unit of processor power
 Expensive software licensing, closed source
 EBCDIC character set
 Batch-oriented, non-interactive
– Mainframe I/O optimized for throughput, not latency
– Some of this is just perception, not reality
 Slow pace of innovation
 Old-fashioned development environment
 Difficulty "gluing" to intranet, internet, distributed applications
 It won’t run on my PC!
© 2005 SNA
Mainframe Linux Becomes Reality
 Linas Vepstas: Bigfoot
 IBM Boeblingen builds “official/unofficial” port
 Marist College distribution
 May, 2000: “It’s official!”
 LinuxWorldExpo 2001: That’s “B” as in “Billion”
 Features of S/390 Linux:
– Native port, not emulation
– ASCII character set just like other platforms
– Runs in LPAR, bare metal, or under VM
– S/390 architecture is in the stock kernel
– Open Source like other versions
© 2005 SNA
Advantages Linux brings to S/390 or zSeries
 Server consolidation on an unprecedented scale
 Leveraging Open Source in large enterprises
 Leveraging existing infrastructure, disaster planning,
hardware support, staffing, while deploying new apps
 Tremendous integrating “glue” for tying mainframe to
intranet or internet
 Licensing costs significantly lower
 Compliant with open standards for API, languages,
and data formats
 New hires now, in effect, trained on Big Iron
 From IBM's perspective, new marketing direction
– Mainframe MIPS sales per year way up since
Linux
© 2005 SNA
Key Advantages of VM for Linux
 Instrumented for performance and reliability monitoring
 Horizontal scalability
 Root access can be given without compromising the
mainframe’s native operating environment
 Run parallel with existing mainframe applications
 Use existing, proven, well-understood management
and backup tools
 Risk-free and rapidly deployed test and pilot
environments
 Distributed application development, centralized
security and infrastructure
 New level of enterprise credibility for Linux as a whole
© 2005 SNA
“Best of Both Worlds”
 Reliability, availability, scalability
as expected from a mainframe
 Common operating system across
all architectures, from mainframe
to embedded
 Rapid innovation from the Linux
and Open Source community
 Synergy of personnel from
diverse IT backgrounds
© 2005 SNA
Disadvantages of zSeries Linux
 Compute-bound tasks may not perform well
– Mainframe optimized for throughput, I/O, not burst latency
– Simple fact of sharing finite resources among a lot of
processes
 Optimized for high throughput, not low latency
– Bad choice for real-time embedded platform (duh!)
– May not be a good choice for streaming media (?)
 Still has financial barrier to entry relative to commodity Intel
systems
© 2005 SNA
Sine Nomine Associates’
Selected Customer Experiences
 Customer 1: Email Infrastructure
 Customer 2: Financial Services
© 2005 SNA
Customer 1: Distributed Mail
 9672-R26 mainframe as host platform
 Providing DNS, SMTP, IMAP4, POP3 as primary services
 Limited FTP and HTTP access as secondary services
 Multiple virtual domains, 5000+ users
© 2005 SNA
Customer 1: Configuration
 Single mainframe with many VM Linux instances for
horizontal scalability and functional partitioning
 SMTP domain servers identically configured, horizontally
scaled
 IMAP/POP servers accessed mail directories using NFS on
common file servers
 Careful gLAN configuration to partition the bandwidth
requirement
© 2005 SNA
All machines
support ssh for
remote
administration
Gigabit OSA
Customer 1:
Block Diagram
to DNS1, Auth,
NFS servers,
SMTP, Hermes
Each service
(except DNS)
may contain
multiple
machines
behind roundrobin DNS
records
Ports 20, 21,
23 forwarded to
login
Router
Point-to-point
QUICKDSP
absolute share 2%
gLAN 4
Port 25 from
external source
forwarded to
incoming
Port 25 from
internal source
forwarded to
SMTP
Legacy services
IP forwarding
VM TCP/IP stack
Port 80
forwarded to
WWW
routing
port-forwarded
point-to-point
ssh to NFS
servers
Port 110
forwarded to
POP
32 MB
64 MB swap
DNS1
DNS2
BIND
load balancer
BIND
load balancer
gLAN 1
auth
relative share
5000
WWW
ftp/login
IMAP
POP
incoming
SMTP
sendmail
NIS master
WWW server
NFS client
automounter
NIS client
ftp
login
unique shell
NFS client
automounter
NIS client
WWW server
NFS client
automounter
NIS client
NFS client
IMAP 2000
automounter
NIS client
NFS client
qpopper
Service machines:
64 MB
128 MB swap
gLAN 2
Point-to-point
from router
NIS client
NFS client
sendmail
gLAN 3
nfs-home
/home
Point-to-point to auth
absolute share 1%
nfs-mail
/var/spool/mail
Point-to-point
from router
absolute share 1%
NIS client
NFS server
LVM
port-forwarded
ssh from router
NIS client
NFS server
LVM
port-forwarded
ssh from router
© 2005 SNA
Customer 1: Lessons
 Some tasks turned out to be more compute-bound than
originally anticipated
– SSH/SSL encryption
– Better with hardware acceleration on newer
mainframes
– Dynamic web content
 More memory in the virtual environment does not always
equate to faster performance
– Ideal situation is actually to starve Linux from using
disk cache, let mainframe smart I/O handle this
– Linux-level swapping is extremely cheap under VM
 NFS is not all that good as a distributed filesystem
– This has mproved in newer versions of NFS available
since this project was completed
© 2005 SNA
Customer 2: Financial Services
 Feasibility study / test implementation
 Existing application newly in production
 Current platform is UNIX and NT/IIS
 Web-based 4-tier implementation in Java
 BEA: WebLogic Server
 Task: Port application, evaluate performance
© 2005 SNA
Customer 2: Application Port
 Three people, six hours, one working app!
 Zero source code changes (config files only)
 Virtual network setup for app-level clustering
 BEA: WebLogic Server is supported on zSeries Linux
 One z/VM instance, two zSeries Linux instances, four IFL
processors – all added to an existing mainframe with zero
downtime
© 2005 SNA
Customer 2: Performance Testing
 Gold standard: Match their existing production system’s
response time
 zSeries with 1 CPU exceeded performance target by
approximately 3X
 zSeries with 2 CPUs was too fast for their test environment
to saturate, but ran at least 6.5X their normal production
load
 External NT systems and LAN were the bottleneck
© 2005 SNA
Customer 2: Test Notes
 Over 30 hours of intense high-load testing, zero failures of
zSeries hardware, z/VM, or Linux
 Even at saturation load on one CPU, no software failures
or crashes
 No application or BEA tuning for Linux platform (same
parameters as on UNIX)
 Added and removed processors dynamically without
rebooting virtual machines
© 2005 SNA
Lessons Learned
 VM is critical to large scale Linux for System/390 scalability
– Limited LPAR count does not offer sufficient cost/benefit to
make the case for Linux on S/390 iron
– Loss of VM resource management and error recovery
substantially complicates system management
 Lack of VM on other platforms is a major differentiator in
favor of zSeries
© 2005 SNA
Challenges for zSeries Linux Deployment
 Political Challenges
 Technical Challenges
 Project Management Challenges
© 2005 SNA
Political Challenges
 Challenge: How to sell Linux and Open Source idea to
senior management?
 zSeries Linux answers:
– Deploy alongside existing mainframe software, without
interruption to production
– Small project first, often infrastructural in nature
– z/VM is key to flexibility of pilot environment
© 2005 SNA
Political Challenges (cont’d)
 zSeries Linux answers (continued):
– Personal case history: Pairing a VM guru with a Linux wiz
– Empowerment of open systems managers via access to
larger-scale resources, new data sources
– New levels of integration between legacy data and new
intranet/internet/extranet applications
© 2005 SNA
Technical Challenges
 Backup/Recovery Solutions and DR
– Can be done within Linux, but not always best way
 Performance Monitoring Instrumentation
 Configuration Management
 Security Management
 Software Replication
– Shared read-only filesystem is one option, but more flexible
options are needed
© 2005 SNA
Project Planning Challenges
 What type of project first?
 Implementation planning
– Who is involved, and at what point in time?
– Need collaboration between mainframe and UNIX/Linux
personnel
– System automation tools from z/VM environment applied to
Linux instances
– Vertical and horizontal scalability planning
© 2005 SNA
Lessons Learned
 Mainframe Linux is now a proven
technology, not just an interesting
experiment
 Performance issues can surprise you -understand the workload, and benchmark
 Don’t assume “conventional wisdom” is
always right
 Consider non-technical factors in project
planning, especially for first Linux
deployment
 VM/Linux may be the wrong answer, but
when it’s right, it can be spectacularly right!
© 2005 SNA
Daddy! I Wanna Play, Too!!!
 Linux Community Development System
– Free root-level S/390 Linux account for 30, 60, or 90 days
– Any reasonable purpose (no spam, pr0n, game servers, but
"just learning" is fine)
– S/390 G5 or G6 system...not blazing, but okay for learning
– http://www-03.ibm.com/servers/eserver/zseries/os/linux/lcds/
 Hercules
– Hardware emulation of S/390 or zSeries
– Possible, but not legal to run VM in this setting
– http://www.conmicro.cx/hercules/
 Flex
– PC-based emulation, but licensed by IBM for z/VM
– Powerful but costly
© 2005 SNA
Resources on the Web
 This Presentation (downloadable)
http://sinenomine.net/node/520
 Linux/390 Project Home Page
http://www.linuxvm.org
 IBM Linux zSeries Home
http://www.ibm.com/servers/eserver/zseries/os/linux/
 IBM VM Linux Resources
http://www.vm.ibm.com/linux/
 Mainframe Historical Timeline
http://www-03.ibm.com/servers/eserver/zseries/timeline/
 IBM Linux Community Development System
http://www-1.ibm.com/servers/eserver/zseries/os/linux/lcds/
 “Dream Machine” Article Online
http://www.linuxplanet.com/linuxplanet/reports/1532/
 Sine Nomine Associates z/VM FAQ
http://sinenomine.net/node/10
(Look under "Supporting guest operating systems" for Linux info)
© 2005 SNA

Virtual Server Farms with Mainframe Linux

Transcription

Similar documents

Optima Camelview Village

slides - Linux Audio Conference

zoning map - Town of Kaukauna

Zimmel Associates - New Jersey Business Magazine

Kate Cairns Associates: Online Courses

SPEAR Processors Presentation by Amir Sherman

Discover what makes Zumasys one of the best places to work

Valmark Associates Produces DVD