HP RA for SAP HANA backup and recovery using Symantec
Transcription
HP RA for SAP HANA backup and recovery using Symantec
Technical white paper HP RA for SAP HANA backup and recovery using Symantec NetBackup HP StoreOnce, HP StoreEver, HP ProLiant and HP StoreFabric Table of contents Executive summary ...................................................................................................................................................................... 3 Introduction .................................................................................................................................................................................... 3 Overview .......................................................................................................................................................................................... 4 Scale-up configurations .......................................................................................................................................................... 4 Scale-out configurations ......................................................................................................................................................... 4 Backup options .......................................................................................................................................................................... 4 Solution components ................................................................................................................................................................... 5 Symantec NetBackup ............................................................................................................................................................... 5 SAP HANA Studio ....................................................................................................................................................................... 9 HP ProLiant DL380p Gen8 server........................................................................................................................................ 10 HP StoreOnce Backup ............................................................................................................................................................ 10 HP StoreEver Tape .................................................................................................................................................................. 17 HP StoreFabric SAN switches ............................................................................................................................................... 18 HP 5920 Series switch ........................................................................................................................................................... 19 Capacity and sizing ...................................................................................................................................................................... 19 Workload data/results – backup performance ................................................................................................................ 22 Workload data/results – restore performance ................................................................................................................ 23 Scalability testing .................................................................................................................................................................... 24 Deduplication efficiency testing ........................................................................................................................................... 25 Disclaimer This is a reference architecture (RA) based on HP solutions and non-HP third-party solutions. HP provides support only for the HP products and solutions. Product support for non-HP products is provided by the respective vendors. Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup General recommendations ................................................................................................................................................... 26 Configuration guidance .............................................................................................................................................................. 27 HP StoreFabric configuration ............................................................................................................................................... 27 HP 5920 Series switch configuration ................................................................................................................................. 27 HP ProLiant DL380p server configuration ......................................................................................................................... 27 HP StoreOnce configuration ................................................................................................................................................. 28 HP StoreEver configuration .................................................................................................................................................. 28 SAP HANA Studio ..................................................................................................................................................................... 29 Symantec NetBackup configuration ................................................................................................................................... 29 HANA node configuration ...................................................................................................................................................... 31 HP StoreOnce configuration – replication ......................................................................................................................... 31 Suggested backup strategy .................................................................................................................................................. 34 Bill of materials ............................................................................................................................................................................ 35 Summary ....................................................................................................................................................................................... 37 Implementing a proof-of-concept .......................................................................................................................................... 37 Appendix 1: Bill of materials of tested configuration .......................................................................................................... 38 Appendix 2: HP Enterprise Services can help ........................................................................................................................ 39 Glossary ......................................................................................................................................................................................... 40 For more information ................................................................................................................................................................. 41 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Executive summary SAP HANA is SAP’s In-Memory-DataBase (IMDB) computing platform, and as customers are realizing the benefits of IMDB technologies, and they are accelerating their plans to deploy/port their existing SAP applications to SAP HANA, they are also recognizing the need to protect their mission critical HANA databases. Because SAP HANA is a relatively new database technology there are very few SAP certified backup and recovery solutions in the SAP HANA marketplace, and SAP HANA customers face new challenges with this relatively new technology. There are four backup-related challenges faced by existing SAP HANA customers: • As their data is growing, there is a need to back up large HANA databases with a mature enterprise backup solution within a short time span. • Customers want to minimize their backup and recovery expenses (both CapEx and OpEx). • Customers want to have a reasonably low recovery time objective (RTO) and zero recovery point objective (RPO) for applications in the event of a disaster. • Recognizing that replication alone within SAP HANA is not a complete data protection solution, replication and backup must be combined together to provide a complete data protection solution. (Replication does not protect from corruptions and deletions.) The solution covered within this white paper details a reference architecture to back up HP AppSystems for SAP HANA using Symantec NetBackup, HP StoreOnce, HP StoreEver, HP StoreFabric, HP 5920 Switch and HP ProLiant Server. The document shows that HP can provide all hardware components required for effective protection of an SAP HANA database appliance. The document also lists the advantages of using HP and Symantec technologies and the potential benefits that a customer can expect in terms of storage efficiency and backup/restore performance. It also substantiates the various internal tests done by HP to ensure that the solution is efficient and all components integrate well and work together seamlessly. Target audience: This white paper is intended to assist SAP solution architects, SAP HANA database administrators, backup and archive solution architects, backup administrators, storage administrators or IT professionals who are involved in architecting, planning or deploying an SAP HANA implementation. The reader is expected to have a basic knowledge of SAP HANA, storage, SAN and backup technologies. Document purpose: The purpose of this document is to describe a reference architecture solution, highlighting recognizable benefits to technical audiences. It provides guidance to deploy a backup solution for SAP HANA using HP and Symantec technologies for the target technical audiences and customers. This white paper describes tests performed by HP during October-November 2013 and February-March 2014. Introduction SAP HANA and HP AppSystems for SAP HANA have been in use for a few years. While there are a number of HANA appliances available from multiple hardware vendors, there are very few backup solutions available to back up a HANA database. Because SAP HANA is a relatively new database technology, some SAP HANA customers are unknowingly using a less than complete data protection solution for their SAP HANA environment, thus creating costly data protection solutions that add complexity and increased risk of data loss. SAP HANA customers are realizing that replication and backups must be used together to provide a complete data protection solution as replication alone does not provide protection from data corruption or data deletion. Furthermore, some SAP HANA customers are dumping their IMDB (In-Memory Database) to disk, then backing up the newly created flat files to a NAS share mounted on the SAP HANA nodes using an existing backup solution. While dumping the SAP HANA database to disk, and then backing up the newly created flat files technically “works,” it is a less than ideal backup solution as it introduces additional OpEx, as well as additional complexity and risk of data loss. “Dumping to disk then backing up flat files” is a “files-based backup” that requires the SAP administrator to keep track of where the files-based backups are written, as well as specify their location in the SAP HANA Studio console during a recovery. Furthermore, using a “files-based backup” can increase the risk of data loss since this manual process introduces the potential for human error. (If the specified location of the files-based backups is located on the same filesystem/disk as the production data then both the production data and the backup data could both be lost in the event of the filesystem/disk failing.) In addition to the cost and risk issues associated with the “dump to disk, and then backup the flatfiles,” writing backups directly to a NAS share is known to cause performance problems. For these reasons, and more, using an SAP certified backup solution, such as Symantec NetBackup, to protect the SAP HANA database is highly recommended. SAP certified backup solutions use the SAP backint API to write the backup directly to any desired media, and are automatically recognized within the SAP HANA Studio console, as the location of the backup data does not have to be specified in the event of a recovery (less OpEx, and less risk of data loss). 3 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Symantec’s latest version of their industry leading backup solution, NetBackup (7.5.0.6 and above), now contains the ability to back up SAP HANA databases using the SAP backint API. HP has developed and tested a solution for backing up HP AppSystems for SAP HANA using various HP technologies and Symantec NetBackup. This paper introduces each of the components of the solution and also provides details of how each component is applicable to the SAP HANA backup solution. The capacity and sizing section, provides guidance on how to derive the right capacity required and suggests a backup strategy that may be adopted. This document also provides the details of internal testing, test results, recommendations and best practices. The Configuration guidance section shows how to configure each component of the solution, as well as how to use all of the components together to produce a complete SAP HANA backup solution. The document closes with a bill of material, summary, and a glossary to cover the acronyms used throughout the document and also some pointers to further reference material. Overview HP and SAP have teamed up to offer HP AppSystems for SAP HANA as a portfolio of optimally configured hardware appliances with preloaded software and a full range of included services – design, factory integration, on-site installation, and proactive support with a single point of contact. Configurations and options follow. Scale-up configurations HP offers single-node configurations (also known as scale-up HANA appliances) based on HP ProLiant DL580 and DL980 servers. These configurations range in size from two processors and 128GB of RAM to eight processors and 1TB of RAM to accommodate a range of business requirements. These configurations are suitable to run any of the SAP applications, for example, SAP Business Suite powered by SAP HANA, SAP NetWeaver Business Warehouse powered by SAP HANA, and operational data mart. Scale-out configurations HP offers scale-out configurations that support very large data volumes on SAP HANA. These configurations use industry-leading HP ProLiant BL680c server blades and HP 3PAR StoreServ storage. Backup options SAP HANA backups can be implemented in multiple ways depending on the customer’s existing IT setup. A customer may choose to go with Catalyst (OST) based backup, or if they have a SAN in place they may opt for VTL based backups also. If the data is very critical, customers may also choose to go with backup replication or media vaulting to an alternate site. This document covers scale-up and scale-out appliances, shown in Figure 1, and provides the options and ways in which a customer can deploy Symantec NetBackup, HP StoreOnce and HP StoreEver to back up a HANA database. The document also provides configuration guidance to configure various solution components and ends with a recommended bill of materials to back up HP AppSystems for SAP HANA based on the HANA database size. Note This document has been written keeping in mind the scale-up HP AppSystems for SAP HANA based on HP ProLiant DL580 and DL980 G7 servers and it also covers scale-out configurations based on HP BladeSystem c7000, HP ProLiant BL680c G7 and HP 3PAR StoreServ. This document also applies to HP ConvergedSystem 500 for SAP HANA Scale-up and Scale-out configurations based on HP ProLiant Gen8 servers, although it might change slightly from the sizing point of view. 4 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Figure 1. HP AppSystems for HANA with backup components Solution components HP solutions for backing up SAP HANA have been designed keeping in mind the available memory on the appliance and also HANA database size. The solution components include Symantec NetBackup and Symantec NetBackup HANA Agent, HP StoreOnce Backup (Virtual Tape Library (VTL) or Catalyst Stores (OST)), HP StoreFabric SAN switches, HP 5920 Series switch, HP ProLiant DL380p Gen8 server and HP StoreEver MSL2024 Tape Library for media vaulting. In the next few sections we briefly introduce each of these components and list the direct or indirect advantages of using these components or their applicable features to Symantec NetBackup and SAP HANA. The Configuration guidance section describes how all of these components can be configured to work together to protect SAP HANA. Symantec NetBackup Symantec NetBackup is a leading enterprise backup software application that is built to protect the largest and most demanding enterprise environments. NetBackup provides unmatched features and functionalities to protect all popular operating systems, databases and applications. Symantec NetBackup offers maximum flexibility with protecting enterprise environments as it enables backups to disk, tape, virtual tape libraries, and deduplication storage devices such as HP StoreOnce Backup devices. NetBackup can be used to consolidate the backup environment as it can simultaneously back up multiple servers and share and optimize the backup resources like tape drives, backup libraries and virtual tape libraries. Symantec NetBackup architecture Symantec NetBackup has a 3-tier architecture that consists of a master server, media servers and clients. Figure 2 briefly explains the NetBackup Architecture and the role of master server, media servers and clients. The master server maintains a database with information about backup images, system configuration, and available backup resources. The NetBackup master server is also responsible for initiating backups as per the schedule of any configured policies. The master server is responsible for scheduling and allocating the required resources to complete a backup or restore job. A NetBackup master server and a media server can co-exist on a single server or a single NetBackup master server can manage multiple media servers within its domain. As shown in Figure 2, a media server owns one or more backup devices such as tape drives, tape library, Virtual Tape Library (VTL) or Open Storage (OST) devices such as HP StoreOnce Catalyst Stores. Backup resources within a NetBackup domain, such as tape drive or a tape library can be partitioned and shared between multiple media servers. The primary function of a media server is to read data from the NetBackup client and write the backup data to the designated backup media (tape, disk, VTL, deduplication storage 5 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup appliance). A NetBackup client is a machine (physical or virtual) that needs to be protected. The NetBackup client includes all binaries necessary for protecting enterprise applications, such as Oracle and SAP HANA (no additional agent needs to be installed). Symantec NetBackup supports all major operating systems on NetBackup clients, including but not limited to Microsoft® Windows®, Linux (SUSE, Red Hat®), UNIX® (HP-UX, IBM AIX, Oracle Solaris), etc. Figure 2. Symantec NetBackup architecture Policies: In NetBackup, policies are created to define the backup parameters of backup jobs. A policy basically has the details of which servers to back up (backup clients), what to back up (backup data selection), when to back up (backup schedule) and where to back up (the backup device or media to be used). A backup policy is used to define the details of the backups to run, including: client systems to be backed up, schedules to be used for backups, storage devices to be used for backups (storage units and storage unit groups), and any special attributes for the backups. Storage units: In NetBackup, storage units are created for the various types of backup devices available in a backup environment. A storage unit could be a tape drive, a tape library, a VTL or disk-based backup device. A storage unit is associated with a media server and it could also be shared among multiple media servers. Storage unit groups: NetBackup allows creation of storage unit groups to create pools of backup resources. Storage unit groups can be assigned to a policy instead of to an individual storage unit. The main advantage of using a storage unit group is that if there is a problem or an issue with a particular media server or a storage unit, NetBackup automatically picks up another available media server or storage unit and executes the backup. This provides flexibility to a Backup Administrator and can prevent backups from failing. A storage unit group can be configured to use storage units in a preferred order or round robin for load balancing. 6 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Figure 3 provides a logical view of the integration between the SAP HANA Node, Symantec NetBackup, HP StoreOnce (Catalyst and VTL) and HP StoreEver (MSL). As shown, the NetBackup client needs to be installed on each SAP HANA node in the SAP HANA environment. As mentioned before the NetBackup client binaries include the NetBackup Agent for SAP HANA, so no additional agents or software needs to be installed, only the NetBackup client. Figure 3. SAP HANA, Symantec NetBackup and HP StoreOnce logical integration Symantec NetBackup for SAP HANA NetBackup (7.5.0.6 and above) contains the ability to back up SAP HANA databases using the SAP backint API. No additional software or agents need to be installed, simply install the NetBackup client software (7.5.0.6 or above) on each node in the SAP HANA environment and the software installation portion of the backup solution is complete. Once the NetBackup client binaries have been installed on each node in the SAP HANA environment, and the necessary configuration steps have been performed (see the Symantec NetBackup configuration section for details), backups can then be initiated from the SAP HANA Studio console. Once a backup has been initiated, it spawns five backup jobs – one parent job and one backup job for each SAP HANA service being backed up (XSEngine, Name Server, Indexserver, and Statistics Server). On a HANA scale-out multi-node cluster there are five jobs on the master node and one Indexserver job on each worker node. Details of each backup job status can be viewed in real time from the NetBackup admin console or from the SAP HANA Studio console, which simply shows “percent complete” for each backup job that runs on each node in the SAP HANA environment. NetBackup uses the SAP backint API to create backups and to automatically create new log files once a backup is completed. NetBackup can also be configured to automatically backup SAP HANA log files on a specified interval. Symantec NetBackup ensures seamless recovery of an SAP HANA database. An SAP HANA database can be recovered to the time of any backup image or to any point in time by applying log files. With point-in-time restore, the last complete backup is restored and then all the sequential logs generated by HANA are replayed to the point in time that the restore is requested. All this is a very simple process that can be initiated from within SAP HANA Studio 7 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup NetBackup Administration Console, shown in Figure 4, is the primary administration tool in NetBackup. This single tool can be used for administering a large number of servers, drives, media, libraries, backup jobs and their schedules. The figure shows the NetBackup components like policies, storage units, reports, and the member hosts like NetBackup master servers, media and clients. The status of backup jobs (Active, Done, Duration, etc.) can be seen in the snapshot. The NetBackup Administration Console provides several intuitive wizards to help the administrator to execute routine tasks, for example, configure storage devices and robotic libraries, media servers, storage servers, and backup policies. Among other options available from the console, several reports can be generated to see the status of backup jobs, client backups, backup images available, tape reports, disk reports and vault reports. Most of the backup administration tasks can be carried out from the NetBackup Administration Console. Figure 4. Symantec NetBackup Administration Console Key point The status of HANA backup jobs can be monitored from the NetBackup Administration Console; if there are any issues with backups, most preliminary troubleshooting information can be gathered from here. Customers should consider email alerting for backup failures on their critical databases. Integrating NetBackup with their existing incident management applications would also help in timely and automatic creation of incidents in cases of backup failures. Symantec NetBackup also offers a rich set of administration commands and options (CLI) for executing various administration tasks, and can be used instead of the GUI interface if desired. 8 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup SAP HANA Studio SAP HANA Studio is the front end application that can be used for multiple purposes, such as HANA configuration, administration, development, and SQL client. Multiple SAP HANA database instances can be added to a HANA Studio. SAP HANA Studio also allows an administrator to configure and initiate backups and system replication from within HANA Studio. As shown in Figure 5, the left side can have multiple SAP HANA instances that can be managed and the right side shows the details of the instance selected on the left. The right side shows some vital information like the current HANA version running, database memory used, disk usage and log volume size. Figure 5. SAP HANA Studio SAP HANA Studio allows hot backups of the HANA database without the need of shutting down the database. There are two options for backups, i.e., file and backint. File backups, by default create five files in a specified folder. These five files correspond to SAP HANA services running on the SAP HANA master server. In case of a scale-out system, one additional job gets created for each worker node. The other option for backups is to use backint. The backint option integrates with Enterprise class backup solutions like Symantec NetBackup and uses the existing NetBackup infrastructure to execute backups. With the backint option, five NetBackup backup jobs are created for the master node and one backup job for each worker node; each of these jobs backs up one SAP HANA service. The default method is to manually initiate backups within HANA Studio and there is no option to schedule automatic backups. Symantec NetBackup, however, provides a workaround to schedule backups using a script. SAP HANA Studio provides three options for restore: • Recover the database to its most recent state: This option uses the last backup for restoring the database and then restores and replays all the applicable logs to restore the database to the most recent consistent state. • Recover the database to the following point in time: This option restores the last database backup taken before the point in time specified and then replays the logs until the point in time selected. This option is useful to roll back the database to a particular point in time or to a time when the database was in a known consistent state. • Recover the database to a specific data backup: Using this option, the database is recovered to a specific data backup and the log area is initialized, all changes made after this backup are not restored. No logs are replayed using this option. HANA database has to be in a shutdown state before a restore job can be started. If the database is running, the recover wizard first initiates a shutdown of the database and its services before the wizard starts. 9 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Figure 6 shows the options provided by SAP HANA Studio to monitor backups. SAP HANA Studio also provides several options to monitor the backup history along with backup statistics like data size and throughput, backup date and time, backup duration, etc. The Backup Catalog tab includes the details of both file and backint backups; the data backup selected on this tab on the left shows further details on the right, like the total size of each service that was backed up. Note that among the five services that were backed up, all except for Indexserver were very small in size which is the main reason why backup load was not balanced into multiple streams. In case of a scale-out system, we would see one additional job for every worker node, this job corresponds to Indexserver service running on each HANA node. Opening the adjacent Configuration tab shows several backup-related parameters such as the default backup destination type, i.e., file or backint, the default backup method, i.e., automatic or manual, and the default location. Figure 6. SAP HANA backup tab HP ProLiant DL380p Gen8 server HP ProLiant servers are industry leading servers in x86 architecture and come in various forms such as rack-mount, tower and blade servers. These servers are also available with popular processors from Intel® and AMD and range from one socket to eight sockets. For the purpose of this solution a medium range server, i.e., HP ProLiant DL380p Gen8 server was chosen. This server has two Intel Xeon® E5-2600 v2 series processors, and can have up to 768GB memory and up to twelve cores per processor. This server has been chosen considering the processor and memory requirements of Symantec NetBackup and the SAP HANA database size. Key points This server functions as a management server for the SAP HANA backup solution. This management server will provide three primary functions, i.e., NetBackup Master Server, NetBackup Media Server and SAP HANA Studio will also run from this server. This server can be ordered with a two port 10Gb Ethernet adapter to be deployed in a NIC team to create a separate backup network; which can speed up HANA backups. HP StoreOnce Backup The HP StoreOnce Backup system is a disk-based storage appliance for backing up multiple servers or PCs to target devices on the StoreOnce appliance. These devices are either configured as Network-Attached Storage (NAS), StoreOnce Catalyst Stores (OST) or Virtual Tape Library (VTL) targets for backup applications. The total number of backup target devices provided by an HP StoreOnce Backup system varies according to model. These devices may be all StoreOnce Catalyst, all VTL, all NAS or any combination of Catalyst, NAS and VTL devices. All HP StoreOnce devices automatically utilize StoreOnce deduplication, ensuring efficient and cost-effective use of disk space. A further benefit of StoreOnce Catalyst devices is that deduplication may be configured to occur on the media server (low bandwidth) or on the StoreOnce Backup system (high bandwidth), allowing the user to decide what makes most efficient use of available bandwidth. 10 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Figure 7 shows the StoreOnce architecture and use cases. As shown in the left rack, there are single node systems (4700/4900), and highly available (HA) scale-out (6500) models available in StoreOnce. Within HA models, a pair of highly available nodes are called a couplet and there can be up to four couplets in a single StoreOnce 6500. Each couplet can have up to 560 drives. Multiple VTLs of different emulations and media types like DLT or LTO are supported within a single StoreOnce unit. This is illustrated with VTL1 on the StoreOnce 4700 and VTL2 to VTL5 on the StoreOnce 6500 in the left rack. Multiple NAS shares running over NFS or CIFS are also supported within a single StoreOnce unit (see NAS Share 2 and 3). HP StoreOnce can be a home for multiple Catalyst Stores for storing NetBackup backups (see Catalyst Store 1 to 4 on the left rack). HP StoreOnce supports long distance low bandwidth replication over Ethernet. Some of the VTLs, NAS shares and Catalyst Stores are mapped with similar devices on the right rack. Physical device models for replication need not be the same for replication. Figure 7. HP StoreOnce architecture and concepts HP StoreOnce – data deduplication HP StoreOnce Technology is a 5-step “inline” data deduplication process (Chunk Hash Match Compress Store). It uses hash-based chunking technology, which analyzes incoming backup data in “chunks” that average 4K in size. The hashing algorithm generates a unique hash value that identifies each chunk and points to its location in the deduplication store. Hash values are stored in an index that is referenced when subsequent backups are performed. When data generates a hash value that already exists in the index, the data is not stored a second time, but rather the count is increased showing how many times that hash code has been seen. Unique data generates a new hash code and that is stored on the StoreOnce appliance. Typically about 2% of every new backup is new data that generates new hash codes. With VTL and NAS shares, deduplication always occurs on the StoreOnce Backup system. With Catalyst Stores, deduplication may be configured to occur on the media server or on the StoreOnce Backup system; these are also called low bandwidth mode or high bandwidth mode. See Figure 8 for a comparison of low and high bandwidth modes, where “Target side deduplication” corresponds to “High bandwidth mode” and “Source side deduplication” corresponds to “Low bandwidth mode”. In low bandwidth mode the OST/Catalyst integration allows for the processor-intensive deduplication tasks of chunking, hashing and compression to be “offloaded” onto NetBackup media servers. Low bandwidth mode is also called source side deduplication and it has two main benefits. The first benefit is that it increases the overall throughput available on the HP StoreOnce appliance when receiving data streams that have already been chunked, hashed and compressed. The second benefit is that low bandwidth backups make the process much more bandwidth efficient. Low bandwidth mode enables backups from remote sites to be performed over a WAN as it requires a much reduced network bandwidth from the media server to the HP StoreOnce Backup system. In high bandwidth mode, backup data is sent directly to the HP StoreOnce Backup system where it is deduplicated; this requires a high bandwidth network. This is also called target side deduplication. In this mode all deduplication steps like chunk, hash, match, compress and store are executed on the target StoreOnce Backup system. 11 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Figure 8. Target side vs. source side StoreOnce deduplication A point worth noting here is that StoreOnce does not provide the option of setting low bandwidth or high bandwidth mode for a VTL or a NAS share. For these two types of devices the deduplication always happens on the target device, i.e., StoreOnce Backup. Key points The obvious benefit of in-line deduplication for SAP HANA backups is the opportunity to save on actual storage consumption. Low bandwidth mode is generally more popular than high bandwidth mode. However for the purpose of this solution it is better to use Catalyst Stores in high bandwidth mode. The reason for this recommendation is that for a HANA backup solution a dedicated backup network is used and bandwidth is not a constraint. Secondly, HP testing revealed that higher throughput was possible in high bandwidth mode. HP StoreOnce – models for SAP HANA HP StoreOnce has been designed to cater to the needs of all types of customers from entry level to large scale enterprises. HP StoreOnce Backup systems deliver scale-out capacity and performance to keep pace with shrinking backup windows, reliable disaster recovery, simplified protection of remote offices and rapid file restore to meet today’s SLAs. The models vary by capacity and connectivity protocol and customers can start out by purchasing a single HP StoreOnce base unit/couplet, and then expand with additional couplets and expansion shelves. HP StoreOnce also has a Virtual Server Appliance (VSA) that is suitable for virtualized environments and smaller and remote offices. HP StoreOnce VSA is a VMware-based virtual appliance for virtualized data protection that utilizes existing infrastructure. Table 1 lists the features of HP StoreOnce models that are recommended for backing up SAP HANA. 12 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Table 1. HP StoreOnce Backup models for SAP HANA Product Description HP StoreOnce 4700 HP StoreOnce 4900 HP StoreOnce 6500 Description and Use Targeted towards mid-range customers. Targeted towards mid-range customers. Industry-leading, scale-out architecture with resilience and high availability. Scaleout architecture allows a customer to pay as they grow. Form Factor 4U Scalable Rack, 2U for every additional drive chassis 7U Scalable rack, 5U for every additional drive chassis 42U Scalable Rack, 5U for every additional drive chassis Total Capacity Up to 192TB (raw), Up to 160TB (usable) Up to 560TB (raw), Up to 432TB (usable) Up to 2240TB (raw), Up to 1728TB (usable) Device Interface 4 x 8Gb Fibre Channel 4 x 8Gb Fibre Channel 8 x 8Gb Fibre Channel Ports per couplet 2 x 10Gb Ethernet 4 x 10Gb Ethernet 4 x 10Gb Ethernet Ports per couplet 4 x 1Gb Ethernet 4 x 1Gb Ethernet 8 x 1Gb Ethernet Ports per couplet Disk Drives 2TB, SAS 7200rpm, 3.5-inch 4TB, SAS 7200rpm, 3.5-inch 4TB, SAS 7200rpm, 3.5-inch Number of Disk Drives 12 (min), 12 x 8 (max) 15 (min), 140 (max) Per Couplet: 30 (min), 560 (max) RAID Support Hardware RAID6 Hardware RAID6 Hardware RAID6 Number of StoreOnce Catalyst, Virtual Tape Libraries and NAS backup targets 50 50 384 Key Point HP StoreOnce Backup systems are built using HP ProLiant servers, hence they share common management tools with HP server products (for example, HP Systems Insight Manager (SIM) and HP Integrated Lights-Out (iLO)) reducing the time and energy it takes to manage the IT infrastructure. An added advantage of HP StoreOnce 6500 is that HP StoreOnce 6500 is a highly available large scale deduplication appliance with Autonomic Restart of backup jobs. Autonomic Restart ensures that, after a failure and system failover, the backup application jobs are able to restart automatically without any manual intervention. For SAP HANA it can go a long way in ensuring a higher backup success rate. HP StoreOnce – Catalyst Stores HP StoreOnce Catalyst targets are recommended for more complex data protection environments where flexibility of data movement is required. StoreOnce Catalyst can also enable distributed deduplication processing, balancing the load for optimum server, backup appliance and network. HP StoreOnce Catalyst delivers a single, integrated, enterprise-wide deduplication algorithm. It allows the seamless movement of deduplicated data across the enterprise to other StoreOnce Catalyst systems. HP StoreOnce has an Open Storage (OST) plugin for Symantec NetBackup. Multiple Catalyst Stores can be created within a StoreOnce device from the web management interface. The OST Plugin enables creation of a storage server within NetBackup. Catalyst Stores are associated with a NetBackup storage server and a NetBackup disk pool. These disk pools have to be configured as NetBackup storage units. In the process of creating a storage unit, a disk pool becomes associated with a NetBackup media server. A backup is actually written to a storage unit. Integration of Catalyst with OST allows Symantec NetBackup to control replication, backup life cycle management as well as perform automated disaster recoveries from backup copies from any of the available sites. 13 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Key points An advantage of using Catalyst Stores is that since they work on Ethernet, a customer can save on the cost of setting up a SAN for backups. Although, in the near future, Catalyst will run on both Ethernet and Fibre Channel, the cost advantage of running Catalyst on Ethernet would remain. The Workload data/results section conveys that these Catalyst Stores can enable high-speed HANA backups. HP StoreOnce provides a security mechanism such that a Catalyst Store can be restricted to be seen only by a valid server. If there are multiple SAP HANA appliances and instances and there is more than one NetBackup media server, security could be implemented to ensure that the Catalyst Store can only be seen by the NetBackup media server that is entitled to do so. HP StoreOnce – VTL VTL targets appear to the backup application as physical tape devices, with backup and recovery jobs managed in the same manner as with a physical tape device. The primary difference being that there is no physical media that must be managed. They are recommended for customers in Fibre Channel (FC) environments, or where a continued investment in tape hardware and software is a key consideration. A VTL is emulated out of a storage array of physical disk drives and functions like a physical tape library. It can have one or more tape drives, multiple virtual media and can even emulate a robot. A single HP StoreOnce device can emulate multiple VTLs at the same time. The StoreOnce enables N-port ID virtualization (NPIV) on the front end FC ports, each VTL logs into the fabric with a unique WWN through the front end FC ports. Key points One advantage of having the ability to create multiple VTLs within a single HP StoreOnce is that a library can be dedicated to each application. For example, if there are multiple SAP or SAP HANA instances running, their backups can be kept separate by allocating separate VTLs/storage units. VTLs are enablers for high speed backups as compared to tape libraries. Backups can be staged to a VTL by writing backups on virtual media and then duplicating them to a physical media using a NetBackup storage lifecycle policy. This eventually reduces the backup window required for taking large backups. VTLs provide the flexibility to emulate any specification of drive and media type. LTOx, LTO5 and DLT drives and media can be created within a VTL. For SAP HANA, a media type can be emulated so that it can accommodate a complete data backup within a single media. This action will ensure faster backups and restores. VTLs also provide the administrator options to choose the number of drives, media, slots and mail slots within a library. All of these indirectly provide the flexibility to define and use as much capacity as is required for the HANA instance backing up to this VTL. In general using physical media brings with it some of the issues like media alignment errors, and mounting and unmounting errors. Moreover physical media have a limited life and they have to be discarded/replaced after a limited number of read/writes cycles. Using VTLs can effectively help to avoid these errors and go a long way in ensuring smooth backups for HANA. HP StoreOnce – NAS NAS appliances provide general disk based file share and protection. HP StoreOnce Backup is designed as target storage for backup applications that need rapid restores and reduced backup windows. The device will present standard NAS protocols over the front-end 10GbE ports of the device, thus providing a LAN backup target (with deduplication) that can be used transparently with qualified enterprise backup applications. Compared with general purpose NAS appliances, HP StoreOnce Backup uniquely offers the following qualities: • Easier setup and administration of backup • Data deduplication, allowing more backup data to be retained on disk for longer periods • Data replication for cost-effective, off-site storage HP StoreOnce offers two options for NAS shares, i.e., as CIFS interface for Windows servers and as NFS interface for Linux and UNIX servers. NAS targets are recommended for customers in non-tape environments. NAS targets consist of one or more file shares that appear to the backup application as standard CIFS or NFS shares, while still supporting data deduplication and replication. The StoreOnce Backup systems should not be used for general purpose NAS operations such as file sharing. 14 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Each NAS share created on the StoreOnce system has its own deduplication “store”; that is, any data backed up to a share will be deduplicated against all of the other data in that store. There is no deduplication between different shares on the same StoreOnce appliance and there is no option to create non-deduplicating NAS shares. Key points Technically, HP StoreOnce can be used as a NAS to create a storage unit in NetBackup and HANA backups can also be written to it. NAS devices in general are known to be slower than FC and considering a high-speed and in-memory database like HANA, it may be better to use VTL or Catalyst, moreover, CIFS/NFS protocols tend to have higher overheads. HP StoreOnce can be mounted as a NAS device on the HANA server. File backups can be started on these NAS shares. A word of caution here would be to use a separate network interface for mounting a NAS share. If a NAS share is mounted using the data or user network interface, it could cause performance issues when a file backup is initiated. The advantage here is that NAS stores are very easy to set up and they can be handy in case of an emergency. An advantage of setting up a NAS share for backups would be the opportunity to save on the cost of setting up a separate SAN for backups. HP StoreOnce – replication HP StoreOnce replication, as shown earlier in Figure 7, enables customers to replicate their VTL, NAS or Catalyst Stores to another HP StoreOnce located at a remote site. This ensures that backups are available at the Disaster Recovery (DR) site in case the primary site is down. While the replication for VTL and NAS are configured with the StoreOnce web interface, the replication of data between Catalyst Stores can be configured and managed within the NetBackup application. Replication can take place between multi-node and single-node StoreOnce Backup systems. Replication provides a point-in-time “mirror” of the data on the source StoreOnce device at a target StoreOnce Backup system on another site, which enables quick recovery from a disaster that has resulted in the loss of both the original and backup versions of the data on the source site. Replication can also be configured to occur at specific times via configurable blackout windows in order to optimize bandwidth usage and not affect other applications that might be sharing the same WAN link. VTL replication allows multiple source libraries to replicate into a single target library; NAS mappings however are 1:1 and one replication target share may only receive data from a single replication source share. Another benefit of replication is that in case a virtual media is not available at the source site or has expired, it is possible to copy a virtual media from a secondary site to the primary site, providing an important backup source in the event of an emergency. Key points HP StoreOnce replication can be an alternative for tape backups. The positive side of having this kind of a replication in place would be faster restore from disks, thus a lower RTO. If the source and target StoreOnce devices are at remote sites, the WAN bandwidth needs to be considered for replication to be effective and would depend on how much of a change (delta) the HANA database is witnessing on a regular basis. HP StoreOnce replication can complement any SAP HANA host based system replication or DR/DT solution that is already in place. Through NetBackup OST integration, and the Catalyst plug-in, backup images on the HP StoreOnce device can be copied “locally” to a second HP StoreOnce device via Storage Lifecycle Policies (SLPs) and/or replicated to remote DR sites via Optimized Duplication (Opt_Dup) and/or “Automatic Image Replication” (AIR) to create multiple copies of the original backup image, for various purposes, all of which are cataloged and can be used for restore purposes. HP StoreOnce offers the capability to restrict bandwidth usage for replication; this is useful when the copy is going across WAN links such that the backup/replication traffic does not starve the other data traffic over WAN. Furthermore, blackout windows can be defined during which no replication jobs can run. 15 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup HP StoreOnce Enterprise Manager HP StoreOnce Enterprise Manager is a standalone software application that provides basic management capabilities in a replication environment and provides an easy way for customers to manage up to 400 StoreOnce devices that are being replicated across multiple sites. Using StoreOnce Enterprise Manager, administrators can create, manage, and remove groups of devices; and, establish and control user and administrator access to the groups and devices. StoreOnce Enterprise Manager allows administrators and users to review the status of the groups and devices to which they have access, either to ensure that replication is happening successfully and on schedule, or to respond when there are replication operation errors. StoreOnce Enterprise Manager also provides the ability to monitor disk usage and deduplication ratios for the devices. This allows administrators the ability to identify devices that may need additional capacity. Trend analysis is also available to look at disk capacity changes that occur on devices over a pre-determined period. It also provides command line interface (CLI), AD-enabled security and a topology viewer for the devices showing relationships and health. HP StoreOnce Enterprise Manager (SEM) can be used for remotely deploying, configuring and managing a StoreOnce Virtual Storage Appliance (VSA). SEM also provides the ability to manage licenses for HP StoreOnce VSA. StoreOnce Enterprise Manager allows building a historical database of health, disk usage, deduplication ratios and more; it has basic trending and prediction abilities to help decision making on infrastructure limits and expansion needs. An email-based digest is also supported to provide a summary of user chosen parameters on events and health. Figure 9 shows a screen shot of StoreOnce Enterprise Manager. There are several options to monitor and manage StoreOnce Backup devices. Figure 9. HP StoreOnce Enterprise Manager 16 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Key point For the purpose of this solution, HP StoreOnce Enterprise Manager may not be required because only one or two units of StoreOnce are targeted; however, it is an excellent tool to manage replication and as the customer environment grows, it will be very useful to manage all StoreOnce devices from a single console. HP StoreEver Tape Tape is still the most cost effective solution for long term data retention. If there is a need to keep data for years or even decades, tape is the answer. LTO tape media is specified with a 30 year shelf life in normal ambient conditions. HP has a complete range of tape-based backup products ranging from direct attached single drives to large enterprise class libraries that can have up to 96 tape drives and nearly 12000 media slots. While all of these libraries are compatible with Symantec NetBackup and any of these can be used for SAP HANA tape backups, an HP StoreEver MSL Tape Library was selected for this solution considering the backup data size and backup frequency. Table 2 provides details of the HP StoreEver MSL models and their respective features. Table 2. HP StoreEver MSL Tape Library Models Product description MSL2024 MSL4048 MSL6480 Drive Technology LTO-6 Ultrium 6250 LTO-6 Ultrium 6250 LTO-6 Ultrium 6250 LTO-5 Ultrium 3280 LTO-5 Ultrium 3280 LTO-5 Ultrium 3280 LTO-5 Ultrium 3000 LTO-5 Ultrium 3000 LTO-5 Ultrium 3000 LTO-4 Ultrium 1840 LTO-4 Ultrium 1840 LTO-4 Ultrium 1840 LTO-4 Ultrium 1760 LTO-4 Ultrium 1760 LTO-4 Ultrium 1760 Form factor 2U 4U 6U (per module) Maximum number of drives 1 (Ultrium 3280, 1840) or 2 (Ultrium 3280, 1840) or 6 HH or 3 FH (per Module) 2 (Ultrium 6250, 3000, 1760) 4 (Ultrium 6250, 3000, 1760) 42 HH or 21 FH (per Library) Number of slots 24 48 80-560 Maximum capacity 150 TB (LTO-6) 300 TB (LTO-6) 3.5 PB (LTO-6, 560 slots) (2.5:1 compression) Drive interface 500 TB (LTO-6, 80 slots) 8 Gb Native Fibre Channel 8 Gb Native Fibre Channel 8 Gb Native Fibre Channel 4 Gb Native Fibre Channel 4 Gb Native Fibre Channel 4 Gb Native Fibre Channel 6 Gb/sec SAS 6 Gb/sec SAS 6 Gb/sec SAS 3 Gb/sec SAS 3 Gb/sec SAS 3 Gb/sec SAS Ultra320 LVD SCSI Ultra320 LVD SCSI Disk-based backups will back up and restore quickly, but it may not be advisable to keep all backups within a single disk storage system for long term retention. Therefore, the recommended strategy is to back up to disks overnight and then to duplicate those backups to tape during the day. This also provides the backup administrator the much required flexibility to move backups to tape if the disk backup system is filling up. HP StoreEver MSL Tape Libraries include the ability to be configured as a partitioned library. Most MSL Libraries may be configured with up to one partition per drive; four partitions if four drives are installed. Another useful feature of HP StoreEver MSL libraries is that HP provides the MSL Library Extender Kit that can be used to connect two MSL libraries to create one larger, logical library. The stacked libraries appear to the host as a single library. The Extender Kit can be used with MSL2024 or 4048 libraries and is compatible with both new and currently installed libraries. HP StoreEver MSL6480 Tape Library is a modular mid-range library that provides scalability to the customer in terms of growing from one to seven library modules in a 42U form factor. HP StoreEver provides AES 256-bit embedded hardware encryption with compression; this enables organizations to achieve the highest level of encryption security to secure confidential business information. HP has two important tools that are very useful to manage StoreEver MSL Tape libraries; these are Library and Tape Tools (LTT) and TapeAssure. 17 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup HP LTT can be used to upgrade firmware of the library, generate support tickets, remotely access the front panel on the library, test the library for any hardware or software errors and report library performance. HP StoreEver TapeAssure Advanced Software provides an intuitive, easy-to-use dashboard and advanced analytics features for performance, health and utilization related to tape drives and cartridges. TapeAssure Advanced Software is well integrated into HP Command View TL Software for a single pane of glass management, monitoring and analytics. HP TapeAssure Advanced Software provides a dashboard display with pie charts, live and historical graphs for performance, health and utilization data, and highlights issues in the tape infrastructure. Advanced analytics features make use of predictive analytics to predict the likelihood of failures, bottlenecks and load balancing issues in the tape infrastructure. It has unique capabilities of analytics around the health and life of the drive and cartridge. Key points While a customer may do routine disk-based HANA database backups using HP StoreOnce, there may still be a need to move the backup images from a virtual media to a physical media for off-site/vaulting purposes. In some countries it is a statutory requirement to store critical data at an alternate location. SAP HANA database is home to the most critical business data for many organizations. When vaulting backup media to an alternate site there is a chance of media getting lost or stolen during transit, and vital company information could be compromised. HP StoreEver encryption is a very useful feature that can protect vital information from leaking even if media is lost. Encryption should be enabled on all media that is moving in and out of company premises. Partitioning allows each partitioned library to be presented to the host as an independent library, separate from other partitions in the library. Hosts for each partition can own independent sets of media. For SAP HANA, this capability allows for the provisioning of separate partitions for production and non-production instances, thereby enabling maximum flexibility and utilization of the MSL library. As mentioned above there is also the option to merge two physical libraries into one logical library. If the HANA environment has grown to a size that the existing capacity is no longer sufficient, the existing library can be scaled up by adding more drives if there are free slots available or by extending the existing library, by adding another tape library. HP StoreFabric SAN switches HP has a comprehensive portfolio of Storage Area Network infrastructure products and solutions that are designed to meet a range of storage networking needs. This solution uses HP StoreFabric B-Series SAN switches for FC links between the NetBackup media servers, HP StoreOnce Backup and HP StoreEver Tape Library. Table 3 shows the B-Series SAN switches available from HP. The switch model chosen for this solution is HP SN3000B top of rack SAN switch that runs at 16Gbps. Table 3. HP StoreFabric B-Series SAN Switch Models Product description SN3000B SN6000B SN6500B SN8000B 8-Slot/4-Slot Port bandwidth Up to 16Gbit/sec Up to 16Gbit/sec Up to 16Gbit/sec Up to 16Gbit/sec Form factor 1U 1U 2U 14U/9U FC ports 12 or 24 Enabled Max 24 24 or 48 Enabled 48 Max 48 or 96 Enabled 96 Max Up to 512/256 Aggregate device bandwidth 384 - 768Gbit/sec full duplex 768 - 1536Gbit/sec full duplex 1536-3072Gbit/sec full duplex 16.4Tbit/sec full duplex/ 8.2Tbit/sec full duplex Switch type Top of Rack Top of Rack Top of Rack Modular Although Table 3 shows B-Series (Brocade) SAN switches, if the customer has an existing C-Series (Cisco) or H-Series (QLogic) SAN in place, HP also has compatible SAN switch options in these series. 18 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Note HP StoreFabric SN3000B SAN switches come with 24 ports and since not all are initially used, there is enough scope for scaling the HANA backup infrastructure. These switches potentially guarantee high speed HANA backups, thereby ensuring a lower backup window. These switches are backward compatible, so they can be integrated within an existing SAN fabric that is already in use for backups. HP 5920 Series switch HP has a wide range of Ethernet switches that can, at the low end, cater to the requirements of a small office, and, at the high end, meet the requirements of very large data centers. The switch options include blade switches, top-of-rack (ToR) switches and large modular switches. These switches are designed keeping in mind the customer requirements and come with various features like FCoE, IPv6, 10GbE and jumbo frames. For the purpose of this solution a mid-range ToR switch was chosen to create a dedicated backup network between SAP HANA nodes, NetBackup master/media server and HP StoreOnce. The HP 5920 Switch series is made up of high-density 10GbE, ultra-deep packet buffering, ToR switches. These switches are part of the HP FlexNetwork architecture FlexFabric solution module and are ideally suited for deployments at the server access layer of large enterprise data centers. The HP 5920 Switch series is also designed for content delivery networks, with the heavy use of server virtualization, as well as bursty multimedia, storage applications, and other critical services. With the increase in virtualized applications and serverto-server traffic, businesses now require ToR switch innovations that will meet their needs for higher-performance server connectivity, convergence of Ethernet and storage traffic, the capability to handle virtual environments, and ultra-deep packet buffering all in a single device. Key points HP 5920 series switch is good for bursty traffic; it is an ideal choice for backup streams generated by SAP HANA and Symantec NetBackup. Since there is no option to install NetBackup media server software on an SAP HANA node to run direct SAN backups, this switch is a good choice as it provides the network bandwidth (10GbE) required by backups and it can enable high speed backups. The Workload data/results section shows that good backup throughput can be achieved on Catalyst Stores that run on these switches. Capacity and sizing Planning the capacity for backups is a complex process that is not just limited to availability of physical storage for storing backups, it also involves sizing the number of NetBackup media servers, the CPU/memory required on media servers and even sizing the SAN and Ethernet links. For databases in general and SAP HANA in particular, backups are very critical for the uptime of a database. These backups are important not just for recovery in case of a failure, but if database logs are not backed up in a timely manner they could fill up the logs filesystem and potentially bring down the database. This therefore merits proper capacity planning for HANA backups. The considerations for arriving at the backup capacity required for an HP AppSystem are the HANA database size, the backup frequency, and the change (delta) that a database is witnessing. In traditional backup devices, the sizing would be a simple calculation of the space required for each backup and the number of backups to be retained. Today’s advanced backup technology is able to share, compress, encrypt, multiplex and deduplicate the data that is going to a backup media. Hence sizing the capacity required for SAP HANA backups is a complex process and there are several factors that play a role in it. In the example below, the storage capacity requirements for the backup are based on the assumptions presented in the Suggested backup strategy section. As stated earlier a customer may have a higher backup frequency if they have a lower recovery time objective and vice-versa. What is presented below is a generic example; therefore, each customer’s actual storage requirements will differ accordingly from the example below. Another assumption taken below is a deduplication ratio of 10:1 (actual data size:size on disk). Although the HP StoreOnce is capable of delivering even better ratios as shown in the Deduplication efficiency testing section, this is a conservative estimate. The assumption for a scale-out system is a four node HANA cluster, each node with 512GB memory; the maximum database from four nodes would be approximately 2TB. With this assumption the storage calculation for a scale-out setup would be as below. 19 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Table 4. Storage and Tape capacity calculation for StoreOnce and StoreEver Disk Backups Scale-Up Scale-Out (4 nodes) Daily backup size (data) 1TB Per Backup 2TB Per Backup Daily backup size (logs) 50 GB (This assumes 100MB every five minutes for 24 hours. 288 log files will be generated, which adds up to 28,800MB or approximately 30GB. 50 GB is taken to be on the safer side.) 80 GB (This assumes 50MB per node every five minutes for 24 hours. 1152 log files will be generated, which adds up to 57,600MB or approximately 60GB. 80 GB is taken to be on the safer side.) Maximum number of data backups on the StoreOnce device at any point in time 7 daily, 5 weekly, 12 monthly, 7 yearly, 7 special 38 backups convert to 38TB 38 backups convert to 76TB 50 GB * 14 (Assuming two weeks’ worth of logs are kept on the disk) 80 GB * 14 (Assuming two weeks’ worth of logs are kept on the disk) 700GB 1120GB Total backup capacity required 38.7TB 77.1TB Total backup capacity required on disk assuming a deduplication ratio of 10:1 ~4TB ~8TB Available raw capacity 24TB (StoreOnce 4700 base unit) 60TB (StoreOnce 4900 base unit) Available usable capacity with RAID6 protection 20TB 36TB 1TB (Per backup) 2TB (Per backup) Maximum number of log backups on the StoreOnce device at any point in time Tape Backups Backup size (data) Number of backups Number of media required 31 (5 weekly, 12 monthly, 7 yearly, 7 special) 31 31 Scale-up The above calculation clearly shows that the HP StoreOnce 4700 is well suited for all HP AppSystems for SAP HANA that are in the range of 1TB of memory. A requirement of ~4TB seems too less versus the actual usable space available, i.e., 20TB; however, customers should consider all their HANA instances and calculate their total capacity required, this additional capacity would be useful for any future growth and any other applications that could share this StoreOnce unit. Even for customers that have a higher backup frequency and need more backup space, the StoreOnce 4700 has ample scope for expansion and can be scaled up to 192TB (raw) and 160TB (usable). Moreover, a Backup Tape library is also proposed for vaulting/offsite purposes; this library can come in handy to ease any temporary space crunch. It should be noted here that the above calculation is for one production HANA instance, for non-production instances such as QA, and development and testing, additional capacity needs to be calculated and procured. Scale-out Considering the above calculation for a HANA scale-out system, it seems that an HP StoreOnce 4900 with a usable capacity of 36TB would be higher than needed for the requirement of ~8TB. However, as mentioned above for scale-up, customers should capture their complete backup requirement for all HANA instances and other applications to arrive at their required capacity. Customers should also consider a StoreOnce 6500 if their added capacity requirements merit going beyond StoreOnce 4900 or if they are looking for highly available backup storage. StoreOnce 6500 is designed in a way that it has no single point of failure and also supports autonomic failover for Symantec NetBackup, the key benefit is that backups will continue even if a node in a couplet fails. Capacity planning on tape would be fairly simple; since these tapes are meant to be sent to an offsite location, they will generally contain only one full backup. LTO6 tapes have a capacity of up to 6.25TB and can hold a complete backup of all 20 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup scale-up and scale-out appliances. Tape media provides flexibility, as compared to disk based solutions, in terms of the ability to procure additional media as and when required. This helps to ramp up capacity in a very short span of time. HP StoreEver MSL2024 supports up to two backup drives, since the frequency of tape backup is not very high, customers may initially start with just one drive and populate the second drive later if required. This may have to be reconsidered if there are more than one HANA instances running and they also need backups to be sent offsite. If this library is on a shared SAN and there are other applications that need tape backups, customers should take a holistic view and consider total capacity required. For HANA scale-out appliances the customers should start with a two drive unit, if the scale-out has more than four nodes the StoreEver MSL4048 (four drive model) should also be considered. Workload description HP has done extensive testing to arrive at the achievable backup and restore performance of NetBackup for an SAP HANA database environment using HP StoreOnce and HP StoreEver. Figure 10 shows the HP lab test setup. The Ethernet links from the HANA server to the NetBackup master/media server were running at 10Gbps and the SAN links from NetBackup master/media server were running at 16Gbps. The SAN links from the StoreOnce 4430, StoreOnce B6200 and StoreEver MSL2024 were running at 8Gbps in this setup. The backup data flow from the HANA node to the NetBackup media server is always through Ethernet switches; the data movement from the NetBackup media server to the backup device depends on the kind of physical or virtual backup device in use. For example if the backup device is a VTL or an MSL then the data moves over SAN switches and if the backup device is a Catalyst Store or a NAS share the data moves over Ethernet switches. A bill of materials listing all of the components used in the tested configuration can be found in Appendix 1. Figure 10. Test lab setup 21 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup • Throughput rate The testing was primarily aimed towards measuring the achievable throughput. As there are a range of components in an Enterprise backup environment, such as the SAN, Ethernet, HBAs, NICs, NetBackup media server, and backup devices like StoreOnce and StoreEver, running a backup or restore just once may not give a realistic picture when one of the components may be busy leading to a higher or lower throughput. Thus, trending of multiple runs of backup and restores of a HANA database were done to arrive at the indicative numbers. Workload data/results – backup performance Scale-up Several tables were inserted in the HANA database to increase the database size. Since the DL980 based appliance supports up to 1TB of data, backups were tested with a database size of 1TB. Table 5 shows a comparison of the time taken and the average backup throughput during a backup run, on a Catalyst store a VTL and a MSL tape library. To arrive at the Catalyst performance, version 2.1 of the HP OST plugin was used and the same tests were repeated on the HANA database, except this time an OST storage unit was used instead of the VTL by replacing the storage unit in the NetBackup policy. Table 5 shows the time taken to successfully backup approximately 1TB of data using Catalyst Stores via OST. Table 5 also shows the characterization of backup performance on an HP StoreEver MSL2024 by running a 1TB HANA backup on LTO6 media. Table 5. HANA backup performance Catalyst vs. VTL vs. MSL on scale-up Catalyst VTL MSL Time taken (HH:MM) Average throughput (KB/s) Time taken (HH:MM) Average throughput (KB/s) Time taken (HH:MM) Average throughput (KB/s) 01:05 268,677 01:10 252,429 01:49 162,806 01:00 291,430 01:10 253,806 01:57 151,516 01:00 290,660 01:09 256,214 01:50 160,110 01:00 294,085 01:10 253,523 01:50 160,793 01:00 296,053 01:10 253,616 01:52 156,776 Scale-out As shown in Figure 10, a four node AppSystem for HANA Scale-out was used to test and measure the achievable throughput and the time taken to back up a 1.6TB HANA database. The HP ProLiant BL680c G7 servers each had 512GB RAM, so each node was grown to 80% of its memory capacity, i.e., 400GB. Table 6. HANA backup performance Catalyst vs. VTL on scale-out Catalyst (OST) 22 VTL Time taken (HH:MM) Avg throughput (KB/s) Time taken (HH:MM) Avg throughput (KB/s) 00:44 161,176 00:42 171,456 00:42 168,677 00:35 204,125 00:42 170,650 00:35 201,714 00:44 161,921 00:35 201,429 00:44 158,808 00:36 196,872 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Observations • On scale-up, using VTL, a backup throughput rate of 250+MB/s was achieved and backup of a HANA database above 1TB was completed in about 1 hour and 10 minutes. Although a StoreOnce is capable of achieving much higher throughput, the structure of a HANA database and its services are such that unless a multinode scale-out HANA installation is used the number of streams cannot be increased to increase the backup throughput. • On scale-up using Catalyst (OST) a backup throughput rate of 250+MB/s was achieved and a backup of a complete 1TB HANA database was achieved in about 1 hour. • In case of HANA scale-up, HP StoreOnce Catalyst (OST) store performed better than HP StoreOnce VTL and achieved slightly higher throughput rate. • MSL consistently performed above 150MB/s for HANA backups. • MSL should be used as a secondary backup device. • MSL should not be used for backing up SAP HANA logs. • MSL may be used to create a second copy of data backups by using NetBackup storage lifecycle policies. • In case of HANA scale-out backups on VTL and Catalyst, multiple backup streams for all nodes ran in parallel and we could complete the backup of a complete 1.6TB database in about 44 minutes on a Catalyst Store and 35 to 40 minutes on a VTL. Considering the high volume of data, backing up in less than an hour is a very high backup speed. Workload data/results – restore performance Scale-up While backups and backup speed is important and backups should be able to complete in a reasonable amount of time, it is also very important that the restore of the database is quick and smooth. HP tested the restoration of a HANA database using the SAP backint API. The restore for an SAP HANA instance can be initiated from SAP HANA Studio. The SAP HANA Studio recover wizard guides a user on starting a restore. A restore can be from flat file backups or from a backint backup. If the requirement is to restore from a NetBackup backup image then backint should be selected. A backint restore creates 5 data restore jobs and several log restore jobs in NetBackup. Except for HDB Indexserver all of these jobs are very small and complete in a few minutes. Each node in a HANA scale-out environment creates one Indexserver job. The backup corresponding to Indexserver takes time depending upon the volume of data in the database. The restore was tested on HP StoreOnce emulated in Catalyst mode and VTL mode and from an HP StoreEver MSL2024 library. The method deployed here was again to do trending of multiple runs of HANA database restores from a physical tape library. This testing is also important as it can help to calculate the recovery time objective that a customer can expect with their backup infrastructure. Table 7 shows restore performance of restoring a 1TB HANA database from a StoreOnce VTL, a StoreOnce Catalyst Store and a StoreEver MSL LTOx tape library. Table 7. Restore Performance Catalyst, VTL and MSL2024 for 1TB HDB Catalyst (OST) VTL MSL2024 Time taken (HH:MM) Avg throughput (KB/s) Time taken (HH:MM) Avg throughput (KB/s) Time taken (HH:MM) Avg throughput (KB/s) 01:17 244,762 01:48 162,980 02:25 120,794 01:18 264,626 01:52 157,750 02:23 122,746 01:13 257,180 01:53 155,488 02:27 120,423 01:14 262,755 01:51 159,135 02:31 116,566 01:16 244,869 01:49 161,067 02:26 120,341 23 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Scale-out Similar to backup performance testing presented in Table 7, restore performance was measured for a 1.6TB database running on four nodes. The testing was carried out on Catalyst Store and VTL devices. Since all HANA nodes carried equal data, the time taken and average throughput numbers are a simple average of the time taken by each node and the throughput of each node to arrive at the aggregate number for a scale-out system. Table 8. Restore Performance Catalyst vs. VTL for 1.6 TB HDB Catalyst (OST) VTL Time taken (HH:MM) Avg throughput (KB/s) Time taken (HH:MM) Avg throughput (KB/s) 00:39 180,806 00:43 165,646 00:43 162,400 00:43 164,842 00:42 168,686 00:43 164,383 00:42 168,158 00:43 165,815 00:41 173,387 00:42 165,684 Observations • For a scale-out, Catalyst (OST) stores performed above 240MB/s, VTL performed above 150MB/s, and MSL2024 performed consistently above 110MB/s on restores. Considering the volume of data restored, the restore timings shown in Table 7 can guarantee lower RTO for customers. • The restore from a Catalyst (OST) performed much better than VTL and MSL. • The restore performance for scale-up HANA from a Catalyst Store was 88% as compared to its backup performance. This shows that restores from a Catalyst could be as fast as backups. • The restore performance from a VTL was 62% as compared to its backup performance. This is due to the processing necessary to rehydrate deduplicated data. • The restore performance from a MSL2024 was 75% as compared to its backup performance. • On a HANA scale-out system, a complete 1.6TB database running on four nodes could be recovered in about 40-45 minutes using both Catalyst Store as well as VTL. Scalability testing The objective of scalability testing was to observe the solution behavior as the environment scales. HP AppSystems for SAP HANA are available in scale-out configurations that can scale-up to 16 HANA nodes, thus it is important that the solution performs to acceptable levels. To test this initially a one node HANA was built and populated with ~400GB of data. The backup and restore testing for this single node was done on Catalyst Store and VTL drives. After this, one more node was added and populated with an additional 400GB of data. All the tests performed with one node were repeated with two nodes and the performance numbers were recorded. The same tests were repeated for three nodes and four nodes. Table 9 and Table 10 present the results of scalability testing. Table 9. Scalability testing on Catalyst Nodes 24 Backup Restore Rate/Hour (GB) Rate/Hour/Node (GB) Rate/Hour (GB) Rate/Hour/Node (GB) One 857 857 887 887 Two 1468 734 1513 756 Three 1949 650 2094 698 Four 2351 588 2620 655 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Table 10. Scalability testing on VTL Nodes Backup Restore Rate/Hour (GB) Rate/Hour/Node (GB) Rate/Hour (GB) Rate/Hour/Node (GB) One 887 887 671 671 Two 1610 805 1248 624 Three 2094 698 1852 617 Four 2654 663 2420 605 Observations • In Tables 9 and 10, we can see that although the average performance per node was reducing with every node added, the aggregated performance kept on increasing. This shows that HP StoreOnce is ideal for backing up a HANA database running on a scale-out environment. • The deterioration in backup speed per node was faster in Catalyst Stores based backups as compared to VTL, this is primarily due to the fact that the VTL runs on SAN and Catalyst Stores run on Ethernet. Deduplication efficiency testing Deduplication is the ability of the storage system or the target backup appliance to identify repetitive patterns from a backup stream and save on storage consumption by not writing those patterns again. The challenge in testing deduplication is that it is dependent on how much change a database is witnessing between backups. In general databases witness a change rate of 1- 3% on a daily basis. Taking this assumption, testing was performed by changing the database 1% between each backup run and recording the deduplication ratios achieved. Similarly deduplication performance was also recorded by changing the database 3% between each backup run. To ensure the numbers were stable, the tests were repeated fourteen times and the numbers were recorded at the end of each run. Table 11 presents the results of deduplication testing performed using the above described procedure. Table 11. Deduplication performance testing Iteration Deduplication ratio with 1% Database change 3% Database change 1 3.262 3.125 2 6.157 5.927 3 8.862 8.480 4 11.366 10.810 5 13.681 12.942 6 15.837 14.903 7 17.855 16.700 8 19.738 18.334 9 21.510 19.905 10 23.174 21.349 11 24.744 22.674 12 26.218 23.966 13 27.597 25.151 14 28.907 26.270 25 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Observations • It was observed that the deduplication ratios kept on improving slowly with every subsequent backup of the database, however customers should expect the dedupe ratios to stabilize at a point between 20 and 30 based on their change rates. • It may be noted here that although these numbers are as close as possible to a real production database, a customer may not get exactly the same numbers as their change rates vary; moreover, if a customer executes a special process like month-end or quarter-end closing process the change rate would be higher and the resulting deduplication ratios may vary. General recommendations • The inline nature of the deduplication process means that it is a very processor and memory intensive task. HP StoreOnce appliances have been designed with appropriate processing power and memory to minimize the backup performance impact of deduplication. Best performance will be obtained by configuring a larger number of libraries/shares/Catalyst Stores with multiple backup streams to each device, although this has a trade off with overall deduplication ratio. For SAP HANA separate VTLs or Catalyst Stores may be created for each instance and even within an instance it may be better to have separate storage units for data and logs. • If servers with very similar data are to be backed up, a higher deduplication ratio can be achieved by backing them all up to the same library/share/Catalyst Store, even if this means directing different media servers to the same data type device configured on the StoreOnce appliance. In case of a HANA Scale-out, if all the HANA nodes back up to the same Catalyst Store, better deduplication ratios can be expected. • If servers contain dissimilar data types, the best deduplication ratio/performance compromise will be achieved by grouping servers with similar data types together into their own dedicated libraries/shares/Catalyst Stores. For example, a requirement to back up a set of Microsoft Exchange servers, SQL database servers, file servers and application servers would be best served by creating four virtual tape libraries, NAS shares or Catalyst Stores; one for each server data type. • The best backup performance to a device configured on a StoreOnce appliance is achieved using less than the maximum number of streams per device (the maximum number of streams varies between models). • When restoring data from a deduplicating device, it must reconstruct the original un-deduplicated data stream from all of the data chunks contained in the deduplication stores. This can result in lower performance than that of the backup process (typically 80%). Restores also typically use only a single stream. • Full backup jobs will result in higher deduplication ratios and better restore performance. • Activate Email Alert on the StoreOnce to communicate in a timely manner if it is nearing the storage full condition. • If using VTL emulation in StoreOnce, use LTO5 emulation as it provides a capacity of 1.5TB and can fit a complete HANA database backup in a single media. This is compatible with MSL2024 which also supports LTO5. • Virtual library devices are assigned to an individual interface. Therefore, for best performance, configure both FC ports and balance the virtual devices across both interfaces to ensure that one link is not saturated while the other is idle. • If using scheduled HANA backups using a Symantec script make sure backups do not overlap. • Deduplication needs to always occur before compression and/or encryption. NetBackup policies have an option to enable encryption and compression; and, enabling NetBackup compression and encryption before HP StoreOnce deduplication is performed will randomize the source data and will not result in a high deduplication ratio for these data sources. Consequently, performance will also suffer. The StoreOnce Backup system will compress the data at the end of deduplication processing anyway, before finally writing the data to disk. For these reasons it is better to disable encryption and compression from the NetBackup settings for both data and log policies. • If there are multiple HANA instances running on a single node, the backup throughput rate would be lower per instance, although the added throughput to a StoreOnce may be higher. • SAP HANA database can only be recovered as a whole and it is not possible to recover a table or an object. The recovery process requires the database to be down, and online recovery is not possible in this condition. • SAP recommends that in addition to the data and logs there should also be a backup of the configuration files on the HANA nodes. These files should be backed up before and after an upgrade of the HANA database or any system changes are executed. • Enable the log backups to initiate automatically rather than manually, this will ensure the logs space is always available and prevent the database from going down due to filesystem full condition. • While successful backup is very important, it is also very important to ensure that the backup images are expiring in a timely manner, this indirectly ensures that there is sufficient capacity available on the backup device for routine future backups. 26 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup • FC devices should be zoned on the StoreFabric switch to be accessible only from a single backup server device. This ensures that other SAN events, such as the addition and removal of other FC devices, do not cause unnecessary traffic to be sent to devices. It also ensures that SAN polling applications cannot reduce the performance of individual devices. • StoreOnce does not implement any selective virtual device presentation, and so each virtual library will be visible to all hosts connected to the same fabric. It is recommended that each virtual library is zoned to be visible to only the hosts that require access. • Although one StoreFabric switch with enough ports can be used for StoreOnce and all the NetBackup media servers, it is recommended to have dual fabrics for high availability and load-balancing reasons. • Cartridge capacities should be set either to allow a full HANA backup to fit on one cartridge or to match the physical tape size for offload (whichever is smaller). Effectively this means that virtual media and physical media should be of the same emulation and size. • Use replication blackout windows to avoid replication overlap with backup operations. Use bandwidth throttling when necessary to prevent oversubscription of the WAN link for replication. • With VTL replication and Catalyst Copy several sources (VTL slots or Catalyst Stores) can be replicated into a single Target device (VTL or Catalyst Store) – this provides the option of consolidation in a single target for easier administration at the target site. It also provides a single disaster recovery pool. NAS source and replication targets have a 1:1 mapping only. Use this consolidation approach when there is similar data on source sites that lends itself to consolidation at a central site. • If HANA data and log backups are going to the same physical or virtual library, NetBackup media volume pools may be deployed to segregate the backups from mixing. Separate volume pools created for data and logs can be attached to their respective policies. • If using storage unit groups, mixing different storage types within a group is not advisable. A storage unit group should have all storage units that are either Catalyst Stores or VTL or physical tape drives. • Using Storage unit group could adversely affect the deduplication performance of HANA backups. This is because the Catalyst Stores deduplicate from within the backups being written to it, if the same backup goes to a different store randomly the deduplication would not perform well. • If using VTLs the number of drives should be more than the number of SAP HANA nodes in a scale-out environment, this will help to ensure that all the backup jobs are able to run in parallel and they are not waiting for other jobs to release the drives. Configuration guidance Enabling HANA backups using Symantec NetBackup requires configuring HP StoreFabric, HP 5920 Series switch, HP StoreOnce, HP StoreEver, HP ProLiant DL380p server, SAP HANA Studio and Symantec NetBackup. The next few subsections briefly explain the configuration steps required in each of these components. HP StoreFabric configuration HP StoreFabric SAN switches need to be physically connected via FC cables to HP StoreOnce, HP StoreEver and HP ProLiant DL380p server; this server is eventually going to become a Symantec NetBackup media server. SAN links should be done in such a way that the SAN is highly available and can sustain single path failure. The StoreOnce appears as an NPIV device on the switch and each VTL has its distinct WWNs. SAN zoning should permit the media server to see the devices from HP StoreOnce/StoreEver as applicable. Zoning should be such that only the media server that is required to see a VTL should be zoned with it. HP 5920 Series switch configuration The HP 5920 Series switch needs to be physically connected to the HANA scale-up appliance, and the NetBackup master and media server, i.e., HP ProLiant DL380p and HP StoreOnce 4700 or HP StoreOnce 4900/6500. The first configuration step on this switch is to assign a management IP to this switch. Once the management IP has been assigned, the switch should be accessible through a browser or SSH session. The second configuration step on this switch requires creating a separate VLAN for all ports participating in the backup network. HP ProLiant DL380p server configuration This server is going to be the NetBackup master and media server. The configuration steps include installing Microsoft Windows Server 2008 R2. For the dedicated backup network, this server’s HP FlexFabric 10GbE ports should be configured to connect to the separate VLAN created in the previous step on the HP 5920 Series switch. To access the Catalyst Stores, the HP OST plugin has to be installed in this server. Further steps to install and configure Symantec NetBackup are explained in a separate section below. 27 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup HP StoreOnce configuration HP StoreOnce can be configured in two ways; it can be configured as a VTL or as a StoreOnce Catalyst OST device. Both of these eventually become storage units in NetBackup. Although HP StoreOnce 4700/4900/6500 VTL can run on Fibre Channel and iSCSI, HP recommends running over FC for better performance. Currently the Catalyst (OST) stores only run on Ethernet. Some of the considerations while creating a VTL follow: • The HANA database size. Selecting an emulation that is higher than the data size would ensure accommodation of one full backup in a single virtual media. • Existing physical tape library, if appropriate. If there is an LTO4 Drive or Library, then an LTO4 type library can be emulated within a StoreOnce. • Number of drives and media to be created. This would depend on the HANA database size, the backup frequency and the volume of backup logs getting generated. If multiple HANA instances or other applications are sharing a StoreOnce, multiple VTLs can be created for each application. This could have a performance impact on the speed of HANA backups. Having the <SID> within the VTL name would help to ensure easier identification when creating the NetBackup storage unit. Before configuring a NetBackup disk pool from a Catalyst Store, the latest HP OST Plugin for Symantec NetBackup (version 2.1 or above) must be installed. The HP OST plug-in used for our testing was version 2.1. The HP OST plug-in needs to be installed on all NetBackup media servers that are going to have a Catalyst Store as a storage unit. Post HP OST plug-in installation, a NetBackup Storage Server for HP StoreOnce must be created and the Catalyst Stores discovered. Each Catalyst Store first becomes a disk pool and eventually a storage unit in NetBackup. Some of the considerations while creating a Catalyst Store follow: • The number and size of Catalyst Stores. • Mode of operation. Mode may be low or high bandwidth; the performance of HANA backups has been tested to be better on high bandwidth mode. It may be better to create separate Catalyst Stores for each SAP or SAP HANA <SID>. Having the <SID> within the Catalyst Store name would help to ensure easier identification when creating the NetBackup disk pool and NetBackup storage unit. HP StoreEver configuration A management IP can be configured from the front panel on the library. The FC model of the library selected for this solution should be physically connected to the two StoreFabric switches and zoned to the NetBackup media servers. • HP StoreEver configuration requires the relevant drivers be installed on the NetBackup media server. Further configuration is required on NetBackup to create a storage unit. NetBackup Device configuration wizard may be used to discover and create a storage unit. • Once a storage unit has been created, it can be selected in a weekly backup policy or any policy that intends to write backups to a tape media. • HP StoreEver retains the familiar and easy-to-use HP Command View interface for administration and management. The interface can be accessed through a browser and all configuration and administration tasks can be done from here. Figure 11 shows Command View MSL, which is used to manage the MSL Library. Several administration and management tasks can be executed from here, such as moving a media, generating a support ticket, creating a user, checking the library status, and ejecting media from the library. 28 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Figure 11. Command View MSL SAP HANA Studio A prerequisite for configuring the backups from the HANA Studio is to add the HANA SID in HANA Studio. This is a simple process that requires the IP/Hostname of the HANA node (IP/Hostname of the master HANA node in case of a scale-out solution), the HANA database SID and Instance ID, and the port# to connect to the HANA database. Once HANA Studio is able to talk to the HANA database, the steps listed below must be followed to configure backint backups. None of the steps below are necessary for file backups. Backup sub-header can be accessed on the HANA Studio by traversing this path: HANA Instance Administration Console Configuration global.ini [Backup] The key parameters to be specified in the global.ini file include: DATA_BACKUP_PARAMETER_FILE – This parameter is used to specify the NetBackup policy to be used for backing up the data files. The complete path and filename of init<SID>.utl file must be provided here. This file contains the policy name for data files. The default path is /usr/openv/netbackup/ext/db_ext/sap/scripts/sap_oracle/initSAP.utl. The policy configured in NetBackup MUST have this same name. LOG_BACKUP_PARAMETER_FILE – This parameter is used to specify the NetBackup policy to be used for backing up the log files. If the target storage unit for a data files policy is different from the log files then, a different log backup parameter file containing a different policy for log backups must be specified. The default path is /usr/openv/netbackup/ext/db_ext/sap/scripts/sap_oracle/initSAP.utl LOG_BACKUP_USING_BACKINT – In general the data and logs both go to NetBackup storage units through backint. If this is set to TRUE then each log generated creates a NetBackup backup job and goes to a NetBackup storage unit. In case the logs are being sent to a local filesystem or NAS mount point and it is not required to be sent to NetBackup storage units, then “FALSE” must be set here. In most case this should be set to TRUE. For file backups we only need to specify the backup destination (folder or mount point) and backup prefix. The default backup prefix is COMPLETE_DATA_BACKUP, the backup admin may change this as appropriate. Symantec NetBackup configuration A prerequisite for this step is that NetBackup Master and Media server software (version 7.5 or above) is installed on the HP ProLiant DL380p server mentioned earlier. The NetBackup master server and NetBackup media server only need to have version 7.5 or above, while the NetBackup client software installed on each SAP HANA node needs to have version 7.5.0.6 or above. The steps to configure NetBackup follow. Create storage unit The VTLs and OST devices created on an HP StoreOnce and StoreEver have to be discovered on a NetBackup media server. A prerequisite for this is that the SAN links should be in place and SAN zoning should permit the media server to see the devices from HP StoreOnce / StoreEver as applicable. Another pre-requisite for configuring a VTL / StoreEver is to install the relevant drivers on the media server. For example, if emulating an MSL library from a StoreOnce device, the drivers for the device have to be installed. 29 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup For configuring a Catalyst Store, a Catalyst license should be installed on the StoreOnce. The latest HP OST Plugin for Symantec NetBackup also has to be installed on the media server. Configuration steps on the NetBackup side include creation of Disk Storage Server (Wizard available). Table 12 shows parameters to create a NetBackup storage server for HP StoreOnce. Table 12. Parameters to create a NetBackup storage server for HP StoreOnce Parameter Response Type of disk storage OpenStorage Storage server name Management IP or Hostname of StoreOnce Storage server type hp-StoreOnceCatalyst Credentials StoreOnce Admin and password Once the storage server has been created in NetBackup, the Catalyst Stores created in StoreOnce Web Management GUI should be visible in NetBackup. The next step is to create of a disk pool (Wizard available) for the Catalyst Store and create a storage unit (Wizard available) for the disk pool. Create policy A NetBackup policy is a set of rules and resources that are allocated to a set of similar hosts. For example, all SAP HANA servers can be added to the same policy to inherit the same storage unit and schedule. A policy should be created to map the storage unit with a HANA instance. The policy created for HANA backup data and log backups should have the policy type “SAP.” The policy should have a schedule “Default Application Backup” and include the HANA servers as a client in the clients tab, in case of a multimode HANA scale-out setup all the HANA nodes, should be included in the clients tab. The policy should point to one of the applicable storage units based on Catalyst, VTL or a physical drive within the MSL Library. If HANA backups are to be scheduled using the scheduling script, then it is required to create a separate policy for each instance. The reason for this requirement is that scheduled backups call a script that is instance specific. The policy type for backups of configuration files should be “Standard.” The path selection for this follows: • For global configuration settings: $DIR_INSTANCE/.. /SYS/global/hdb/custom/config • For host-specific configuration settings: $SAP_RETRIEVAL_PATH Backup schedule SAP does not provide an option to schedule a backup and the default method to initiate a data backup is to manually initiate it within HANA Studio. Manually initiating backups may be tedious and error prone, hence Symantec has come up with a script that can be integrated with a NetBackup data backup policy to initiate backups automatically at a scheduled time and frequency. Configuring this script involves updating the script with the HANA instance name, instance ID and database user ID, and password on the script. A new schedule can be added to the existing HANA data backup policy or a new policy could be created. The new schedule has to be of the type “Automatic-Full-Backup” and should have this script with full path in the NetBackup policy selection tab. Create storage unit groups A storage unit group is optional when configuring backups for a scale-up HANA appliance. However, in scale-out appliances and especially when deploying more than one NetBackup media server, a NetBackup storage unit needs to be created on each media server and then add all the storage units created in a storage unit group and assign this storage unit group to a data or log backup policy. It is better to keep similar storage units in a group. Dissimilar storage units like VTL and OST should not be mixed. Depending on the backup strategy, three storage unit groups may be created for Catalyst, VTL and physical tape library. Even if a backup device is shared across multiple media servers, storage units from all those media servers should be added to the same storage unit group. This will ensure that if a particular media server is down or a storage unit is down, the backup would not fail; secondly all backup jobs would automatically be spread across all media servers in a round robin or media server load-balancing fashion. Storage unit groups may be named in a way to identify the HANA instance it is serving or whether it is serving production or non-production instances. The naming convention may also signify the type of storage units in the group like Catalyst, VTL or Tape. 30 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup HANA node configuration A pre-requisite to HANA node configuration is to install a 10GbE NIC as per the Bill of materials Set 1. This is required to create a separate backup VLAN. This is required only on HP AppSystems of SAP HANA based on HP ProLiant DL580 G7 servers. This is not required on HP AppSystem based on HP ProLiant DL980 G7 or ConvergedSystem based on HP ProLiant DL580 Gen8 as these appliances will be preloaded with a dedicated NIC for creating a backup network. HANA node configuration requires installing a third party software on the SAP HANA node; it is therefore recommended to go through SAP’s policy on what is allowed and what is not allowed to be installed on an SAP HANA node. The relevant SAP notes on SAP’s policy are as below; these can be accessed using an SAP provided S-User ID: 1730928 - Using external software in a HANA appliance 1730929 - Using external tools in an SAP HANA appliance 1730932 - Using backup tools with Backint for HANA 1913568 - Support Process for Symantec's NetBackup As shown earlier in Figure 3, since the NetBackup HANA Agent gets installed on the HANA node, a NetBackup client and the HANA Agent have to be configured on the HANA node. The steps to configure the HANA node follow. Installation and configuration of NetBackup Client for Linux NetBackup client can be downloaded from symantec.com and installed on all HANA nodes. The NetBackup master server and all the media servers should be entered in the /usr/openv/netbackup/bp.conf on all HANA clients. Ensure all hostnames of master/media servers are resolving seamlessly either using DNS or a local entry in /etc/hosts file. Create a soft link for backint ln –s /usr/openv/netbackup/bin/hdbbackint_script /usr/sap/<SID>/SYS/global/hdb/opt/hdbbackint. This softlink is internally used at the time of backup. Create nodename.txt file The nodename.txt file needs to be created under /usr/openv/netbackup/ext/db_ext/sap on each HANA node. This file should contain the node names of all HANA nodes, including any nodes to be used for High Availability (HA) of SAP HANA. If there are multiple HANA nodes in a scale-out HANA environment, the names of all the nodes should be included in this file. Create init<SID>.utl file A template of the init<SID>.utl file can be found at /usr/openv/netbackup/ext/db_ext/sap/scripts/sap_oracle/ The file name of this template file is initSAP.utl. This file should be copied as init<SID>.utl. Once copied, the init<SID>.utl file should have the “policy” line uncommented, and the name of the NetBackup policy specified as the corresponding policy value. This policy value must match the SAP HANA policy name within NetBackup. If it is desirable to have a separate NetBackup storage unit for logs then another init<SID>.utl file must be created and the NetBackup logs policy name must be specified in this file. This file also allows the administrator to change several other parameters to tune the HANA backups. Some of the parameters include number of drives, retry_backup, fail_backup, restore_filter, bplist_filter, backup_stream_buffersize, and restore_stream_buffersize. Although there are many more such parameters in this file, it is better to leave them untouched unless there is a specific need to do so. HP StoreOnce configuration – replication Configuring storage based replication is an optional step that is applicable only for those customers that have more than one HP StoreOnce and desire to have a replication of the virtual tape media or the backup images in their Catalyst Store replicated to another Catalyst Store at a DR site. A prerequisite for configuring replication is to have network connectivity between the two StoreOnce devices as shown in Figure 7. Replication can be achieved in two ways; for VTL and NAS stores it can be configured from within StoreOnce Web Management GUI and for Catalyst Stores this has to be configured from within NetBackup Administration Console. Virtual tape: The replication can be configured using the Wizard available in the StoreOnce Management interface to create VT mappings. The wizard has three steps, i.e., Select Target Appliance, Select Target Library and Edit Slot mappings. In the first step the IP and port details of the remote StoreOnce must be provided. In the second step, the wizard prompts to select a VTL on the target StoreOnce; there is also an option to create a new VTL on the target if required. Move forward to the third step to create a mapping name and select the virtual media from the source to virtual media on the target, this is done 31 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup using the media slot numbers on the source and mapping them to a slot on the target. The VTL names and the mapping names here may be selected in a way that it is easy to co-relate the source library and the target library. Figure 12 gives an idea of how replication should look once it is configured and synchronized. Key parameters to check the status of a replication configuration are source and target library names, synchronization status of the library, i.e., synchronized or pending, and synchronization status of media. If synchronization is pending then it also shows how many hours since the last sync. Figure 12. VTL replication mapping NAS Shares: If used for backups, NAS shares can also be configured for replication. The process is to follow a simple and intuitive configuration wizard that is very similar to the wizard we described above for VTL. Figure 13 shows a sample NAS replication that has been configured on a primary and secondary StoreOnce device; it may also be noted that both the shares are in sync. Figure 13. NAS Replication mapping 32 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Catalyst Stores: Replication on Catalyst Stores can be configured, initiated and managed from within NetBackup. A prerequisite to Catalyst replication is to have two or more StoreOnce configured as storage servers in NetBackup. Each of these should also have a disk pool and storage units created. The process to configure replication for Catalyst Stores involves creation of a NetBackup Storage Lifecycle Policy (SLP) and assigning this as a storage unit to a HANA Data Backup policy. Within an SLP, there is an option to create multiple operations like Backup, Duplicate, Import and Snapshot, and the option to associate a distinct storage unit and a retention period for each of these operations. Each SLP could be of the type Platinum, Gold, Silver or Bronze. These storage lifecycle policies can be assigned to a HANA data or log backup policy, based on the kind of data that is being backed up and the desired retention cycle for each copy of data. Figure 14 shows how an SLP can be created. Figure 14. NetBackup Storage Lifecycle Policy creation 33 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Suggested backup strategy A suggested backup strategy would be to take one database backup daily on a disk-based backup device like VTL or Catalyst. For offsite and vaulting purposes, one backup should be taken on a physical tape device every week. Table 13 suggests a backup schedule, backup storage unit and backup retention duration. This strategy assumes that a customer has a backup network and also a SAN in place; however, if the customer does not have one of these then this strategy may have to be revised accordingly. If the VTL option is not available in the absence of a SAN, all VTL based backups will have to be moved to Catalyst, similarly if the Catalyst option is unavailable due to lack of Catalyst license, Catalyst based backups will move to VTL. Table 13. Suggested backup strategy NetBackup policy Disk-based backup – Catalyst Disk-based backup – VTL Tape-based backup Retention HANA_SID_Data_Daily Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday 7 days HANA_SID_Shared_Daily Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday 7 days HANA_SID_Data_Weekly Saturday Saturday 4-5 weeks, depending on the number of Saturdays in the month. HANA_SID_Data_Monthly Last working day of the month Last working day of every month 12 Months HANA_SID_Logs Every 15 minutes or earlier 14 days, Point in time restores have a dependency on logs having a retention of 14 days means that we can restore a database to a point in time in the last 14 days. HANA_SID_Config_files Weekly One month, take special backup before and after system configuration changes. Key points The recommendations in Table 13 may be adjusted by organizations, based on the data volume, the rate at which the data is changing, the recovery time and recovery point objectives of the organization, and also the statutory requirements to retain data applicable from country to country. NetBackup storage lifecycle policy may be created for duplicating the daily backup to a weekly backup and also to a monthly tape as applicable. When duplicating to a weekly/monthly backup, the storage unit should be physical tape. The frequency of log backups may be adjusted considering the delta on the HANA database. For non-production instances, it may not be necessary to take tape backups. In Table 13 we have suggested using the policy HANA_SID_Config_files for backing up the config files. The policy type of the HANA_SID_Config_files should be “standard.” Customers are free to use the policy names as per their existing policy naming convention, the policy names used should match the policy name mentioned in the init<SID>.utl file mentioned in the HANA node configuration section. The Policy HANA_SID_Shared_Daily should backup /hana/shared/SID on all HANA nodes; enable follow NFS for this policy and keep the policy type as “Standard.” 34 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Bill of materials HP AppSystems for SAP HANA are available in several sizes that can be broadly grouped into scale-up and scale-out configurations. Within scale-up appliances there are two categories, the first category includes sizes XS, S and M, and is based on the HP ProLiant DL580 G7 server. The second category includes sizes M+ and L and is based on the HP ProLiant DL980 G7 server. Going beyond scale-up, the HP Scale-out appliance is based on HP ProLiant BL680c and HP 3PAR StoreServ Storage. A bill of materials for a backup solution would depend on some of the key choices that a customer makes. Some of the key considerations for a customer are Catalyst or VTL, FC or Ethernet and whether vaulting/offsiting and backup replication are required. If a customer chooses to go for Catalyst then it is a licensed feature of StoreOnce and going with this requires a dedicated backup Ethernet network for adequate performance; this option therefore requires additional Ethernet switches. Similarly, if a customer chooses to go with a VTL model, two SAN switches are required for redundancy; the advantage of going with this is that the MSL2024 library can also run on these SAN switches. It is also possible for a customer to have both options; this will give them more flexibility in terms of balancing the backup traffic over SAN and Ethernet, which will be an expensive option. Table 14 lists the hardware required to implement a Symantec NetBackup based HANA backup solution. The table provides four sets. Set 1 is basic and minimally required. Sets 2, 3 and 4 are add-ons that bring in additional capabilities as listed in the table. Depending on customer choices additional equipment required at the remote site may include HP 5920 Series switch, HP StoreFabric Switch, HP Rack, etc. Table 14, Set 3, lists only the most basic hardware required at the remote site. A point to note here is that remote replication for a VTL/NAS is a licensed feature in HP StoreOnce. 35 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Table 14. Bill of materials Part Number Description Set 1 – Basic Setup – this is required to run Catalyst, this does not provide for using MSL and VTL – Required Scale-Up Scale-Out HP AppSystem HP AppSystem Size – XS, S, M Size – M+, L 1 1 BB879A HP StoreOnce 4700 Backup* BB903A HP StoreOnce 4900 Backup* JG296A HP 5920 Series Switch^ 2 665249B21 HP Ethernet 10Gb 2-port 560SFP+ Adapter# 1 709943-xx1 HP ProLiant DL380p Gen8 Server** HP AppSystem Gen 1.2 1 2 2 1 1 1 Set 2 –This option provides the facility to use a VTL and MSL over SAN. This is required to copy backups from virtual to physical media – Optional AK379A HP StoreEver MSL2024^^ 1 1 1 QW938A HP StoreFabric SN3000B 24/24 FC Switch 2 2 2 Set 3 – This option provides remote replication capability – Optional BB879A HP StoreOnce 4700 Backup 1 1 1 BW904A HP 642 1075mm Shock Intelligent Rack## 1 1 1 1 1 1 Option 4 – This is required if the customer does not have 20U of rack space available – Optional BW904A HP 642 1075mm Shock Intelligent Rack## * HP StoreOnce Catalyst is a licensed feature and needs licensing on all appliances requiring Catalyst Stores. Catalyst Replication (Copy) does not need a replication license as the Catalyst copy functionality is included in the basic Catalyst license. ^ HP 5920 Series switch is required to create a backup network, even if a customer chooses to go with VTL this is required to connect the HANA node with the NetBackup media server. # HP Ethernet 10GbE 2-port 560SFP+ Adapter is required on HP ProLiant DL580 G7 based HANA appliances to create a backup network. This will be preloaded on the HP AppSystems for SAP HANA that are based on HP ProLiant DL580 Gen8; there is no need to procure this card from Gen8 onwards. ** The server models chosen should have HP FlexFabric 10GbE 2-port 533FLR-T Adapter. Servers with part number 709943-xx1 will be a good choice. ## A new rack may be skipped if the customer already has about 20U of rack space available in their existing rack. ^^ This tape library base, this does not include any tape drives, drives need to be ordered separately in addition to the base. Please add two drives with part number C0H28A (HP StoreEver MSL LTO-6 Ultrium 6250 Fibre Channel Drive Upgrade Kit). Caution Having an additional StoreOnce at a remote site does not provide the capability to restore at a remote site unless there is a complete HANA appliance and a complete backup solution deployed at the remote site. Having a StoreOnce at a remote site can only save on manually vaulting tape media and can also facilitate faster recovery by providing an option to copy a remote virtual media to a local StoreOnce and restore from it. 36 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Summary SAP HANA is a high speed in-memory database (IMDB) commonly used for mission critical enterprise applications, and therefore customers need an SAP certified backup solution for their SAP HANA databases. Customers demand the highest efficiency and performance in their SAP HANA environment, while minimizing cost (both CapEx and OpEx), and minimizing risk of data loss. SAP customers need an Enterprise class backup solution, like Symantec’s NetBackup, that they can count on to perform reliable backups and restores of SAP HANA databases, but also to minimize cost, and minimize risk of data loss. The HP backup solution for SAP HANA has been built and tested using the best in class components: Symantec NetBackup, HP StoreOnce Backup, HP StoreEver, HP StoreFabric, HP 5920 Switch and HP ProLiant DL380p. The workload data/results section substantiates that using NetBackup for the HP AppSystems for SAP HANA solution will enable customers to meet strict SLAs with demanding Recovery Time Objectives (RTO) and Recovery Point Objectives (RPO). HP StoreOnce comes with highly efficient deduplication algorithms that can help customers keep their storage utilization and costs under control. HP StoreEver provides long term data protection and also complements HP StoreOnce. HP StoreFabric provides a high speed SAN, yet it comes with familiar management interfaces of its previous generations. Since the HP backup solution for SAP HANA is built using best in class, well established and familiar components there is no real investment required for training and management. As established throughout this document all of these components work together seamlessly to provide an HP backup solution that extends HP AppSystems for SAP HANA to deliver return on investment and add to profitability. Implementing a proof-of-concept As a matter of best practice for all deployments, HP recommends implementing a proof-of-concept using a test environment that matches as closely as possible the planned production environment. In this way, appropriate performance and scalability characterizations can be obtained. For help with implementing a proof-of-concept, contact an HP Services representative (hp.com/large/contact/enterprise/index.html) or your HP partner. 37 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Appendix 1: Bill of materials of tested configuration Table 15 is a bill of materials of the actual tested configuration. It may however be noted that since StoreOnce 4430 and B6200 have been replaced by newer models, i.e., 4700, 4900 and 6500, we recommend customers choose from the recommended bill of materials covering newer models. Table 15. Bill of materials for the tested configuration Part Number 38 Description Scale-Up Scale-Out HP AppSystem HP AppSystem Size – XS, S, M Size – M+, L 1 1 BB857A HP StoreOnce 4430 Backup EJ022A HP StoreOnce B6200 Backup JG296A HP 5920AF-24XG Switch 2 665249B21 HP Ethernet 10Gb 2-port 560SFP+ Adapter 1 A2E62A HP ProLiant DL380 G7 Server AK379A HP AppSystem Gen 1.2 1 2 2 1 1 1 HP StoreEver MSL2024 1 1 1 QW938A HP StoreFabric SN3000B 24/24 FC Switch 2 2 2 BW904A HP 642 1075mm Shock Intelligent Rack## 1 1 1 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Appendix 2: HP Enterprise Services can help HP and SAP Some of the HP Enterprise services for HANA include: • Business Warehouse on SAP HANA • Business Suite on SAP HANA • Enterprise Edition on SAP HANA • SAP Rapid-deployment solutions (10+ available) • HP Application Management Services – Provides industry-leading services that leverage HP RunSAP certifications and business outcome-based service level agreements for all your applications running on SAP HANA Learn more at HP Enterprise Application Services for SAP HANA hp.com/go/saphana 39 Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup Glossary 40 B-Series HP StoreFabric B-Series are Brocade SAN Switches C-Series HP StoreFabric C-Series are Cisco SAN Switches CNA Converged Network Adapter CIFS Common Internet File System DLT Digital Linear tape DR Disaster Recovery DT Disaster Tolerance FC Fibre channel FCoE Fibre Channel over Ethernet H-Series HP StoreFabric H-Series are Q-Logic SAN Switches HBA Host Bus Adapter iLO HP Integrated Lights-Out IMDB In-Memory Database (SAP HANA) IPv6 Internet Protocol version 6 iSCSI Internet Small Computer System Interface LTO Linear Tape Open LTT HP Library and Tape tools NAS Network Attached Storage NFS Network File System NIC Network Interface Card NPIV N-port ID virtualization OST Open Storage technology OFC Optic Fibre Cables RPO Recovery Point Objective RTO Recovery Time Objective SAN Storage Area Network SEM HP StoreOnce Enterprise Manager SLP Storage Lifecycle Policy VSA Virtual Server Appliance VT Virtual Tape VTL Virtual Tape Library WWN World Wide Name Technical white paper | HP RA for SAP HANA backup and recovery using Symantec NetBackup For more information HP & SAP Solutions, hp.com/go/sap HP StoreFabric Storage Networking, hp.com/go/storefabric HP Networking, hp.com/go/networking HP ProLiant, hp.com/go/proliant HP Solutions for SAP HANA, hp.com/go/hana Sizing for SAP, http://h71028.www7.hp.com/enterprise/cache/42968-0-0-225-121.html HP Storage, hp.com/go/storage HP StoreOnce Backup, hp.com/go/storeonce HP StoreOnce Software Depot Kiosk*, software.hp.com/kiosk HP Backup, Recovery and Archive Solutions, hp.com/go/burasolutions HP StoreEver Tape, hp.com/go/storeever; hp.com/go/tape HP Single Point of Connectivity Knowledge (SPOCK)*, hp.com/storage/SPOCK HP Enterprise Backup Solutions, hp.com/go/ebs HP TapeAssure, hp.com/go/tapeassure Symantec NetBackup, symantec.com/netbackup Symantec tech note for scheduling SAP HANA backups, symantec.com/docs/TECH209343 *HP Passport Account and credentials required. To help us improve our documents, please provide feedback at hp.com/solutions/feedback Disclaimer This is a reference architecture (RA) based on HP solutions and non-HP third-party solutions. HP provides support only for the HP products and solutions. Product support for non-HP products is provided by the respective vendors. Sign up for updates hp.com/go/getupdated © Copyright 2013-2014 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. Microsoft and Windows are U.S. registered trademarks of Microsoft Corporation. AMD is a trademark of Advanced Micro Devices, Inc. Intel and Xeon are trademarks of Intel Corporation in the U.S. and other countries. Oracle is a registered trademark of Oracle and/or its affiliates. UNIX is a registered trademark of The Open Group. Red Hat is a registered trademark of Red Hat, Inc. in the United States and other countries. Symantec, the Symantec Logo, and the Checkmark Logo are trademarks or registered trademarks of Symantec Corporation or its affiliates in the U.S. and other countries. 4AA5-0238ENW, June 2014, Rev. 1