From Business Warehouse to BI and Analytics : Bell Helicopter`s

From Business Warehouse to BI and Analytics :
Bell Helicopter’s Journey
Danielle Lindquist-Kleissler
Magesh Tarala
Bell Helicopter
Session ID# 4182
AGENDA
•
•
•
•
•
•
•
•
Initial State (4Q 2012)
QlikView
BW on HANA
Data Warehouse
– EDW to BW
– Near Real Time Data
Big Data
Analytics
Server Consolidations and Software Upgrades
– BI 4.0
– Data Services Consolidation
Where are we now and Future Steps
INITIAL STATE
In the beginning….
3
INITIAL STATE : OVERVIEW
• Multiple Data Repositories
– SAP BW on Oracle
– Oracle Based Enterprise Data Warehouse
•
•
•
•
Multiple Versions of Data Services
Multiple Reporting Tools
Older Versions of BW and BI Stack
Business Systems Modernization (~3 Year Project) go
live 1Q 2013
• Primary Issues:
– Data Availability and Data Quality
– Speed and Performance of Data Acquisition and Retrieval
4
INITIAL STATE : ARCHITECTURE
Data Services
Operational Tier
Integration Tier
SAP
Semantic Tier
Infocube
Non SAP Data
Store (DS)
EDW
Operational
Reporting
Universe
Universe
Management
Reporting
MDM
Non SAP
Engineering
B
E
C
….
Data
Mart
D
Data
Mart
Reference
Manufacturing
Universe
Universe
Universe
Metadata
Data
Mart
Data
Mart
Data Integration
Infrastructure, Support and Governance
Universe
Analytical
Reporting
Information Delivery Portal
Universe
Universe
DSO
A
HR System
Legacy Data
Functional Tier
SAP/BW
Data Extraction
SD, MM, PP,
FI, CO, PS,
GTS, iMRO,
CRM
Information Delivery
Services
INITIAL STATE : DATA FLOW
Legacy
BO XI 3
MS Access, Excel, UNIX, etc
Reporting Tools
Oracle DB
EDW Database
Oracle DB
Oracle DB
Oracle DB
BW 7.0
ETL Tools
BODI XI R2
SAP
Legacy Systems
BODS XI 3
Manufacturing
BODS XI 4
Time Entry
HR System
Source Systems
6
CranSoft
Engineering
INITIAL STATE : ISSUES
• Lag between BSM Go Live and EIM Readiness
– Availability of Data for Reporting
• BSM Go Live Process Stabilization
– Data Quality
• Reports Consistency
– Delivered by Multiple Groups from Multiple
Sources
• Slow Delivery of Reports
– Data load times limited data load frequency
7
QLIKVIEW
Interim Solution
8
QLIKVIEW : INTERIM SOLUTION
• Interim Solution, to mitigate BSM Stabilization
Issues
• Data Sources : SAP, BW, Oracle, etc.
• Created Mechanisms to Enable Enterprise Use
Paradigm
– Access to Data Sources
– Development Process
– Access to Dashboards
– End User Training
9
QLIKVIEW : EIM LANDSCAPE
Legacy
BO XI 3
MS Access, Excel, UNIX, etc
QlikView
Reporting Tools
Oracle DB
EDW Database
Oracle DB
Oracle DB
Oracle DB
BW 7.1
ETL Tools
BODI XI R2
SAP
Legacy Systems
BODS XI 3
Manufacturing
BODS XI 4
Time Entry
HR System
Source Systems
10
CranSoft
Engineering
QLIKVIEW : HYPE CURVE
We Started Here!
BW ON HANA
Long Term EIM Solution
12
BW ON HANA : STRATEGY
• Primary Reasons:
– Improve Data Load Times and Query/Report
Performance
– Consolidate Data and Business Logic
– Streamline Reporting
• Technical Migration of entire system
• BW Stats
– DSOs: 300
– Cubes: 140
– MultiProviders: 90
13
BW ON HANA : SIZING
• Parameters:
– DB Size, Current and Projected Volume
– Number of Concurrent Users
• Sizing Tools
– ABAP Report in BW: /SDF/HANA_BW_SIZING
– Database Size
• Sizing Result
– Half TB (512 GB) System for Prod (Single Node)
– Similar System for Sandbox + Dev + Test combined
14
BW ON HANA : PLANNING AND EXECUTION
• Started with a published Cookbook and
tweaked
• Tested in Sandbox, verified in Test.
• Team Composition: Basis, Infrastructure, BW,
Security
• Total Duration : 4 Months
• Production Go Live over Thanksgiving
Weekend!
15
BW ON HANA : EXPERIENCE
• Development
–
–
–
–
Eclipse Based Modeling Studio
Attribute, Analytic and Calculation Views
Script Calculation Views
Models Created on BW Tables. Importing BW Models
Caused Issues.
– Manual migration of HANA models between Dev, Test
and Prod
• Reporting
– Create Universe on HANA Model to Consume in Webi
– Lumira Not Mature Enough for Deployment
16
BW ON HANA : EXPERIENCE
• Basis
– Administration is Very Different
– Take on Traditional DBA Roles. Need for DBA Role
is minimal.
– Alerts Configuration
• Security
– Similar to Traditional DB Security
– Administrator and Developer Composite Roles
were Tweaked.
– Business Analyst / Power User was Created.
17
BW ON HANA : EXPERIENCE
• Issues
– Load Failures and System Failures During Full Load
of 60 Million Records
– Daylight Savings Time Caused System Failure
• Size
– Started with 180GB and now at over 400 GB.
• Summary
– Overall, Stable System with Good Performance.
18
BW ON HANA : EIM LANDSCAPE
Legacy
BO XI 3
MS Access, Excel, UNIX, etc
BI 4
QlikView
Reporting Tools
Oracle DB
EDW Database
Oracle DB
Oracle DB
Oracle DB
BW 7.1
ETL Tools
HANA
SAP
BODI XI R2
Legacy Systems
BODS XI 3
Manufacturing
BODS XI 4
Time Entry
HR System
Source Systems
19
CranSoft
Engineering
SAP BW ON HANA : EVOLUTION
– Processing Performed in the BW Layer Pushed Down
to the DB Layer
– ABAP Code to be Replaced with SQL
– Data Models Created in the HANA Layer
– Aggregations are not Needed
– Cubes are not Needed
– Multiple Copies of Data in the Data Flow are Not
Needed
– Real-Time Replication, Smart Data Access, etc will
Streamline Data Acquisition
DATA WAREHOUSE
Single Source of Data
21
DATA WAREHOUSE : CONSOLIDATION
• Purpose
–
–
–
–
Eliminate Oracle based EDW
Bring non-SAP data into BW on HANA
Consolidate several Disparate Data Sources
Single Source of Data
• Methodology
–
–
–
–
Identify Reports Used
Identify Source Tables Used
Select Data Sets – Fields, Filter, etc
Source Systems  Data Services  BW
22
BW ANALYSIS : LINEAGE AND UTILIZATION
• Identification and Prioritization of Artifacts
– Lineage from BW objects to BO Reports
– Users and Usage of Reports
DATA WAREHOUSE : CONSOLIDATION
Legacy
BO XI 3
MS Access, Excel, UNIX, etc
BI 4
QlikView
Reporting Tools
Oracle DB
EDW Database
Oracle DB
Oracle DB
Oracle DB
BW 7.3
ETL Tools
HANA
SAP
BODI XI R2
Legacy Systems
BODS XI 3
Manufacturing
BODS XI 4
Time Entry
HR System
Source Systems
24
CranSoft
Engineering
DATA WAREHOUSE : NEAR REAL TIME DATA
• Why?
– Frequency Depends On Time To React
• Oracle CDC
– Heavy Duty Solution
– More Expensive to Implement
– Core Business Critical Systems Touched
• Alternate Solution (CDC-Lite)
– Database Query Based
– Works When New and Modified Records Can be
Identified. And Records Have a Unique Id.
– If not Available and Tables are Small, Full Loads Will
Work
25
BIG DATA / HADOOP
Complement Data Warehouse
26
HADOOP : WHY DO WE NEED?
• Aircraft Sensor Data, Vibration Data, etc.
– High Volume
– Low Value
– Semi-Structured and Unstructured
• Engineers / Data Users Familiar with Native
Technologies
• Regime Recognition and Analytics Coding in Python
• Unstructured Data Analysis
• Advanced Analytics
• Text Retrieval and Search
27
HADOOP : ARCHITECTURE
• Hortonworks Distribution Selected
– SAP and Microsoft are Hortonworks Partners
– Hortonworks Is Pure Open Source. No Proprietary
Add-Ons
• Hadoop – BW Integration
– Data Services to Extract and Ingest Data
– Data Services 4.2 on Linux has Better Integration
– SLT RS provides Hive Integration
28
PREDICTIVE ANALYTICS
PREDICTIVE ANALYTICS
• Lead Evaluation and Analytics Platform (LEAP)
– Exploratory Platform for slicing/dicing/visualizing
data
• Predict
– Replacement of existing helicopter
– First helicopter purchase
• Tools
– QlikView, Python, RapidMiner, Azure ML
30
SOFTWARE UPGRADES & SERVER
CONSOLIDATIONS
Operational Activities
31
SIMPLIFICATION EFFORTS
• QlikView Reassessment
• BODS Consolidation
• BO Upgrades and Cleanup
32
FUTURE STATE
Where are we headed?
33
SAP TECHNOLOGY ROADMAP
• SAP is Adopting Industry Standards / Fundamental
Technologies Rapidly
–
–
–
–
–
–
Eclipse Based IDE
Leverage SQL More
Predictive Analytics Library (PAL) in Python
Reduce BW Specific Nomenclature and Artifacts
Simplified Data Provisioning (SLT)
Access to Data (SDA)
Simplify, Modernize, Consolidate : Using Industry
Standard Technologies and Paradigms
FUTURE INDUSTRY TRENDS
•
•
•
•
•
•
•
More Automation in Predictive Analytics Space
Extensive User Collaboration and Sharing
Enhanced Visualization
No SQL, In-Memory, Graph Databases Become Commonplace
Analytics as a Service
Tightly Coupled Analytics and Operational Applications
Specialized Tools Optimized for Certain Areas (Log Analytics,
Sentiment Analysis, etc.)
Data Driven Digital Enterprise
NEXT STEPS FOR BELL
•
•
•
•
•
Increase BW Appliance Size to 1TB
Upgrade to BW 7.5 and HANA SPS11
Enterprise HANA
Tighter Hadoop Integration
Statistical Discovery, Data Mining, Predictive
Analytics / Machine Learning
• Ad-Hoc Data Exploration and Discovery Tools
– Lumira, QlikSense, Power BI
Align with SAP Trajectory and Bell’s SAP Team’s Roadmap
36
PLANNED END STATE
Reporting and User Interfaces
Custom / XS Apps
Big Data &
Analytics
BI 4
QlikView
Data Store
HANA
Sidecar
Hadoop
HANA BW
ETL Tools
BODS XI 4
SLT RS
Source Systems
SAP
Mission Link
Manufacturing
37
Time Entry
HR System
Engineering
3 KEY LEARNING POINTS
• Technology roadmap should match the
organization’s DNA and Industry Direction
• Technology Adoption Alone Does Not Solve
Business Problems. Organization and
Structure, Process and Technology need to be
in sync.
• Both SAP and non-SAP tools are rapidly
evolving. Early adoption comes with a price.
Not adopting new technologies is not an
option either.
38
RETURN ON INVESTMENT
• HANA: Have not seen development
efficiencies. Savings from faster and more
frequent data is there, but hard to quantify.
• BO Consolidation: Reduction in servers,
licenses and operational costs are $xxx.yy
• Big Data: Solr project delivered a massive ROI.
Rest of the projects have very high potential.
• Analytics: ROI from operational efficiencies
and targeted marketing efforts will be
realized.
BEST PRACTICES
• HANA : It is expensive! Have archival/purge
policies and complement with other sources
for lower value data.
• Education : Several paradigm shifts are
happening. Workforce retraining and
alignment is critical.
• Tech Evolution : Easy to be taken over by
industry buzz and hype. Perform detailed
POCs before adopting any product.
CLOSING
Questions?
Danielle : [email protected]
Magesh: [email protected]
FOLLOW US
Thank you for your time
Follow us on
at @ASUG365