Service Virtualization - Northwestern University Information

SOA Technical Overview
Achieve Business Agility
Glenn Gettman, WebSphere Connectivity Specialist
1
History has not been Kind to Brittle Businesses
•
In 1917, BC Forbes compiled his first list of the 100 largest American companies
•
In 1987, Forbes published its Forbes 100 list and compared it to its 1917 list
•
Of the original group, 61 had ceased to exist!
Of the remaining 39, only 18 had
managed to stay in the top 100.
How will your company transform its
business to beat competition and
deliver increasing customer value
over time?
Be prepared for change.
It’s the only thing you can count on.
2
Business flexibility depends on IT flexibility
“Today’s IT architectures are the biggest
roadblocks most companies face when making
strategic moves.”
–McKinsey
“Flexible IT, Better Strategy”
Why are today’s architectures roadblocks?
Mainframe apps - Blue
PC/NT apps - Green
Unix apps - Yellow
3rd party interface - Orange
Lines: Colors have no special meaning.
DRAFT
AIS Reports
Budget
Analysis Tool
Vendor
Maintenance
Insertions
Orders
AIS Calendar
Due Dates
General
Maintenance
Broadcast
Filter
Stores & Mrkts
Best Buy - Application Diagram V4
November 10, 1999
Vendor Setup
They are to help make the diagram easier to
read.
For More Information: See the database
containing information about each
application: Application V4.mdb
Process Servers
(Imaging)
Printer
Maintenance
NEW Soundscan
NPD Group
AIG Warranty Guard
Mesa Data
S20-Sales
Polling
Printer PO
DRAFT
Page 1 of 2
Depository
Banks
UAR - Universal Account
Reconcilliation
Sterling VAN
Mailbox (Value)
Roadshow
I13- Auto
Replenishment
I06 - Customer
Order
S01 - Sales
Corrections
I17 Customer Perceived
In-Stock
I15 Hand Scan
Apps
I06 Warehouse
Management
Print Costing
Invoice App
E13
E3 Interface
Fringe PO
Smart Plus
M03 - Millennuim 3.0
Smart Plus
Launcher
•
S04 - Sales Posting
S07 - Cell
Phones
P16 - Tally Sheet
Complexity
L02-Resource
Scheduling
(Campbell)
A04 - Cust
Refund Chks
E01-EDI
Frick
Co
CTS
ACH
V02-Price
Marketing
Support
CTO2.Bestbuy.
com
V04-Sign
System
Prodigy
I10 Cycle Physical
Inventory
I04 Home
Deliveries
I02 Transfers
B01 - Stock
Status
Spec Source
SKU Tracking
S08 - Vertex
Sales
Tax
Intercept
I11 Price
Testing
I09 Cycle Counts
S02 Layaways
NPD,
SoundScan
Spec
Source
Scorecard - HR
S03-Polling
I35 - CEI
Hidden interfaces
Store
Monitor
K02
Customer Repair
Tracking
I18
SKU Rep
SKU Selection
Tool
Arthur Planning
I35 Early Warning
System
I55 SKU
Information
ELT
PowerSuite
SKU
Performance
L60 MDF
Coop
I05
Inventory Info
V01-Price Management
System
ASIS
Rebate
Transfer
X92-X96
Host to AS400
Communication
Supplier
Compliance
I01 PO
Receiving
V03- Mkt
Reactions
P09
Bonus/HR
Washington,
RGIS,
Ntl Bus Systems
S11 - ISP
Tracking
U18 - CTO
POS
Plan Administrators
(401K, PCS, Life,
Unicare, Solomon
Smith Barney)
E02-Employee
Purchase
•
L01-Promo
Analysis
1
AAS
P01Employee
Masterfile
P09 - P17
Cyborg
Banks - ACH and Pos to
Pay
S09 - Digital
Satellite
System
I12 Entertainment
Software
P14 On-line New
Hire Entry
Cobra
Monolithic and silo’d applications
S06 - Credit App
Equifax
P15 EES Employee
Change Notice
Resumix
•
I03 Return to
Vendor
D01 Post Load
Billing
M02 - Millennium
Stock Options
I07 Purchase
Order
Ad Expense
G02 - General
Ledger
Store
Scorecard
Sign
System
Texlon 3.5
NARM
I14 Count Corrections
Store Budget
Reporting
Valley Media
U16-Texlon
•
B02 Merchandise
Analysis
CopyWriter's
Workspace
Custom coded connections
Merch Mngr Approval
Batch Forcasting
Ad Measurement
AIMS Admin
Ad
Launcher
AIMS
Journal Entry Tool Kit
A05 - AP
Cellular
Rollover
AIMS
Reporting
S05 - House
Charges
Optika
PSP
C02 - Capital
Projects
Data Warehouse
•
BMP - Bus
performance Mngt
EDI
Coordinator
(Interfaces to and from the
Data Warehouse are not
displayed on this diagram)
Not designed for change
US Bank Recon
File
Connect 3
ICMS Credit
SiteSeer
In-Home
Repair
Warranty
Billing
System
OTHER APPS - PC
AP - Collections/Credit
TM - Credit Card DB
F06 - Fixed
Assets
Star Repair
Connect 3
PDF Transfe
Connect 3
Reports
Cash Over/
Short
Cash Receipts/Credit
Misc Accounting/Finance Apps - PC/NT
COBA (Corp office Budget Assistant)
PCBS(Profit Center Budget System)
Merchandising Budget
INVENTORY CONTROL APPS - PC
Code Alarm
Debit Receivings
Devo Sales
Display Inventory
In Home
Junkouts
Merchandise Withdrawl
Promo Credits
RTV Accrual
Shrink
AP Research - Inv Cntrl
AP Research-Addl Rpts
Book to Perpetual Inventory
Close Out Reporting
Computer Intelligence Data
Count Corrections
Cross Ref for VCB Dnlds
Damage Write Off
Debit Receivings
DFI Vendor Database
Display Inventory Reconcil
Display Inventory Reporting
INVENTORY CONTROL APPS - PC
DPI/CPI
IC Batching
Inventory Adj/Count Correct
Inventory Control Reports
Inventory Levels
Inventory Roll
Merchandise Withdrawl
Open Receivings
PI Count Results
PI Time Results from Inv
Price Protection
Sales Flash Reporting
Shrink Reporting
SKU Gross Margin
SKU Shrink Level Detail
USM
VCB Downloads
Actual application architecture for a consumer electronics company
3
ACCTS REC APPS - PC
990COR
Bad Debt
Benefical Fees
Beneficial Reconcil
JEAXF
JEBFA
JEBKA
JEDVA
JESOA
JEVSA
JEVSF
NSF
TeleCredit Fees
Prepared by Michelle Mills
The Big Divide
Verify Part availability
Fulfill order
Bill and collect
EJBs and servlets
access database
PeopleSoft adapter
RPG Program
• Poor communication between LOB and IT
• Business managers view IT as costly and slow
• IT maintenance cost absorb 70+% of the funding
• IT too often viewed as a cost center not a strategic
business tool
4
What is a Service?
• Consider interacting with a waiter in
a restaurant:
o
o
o
o
Order food
Brings food
Refills glasses
Brings bill
• How the waiter achieves the task is
not important, only that he does
• Waiter acts as an Interface to the
restaurant
• Waiter is your view of the
restaurant service
5
Services from Existing Software
• Existing software can be described as Services
o Through adapters
o Through “Web Services”
o Through technology
• J2EE
• .NET
• … others
6
Services from New Applications
• Services can be constructed for new applications
o Web Services
o J2EE EJBs
o .NET assemblies
• Services can be constructed for new applications that
interact with existing software
o Bridges
o Gateways
7
Services in your Business
• A wide variety of systems to
be integrated
• Each one has a distinct set of
APIs and associated
semantics
• A very difficult task to write
“glue” code to join them all
together
8
Services Oriented Architecture Steps
• Step 1: Break down your business into components
o Decide what is strategically important, what is just operations in the value chain,
prioritize and scope your transformation projects
• Step 2: Define a Service Model
o Identify your business services based on your business components, Identify your
business processes, specify the services, processes and components accordingly
• Step 3: Implement a Service Model
o Develop a service-oriented architecture to support the Componentized Business,
Implement service based scoping policy for projects, Implement appropriate
governance mechanism
9
1
0
For Successful SOA, Keep It Simple
Instead of:
10
Start with:
 Enterprise-wide initiatives
 End-to-end departmental
projects
 Architecture for
architecture’s sake
 Focus on IT
professional’s needs
 Complete re-engineering
 Integrating existing
organizational structures
Dispelling Myths about SOA
• Web Services
• ESB
=
=
SOA
SOA
• Component Architectures (and previous architectures)
failed
• The primary value of SOA is code reuse and
programmer productivity
11
SOA Reference Architecture
Business Innovation & Optimization Services
Business Monitoring
Business Dashboards
Interaction Services
Process Services
Ad hoc composition
Service Choreography
Master Data Management
User Integration
Business Rules
Information Integration
Device Integration
Staff
Data Management
IT Service
Management
Development
Services
Business Modeling
Information Services
Implement
Interoperability
Mediation
ESB (Connectivity)
Security
Registry/Repository
Asset
Mgmt.
Policy
Partner Services
Business App Services
Access Services
Partner Management
Component
Service Enablement
Protocol
Data
Object Discovery
Document Handling
Edge
Event Capture
Infrastructure Services
Workload Management
12
Virtualization
High Availability
Apps &
Info Assets
Deployment
IT
Monitoring
SOA with an ESB – Simplifying Interfaces and Applications
Turning this…
Application
Application
Application
…into this.
Application
Service
Service
Service
Service
Enterprise Service Bus (inc Backbone)
Application
Application
Application
Application
Service
Service
Service
Service
= interface
SOA + ESB:

Introduces rich

business abstractions
to describe the
application interface.
Decouples
interfaces from the
business
applications and
reduces technical
complexity.

Consolidate multi vendor
platforms into a unified
messaging backbone,
enable re-use of both the
business applications
and their interfaces,

QoS to match
business need,
sending the right
data to the right
service, logs and
correlates events
The ESB  Virtualizes access to services.
13
Two Core Principles Enable Flexibility
The ESB facilitates the decoupling of interactions between
requestor(s) and provider(s)
Service
Provider
Service
Requestor
Service Virtualization
Service
Requestor
Service
Requestor
Aspect Oriented Connectivity
 Routing
 Security
Log and Audit
 Protocol and transports
 Management
etc …
Event tracking
 Transformation of interfaces
14
Service
Requestor
Service
Provider
Agile Connectivity Begins with Integration
The Enterprise Service Bus (ESB)
Connects everything
to everything
Matches & routes communications
between applications
ESB
Distributes
Business events
Converts between
different transport
protocols
Transforms between
different data formats
An ESB enables flexible connectivity for integrating
business applications, services and processes
15
WebSphere Message Broker
•
Universal Connectivity
–
•
Simplify application connectivity to provide a flexible and dynamic infrastructure
Routes and transforms messages FROM anywhere, TO anywhere
–
Supports a wide range of protocols
• MQ, JMS 1.1, HTTP(S), Web Services (SOAP, REST), File, ERP (SAP, SEBL…), TCP/IP, SCA
–
Supports a broad range of data formats
• Binary (C/COBOL), XML, SOAP, CSV, Industry (SWIFT, EDI, HL7…), IDoc, User Defined
–
Interactions and Operations
• Route, Filter, Transform, Enrich, Monitor, Distribute, Decompose, Sequence, Correlate, Detect
•
Simple programming
–
Patterns based for top-down, parameterized connectivity of common use cases
• e.g. Web Service façades, Message oriented processing, Queue to File…
–
Construction based for bottom-up assembly of bespoke connectivity logic
•
•
•
•
•
Operational Management and Performance
–
–
–
–
16
Message Flows to describe application connectivity comprising…
Message Nodes which encapsulate required integration logic which operate on…
Message Tree which describes the data in a format independent manner
Transformation options include Graphical mapping, PHP, Java, ESQL, XSL and WTX
Extensive Administration and Systems Management facilities for developed solutions
Wide range of operating system and hardware platforms supported
Offers performance of traditional transaction processing environments
Available in Trial, Remote Deployment, Get Started and Enterprise deployment options
Message Broker Toolkit is Visual and Graphical
17
Event Driven Architecture (EDA)
• Publish and Subscribe
– Publication node to publish using WebSphere MQ
• Multiple Inputs
– Collector node groups incoming messages from multiple sources
– Correlated set of messages or events propagated downstream
• Sequence / Resequence nodes
– Messages arrive in any order, propagated according to monotonic
sequence number
– Sequence groups have defined start and end, propagated in order
within group
– Strong synergy with aggregation and collector nodes
– Fully multi-threaded and handles parallel sequence groups
• Fan out / Fan In (Aggregation)
– Generate multiple requests from single request
• Coordinate the responses to provide single consolidated response
– Improve response time because requests performed in parallel
– Fits nicely with several typical ESB patterns
18
Extensive Security Capabilities
• PEP node for Authentication, Authorization and Identity Mapping
– Can be placed anywhere in message flow to perform security functions
• Additional flexibility adds to existing input node security
• Enhanced Security Token Support for PEP and Input nodes
– SAML, Kerberos, LTPA & RACF PassTickets complement existing user, password & X509 tokens
– PEP node: all new tokens available
• Users can easily extend to support custom and non-standard tokens
– e.g. LTPA v1.1, Liberty tokens, TAM credentials…
– SOAP nodes: adds support for SAML, Kerberos & LTPA tokens
• SAML WS-Security supports Passthrough/‘Bearer’ scenarios
• Implementation allows for future addition of Sender Voucher & Holder of Key authentications
– MQ/HTTP nodes: adds SAML support to existing tokens
• Security Policy Decision Point (PDP)
– WS-Trust 1.3 for token authentication & authorization
– TFIM and LDAP PDPs also supported out-of-the-box
– Kerberos WS-Security will exploit Key Distribution Cache
• e.g. Active Directory on Windows…
• Operational Control
– Security Profiles allow operational reconfiguration of PDP without redeploy
– Resource manager security statistics; totals, passes, exceptions, cached
19
Derive Value from an Application Inventory
Understand your application assets and control their access dynamically
 Catalog application and service assets using a registry, e.g. WSRR
– Web Service and MQ Service definitions
– Classifications: by function, owning department
– Relationships: applications dependencies for lifecycle management, versioning
– User defined properties (metadata): Application=GOLD or Service=SILVER
Service
Provider
 Use registry information in ESB routing
– Built-in facilities allow ESB to access registry
– Enables policy based processing
Virtual
Service
Primary use cases:
Service
Requestor
 Visibility: application catalog & relationships
 Governance: who accesses which
applications/services
 Dynamicity: update registry to change ESB
behavior without redeploy
 Policy based Processing: policy
enforcement and policy based service
selection
20
Use metadata to implement
‘smart’ mediations
3
2
Advertise
availability of the
‘virtual services’
A’
A
Service
Registry
1
Capture metadata
about services for
use by Service Bus
Why Governance is Important
Sales
Accounting department
App. 1
Legal
App. 2
Order
fulfillment
Purchasing
Currency conversion service
1. A currency
service is created
for a specific line
of business (LOB)
21
2. Other
LOBs
start using
the service
3. LOBs
increase use
of service /
quality suffers
4. Service
is fixed at
provider’s
expense
5. Fix works
6. Maintenance
temporarily
costs soar /
but problem
provider
reappears
ends service
Without Proper Management and Governance of SOA…
This could become…
The promise of SOA
… like this
A pile of services
… and so would go the promised benefits of SOA
22
Basics of an End-to-End SOA Governance Infrastructure
Design-Time,
Service Development,
Registry / Repository
Run-Time,
Systems
Gateways,
Policy Enforcement
Management
Run-time
Policy
SOA GOVERNANCE
23
What is a registry … a repository?
Registry?
Repository?
Contains Service Metadata
Stores Service Artifacts
An integrated Registry / Repository Solution
is needed govern and manage SOA for maximum value
Business process
vitality
24
New value
through reuse
of assets
Improved
connectivity
Closer
alignment of IT
to business
Business
Flexibility
The WebSphere Service Registry and Repository provides
Value throughout the SOA Lifecycle
WebSphere Service Registry and Repository
Publish
Publish
Find
Enrich
Manage
Govern
Find
Enrich
Encourage Reuse
Find and reuse services
for building blocks for new
composite applications.
Govern
Enable Governance
Govern services
throughout the service
lifecycle
25
Enhance Connectivity
Enable dynamic and
efficient interactions
between services at
runtime.
Manage
Help optimize
service performance
Enable enforcement of
policies. Impact analysis
WSRR is your Enterprise SOA Registry/Respository
Highly Prescriptive Service Lifecycle Governance
 Register your services for visibility and reuse
 Optimize resource costs by governing service
consumption with service contract management
 Accelerate reuse with Automated Service Discovery
across heterogeneous environments
 Reduce business risk with prescriptive service
lifecycle governance including MQ Applications
 Tailor Service Governance implementation to fit your
business needs
 Apply operational policies consistently to help
achieve reliability and compliance
26
WSRR 3 Layer Security Architecture
User
Web
Interface
Eclipse
Plug -in
External
ESBs
Systems
Process
Servers
Appliances
3rd Registries &
Party Repositories
+
Events
generated
Programming
Interfaces
Content
models
WSRR Authorization
Java
SOAP
Registry &
Repository
Create
Retrieve
Update
Delete
Query
Validation
Admin
Import/Export
configure
Access Control
Governance
Transition
Validate
Notify
Impact analysis
Audit
Lifecycle
Notification
Classifications
Validators
JMX
Events
RDB
WAS Authorization
J2EE Security
WAS Authentication
WAS Server
JAAS
JMX Security
Security
Server
URI
User
Registry
27
Extensions &
Integrations
WS-SEC
Granting Access to Service
1. Service found in WSRR
User
Web
Interface
2. Request for Access Made
Eclipse
Plug -in
External
ESBs
Systems
Process
Servers
Appliances
3rd Registries &
Party Repositories
Events
generated
3. Governance Board Approves Request
Programming
Interfaces
4. WSRR Access Updated
Java
Extensions &
Integrations
SOAP
5. Security Info Given to Requestor
Registry &
6. Requestor Makes Updates
WSRR Authorization
Admin
Import/Export
configure
Access4Control
Governance
Transition
Validate
Notify
Impact analysis
Audit
Lifecycle
Classifications
Validators
1 Repository
Content
models
Create
Retrieve
Update
Delete
Query
Validation
4
JMX
Notification
WAS Authorization
J2EE Security
WAS Server
3
Events
JAAS
JMX Security
Security
Server
URI
User
Registry
28
2
RDB
WAS Authentication
+
WS-SEC
Configure Fine Grain Role Based Security with Ease
WSRR Access Control Editor
•
•
•
•
29
WSRR Access Control editor provides
guidance for authoring roles and permissions
Allows for advanced permissions with the
specification of an XPath expression
Click-to-Assign permissions to roles
Easy view for understanding what
permissions are assigned to the various roles
ESB Integration
+
Dynamic Endpoint Selection
1)
2)
3)
4)
ESB mediation is invoked
Mediation queries
WebSphere Service
Registry and Repository for
information about the
requestor and candidate
provider
Mediation matches
requestor with best
candidate provider
Message is routed
WebSphere Service Registry and Repository
Publish
Find
Availability Management
1)
2)
3)
4)
Selected provider fails to
respond due to failure
Mediation queries
WebSphere Service
Registry and Repository
to find other candidate
providers
Mediation matches
requestor with best
candidate provider
Message is routed
Manage
Enrich
Policy Enforcement
1)
2)
3)
4)
30
Govern
Mediation queries
WebSphere Service
Registry and Repository for
information about the
requestor and candidate
provider
Mediation retrieves policy
information from registry
Requestor and provider are
matched based on these
policies
Message is routed
31
32