Enterprise Integration Patterns Asynchronous Messaging Architectures in Practice Gregor Hohpe

Transcription

Enterprise Integration Patterns
Asynchronous Messaging Architectures
in Practice
Translator
Test Message
Splitter
Aggregator
Enricher
Gregor Hohpe
Integration Challenges
• Users want to execute business
functions that span multiple applications
• Requires disparate applications to be
connected to a common integration
solution
• However:
–
–
–
–
Networks are slow
Networks are unreliable
No two applications are alike
Change is Inevitable
Copyright © 2003 Gregor Hohpe
Isolated Systems
Unified Access
2
Message-Oriented Middleware
Message-oriented architectures provide loose
coupling and reliability
System
A
Channel
System
B
Message
¾ Remove location
• Channels are separate from
dependencies
applications
• Channels are asynchronous & ¾ Remove temporal
dependencies
reliable
¾ Remove data format
• Data is exchanged in selfdependencies
contained messages
3
Thinking Asynchronously
Order Mgmt
Web Site
Shipping
Inventory
New Order
Order Mgmt
Web Site
Shipping
Inventory
New Order
Confirm
New Order
Confirm
Idle
New Order
Confirm
Confirm
Synchronous
Asynchronous
4
Many Products & Implementations
• Message-oriented middleware (MOM)
– IBM WebSphere MQ
– Microsoft MSMQ
– Java Message Service (JMS) Implementations
• EAI Suites
– TIBCO, WebMethods, SeeBeyond, Vitria, ...
• Asynchronous Web services
HOT
HOT
– WS-ReliableMessaging, ebMS
– Sun’s Java API for XML Messaging (JAXM)
– Microsoft’s Web Services Extensions (WSE)
The Underlying Design Principles Are the Same!
5
Catalog of 65 Patterns
1. Request-Reply
Example
Message
Construction
Command Message
RPC Message
Query Message
Document Message
Event Message
Reply Message
Return Address
Correlation Identifier
Message Sequence
Message Expiration
Canonical Data Model
Format Indicator
2. Order Management
Example
Message
Routing
Content-Based Router
Message Filter
Recipient List
Splitter
Aggregator
Resequencer
Distribution w. Aggr. Resp.
Auction
Routing Table
ProcessManager
Message
Transformation
Data Enricher
Content Filter
Check Luggage
Endpoint
Endpoint
Message
Application
A
Messaging
Endpoints
Messaging Adapter
Polling Consumer
Event-Driven Consumer
Transactional Client
Competing Consumers
Message Dispatcher
Message Selector
Idempotent Receiver
Messaging Mapper
Channel
Router
Translator
Application
B
Messaging
Channels
Point-to-Point Channel
Publish-Subcr. Channel
Durable Subscriber
Datatype Channel
Invalid Message Channel
Dead Letter Channel
Guaranteed Messaging
Channel Adapter
Monitoring
Systems
Management
3. Bonus
Control Bus
Message Header
Envelope Wrapper
Message History
Message Store
Channel Purger
Test Message
6
Pattern: Request-Reply
Consumer
Request
Provider
Request Channel
Reply Channel
•
•
•
•
Reply
Service Provider and Consumer (similar to RPC)
Channels are unidirectional
Two asynchronous Point-To-Point Channels
Separate request and reply messages
7
Multiple Consumers
Requests
Requests
Provider
Request Channel
Consumer
Consumer
11
Consumer
Consumer
22
Reply Channel 1
Reply Channel 2
??
Replies
• Each consumer has its own reply queue
• How does the provider know where to send the reply?
– Could send to all consumers Æ very inefficient
– Hard code Æ violates principle of context-free service
8
Pattern: Return Address
Reply
Channel 1
Reply
Channel 2
Provider
Request Channel
Consumer
Consumer
11
Consumer
Consumer
22
Reply Channel 1
Reply Channel 2
Replies
• Consumer specifies Return Address (reply channel) in the
request message
• Service provider sends reply message to specified
channel
9
Multiple Service Providers
Provider
Provider 11
Request Channel
Consumer
Consumer
Provider
Provider 22
Reply Channel
• Request message can be consumed by more than one
service provider
• Point-to-Point Channel supports Competing Consumers,
only one service receives each request message
• Channel queues up pending requests
10
Multiple Service Providers
Consumer
Service 1
(slow)
Request 1
Request 2
Reply 2
Reply 1
Service 2
(fast)
• Reply messages get out
of sequence
• How to match request and
reply messages?
– Only send one request at a
time
Æ very inefficient
– Rely on natural order
Æ bad assumption
11
Pattern: Correlation Identifier
Consumer
1
2
1
2
Message
Identifier
Request Channel
Correlate
Request &
Reply
2
1
2
1
1
Provider
Provider 11
2
2
Provider
Provider 22
1
Response Channel
Correlation
Identifier
• Equip each message with a unique identifier
– Message ID (simple, but has limitations)
– GUID (Globally Unique ID)
– Business key (e.g. Order ID)
• Provider copies the ID to the reply message
• Consumer can match request and response
12
Pattern: Pipes-And-Filters
Pipe
Pipe
Decrypt
Decrypt
Incoming
Order
Filter
Pipe
Authenticate
Authenticate
Filter
Pipe
De-Dup
De-Dup
Filter
‘Clean’
Order
• Connect individual processing steps (filters) with message
channels (pipes)
– Pipes decouple sender and receiver
– Participants are unaware of intermediaries
– Compose patterns into larger solutions
13
Multiple Specialized Providers
Order
Messages
Order
Order
Entry
Entry
??
Widget
Widget Inv.
Inv.
Gadget
Gadget Inv.
Inv.
• Each provider can only handle specific type of message
• Route request to the “appropriate” provider based on the
content of the request message
– Do not want to burden sender with decision (decoupling)
– Letting each consumer “pick out” desired messages requires
distributed coordination
14
Pattern: Content-Based Router
Order
Messages
Widget
Widget Inv.
Inv.
Order
Order
Entry
Entry
ContentBased
Router
Gadget
Gadget Inv.
Inv.
• Insert a Content-Based Router
• Message routers forward incoming messages to different
output channels
• Message content not changed
• Mostly stateless, but can be stateful (e.g. de-duper)
15
Composite Message
Order
Message
Order
Order
Entry
Entry
??
Widget
Widget Inv.
Inv.
Gadget
Gadget Inv.
Inv.
• How can we process a message if it contains multiple
elements, each of which may have to be processed in a
different way?
– Treat each element independently
– Need to avoid missing or duplicate elements
– Make efficient use of network resources
16
Pattern: Splitter
Order
Message
Widget
Widget Inv.
Inv.
Order
Order
Entry
Entry
??
Splitter
Order Order
Item 1 Item 2
Gadget
Gadget Inv.
Inv.
• Use a Splitter to break out the composite message into a
series of individual messages, each containing data
related to one item.
17
Composite: Splitter & Router
Order
Item 1
Order
Message
Widget
Widget Inv.
Inv.
Order
Order
Entry
Entry
Splitter
Order Order
Item 1 Item 2
Gadget
Gadget Inv.
Inv.
Router
Order
Item 2
• Use a Splitter to break out the composite message into a
series of individual messages, each containing data
related to one item.
• Then use a Content-Based Router to route the individual
messages to the proper destination
18
Producing a Single Response
Order
Item 1
Response 1
Confirmed
Order
Widget
Widget Inv.
Inv.
??
Gadget
Gadget Inv.
Inv.
Order
Item 2
Billing
Billing
Response 2
• How to combine the results of individual, but related
messages so that they can be processed as a whole?
–
–
–
–
Messages out of order
Message delayed
Which messages are related?
Avoid separate channel for each system
19
Pattern: Aggregator
Order
Item 1
Response 1
Confirmed
Order
Widget
Widget Inv.
Inv.
Billing
Billing
Gadget
Gadget Inv.
Inv.
Order
Item 2
Aggregator
Response 2
• Use a stateful filter, an Aggregator, to collect and store
individual messages until a complete set of related
messages has been received.
– Aggregator publishes a single message distilled from the
individual messages.
– Correlation
– Completeness Condition
– Aggregation Algorithm
20
Pattern: Scatter-Gather
• Send a message to a dynamic set of recipients,
and return a single message that incorporates the
responses.
Scatter-Gather
Pub-Sub
Channel
Quote
Request
“Best”
Quote
Vendor
Vendor A
A
Quote
Vendor
Vendor B
B
Vendor
Vendor C
C
Aggregator
21
Composing Patterns
• Receive an order, get best offer for each item
from vendors, combine into validated order.
Scatter-Gather
Pub-Sub
Channel
New
Order
Splitter
Validated
Order
Aggregator
Quote Request
for each item
“Best” Quote
for each item
Vendor
Vendor A
A
Quote
Vendor
Vendor B
B
Vendor
Vendor C
C
Aggregator
22
System Management
• Messaging systems are asynchronous and
distributed
– Multiple platforms
– Difficult to detect errors
– Difficult to configure (property file hell)
• How can we effectively administer a messaging
system that is distributed across multiple
platforms and a wide geographic area?
23
Pattern: Control Bus
Application Message Flow
Control Bus
Management
Console
•
•
•
•
•
•
Configuration
Heartbeat
Test messages
Exceptions / logging
Statistics / Quality-of-Service (QoS)
Live console
24
How To Inspect Messages?
Point-To-Point Channel
Sender
Sender
Receiver
Receiver
Message
??
Management
Console
• Cannot add another receiver because it would consume
the message
• Cannot switch to Publish-Subscribe-Channel because
Point-To-Point Channel
may already have Competing Consumers
25
Pattern: Wire Tap
Wire Tap
Sender
Sender
Receiver
Receiver
Message
Message
Management
Console
• Simple Router that duplicates message to two output
channels
• Also known as Tee
• Some side effects: Message ID changes, latency
26
Track Messages
Wire Tap
Wire Tap
Component
Component
Analysis
• E.g., message run time, message volume
• Missed messages if channels or component unreliable
27
What if Return Address is Used?
Wire Tap
Consumer
Consumer
Consumer
Consumer
Request Channel
Provider
Reply Channels
??
• Provider routes reply message to dynamic channel
• Cannot dynamically inject Wire Tap
28
Pattern: Smart Proxy
Smart Proxy
Consumer
Consumer
Request Channel
Provider
Fixed Reply Channel
Consumer
Consumer
Reply
Channels
• A Smart Proxy stores original Return Address and
replaces it with a fixed channel address
• Intercepts reply messages and forwards them to correct
channel
• Allows analysis of request and reply messages
29
Pattern: Test Message
Test Message
Injector
Test Message
Separator
Processor
Processor
Appl. Appl.
Msg. 1 Msg. 2
Test
Message
9
Test Data
Generator
Appl.
Msg. 1
Test
Result
Appl.
Msg. 2
Test Data
Verifier
Control Bus
Management Console
• Inject application specific test messages
• Extract result from regular message flow
• Compare result against predefined (or computed) result
30
In Summary…
• Visual and verbal language to describe
integration solutions
• Combine patterns to describe larger solutions
• No fancy tools – whiteboard or PowerPoint
• No vendor jargon
• Not a precise specification language
– (e.g., see OMG UML Profile for EAI)
• Not a new “methodology”
• Each pattern describes trade-offs and
considerations not included in this overview
31
Resources
• Book (late October):
– Enterprise Integration Patterns
– Addison-Wesley, 0-321-20068-3
• Contact
– Gregor Hohpe
– [email protected]
• Web Site
–
–
–
–
http://www.eaipatterns.com
Pattern catalog
Bibliography, related papers
[email protected]
• www.thoughtworks.com
32

Enterprise Integration Patterns Asynchronous Messaging Architectures in Practice Gregor Hohpe

Transcription

Similar documents

Document 6457351

Enterprise Integration Patterns Introduction

Find the Investment Opportunities in Dubai at INV500

6th Grade Summer Reading Overview

Wanted - Source

Gregor Maehle Poster

The Metamorphosis

PEA-0137 A1 Real Life Poster_M4, 03.indd

Wedding Stationery Order Form

Felicity`s Blog

How Investology easily promotes their courses

News from the mews