Middleware Technologies (CSIZG524)

Dr. Rama Satish K V

satishkvr@wilp.bits-pilani.ac.in
BITS Pilani
Pilani Campus
 BITS Pilani,
BITS Pilani, PilaniPilani Campus
Campus
Important Instructions

➢ It is important to know that just login to the session does not guarantee the
attendance.
➢ Once you join the session, continue till the end to consider you as present in
the class.
➢ IMPORTANTLY, you need to make the class more interactive by responding
to Professors queries in the session.
➢ Whenever Professor calls your number / name ,you need to respond,
otherwise it will be considered as ABSENT

BITS Pilani,
BITS Pilani, PilaniPilani Campus
Campus
 Text Books

T1: Letha Hughes Etzkorn - Introduction

to middleware _ web services, object
components, and cloud computing-
Chapman and Hall_CRC (2017).

T2: William Grosso - Java RMI (Designing

& Building Distributed Applications)

R1: Gregor Hohpe, Bobby Woolf - Enterprise Integration Patterns_

Designing, Building, and Deploying Messaging Solutions -Addison-
Wesley Professional (2003)

R2: MongoDB in Action

Note: In order to broaden understanding of concepts as applied to Indian IT industry, students are
advised to refer books of their choice and case-studies in their own organizations
BITS Pilani, Pilani Campus
Modular Structure

No Title of the Module

M1 Introduction and Evolution

M2 Enterprise Middleware

M3 Middleware Design and Patterns

M4 Middleware for Web-based Application and Cloud-

based Applications

M5 Specialized Middleware

BITS Pilani, Pilani Campus

 BITS Pilani
Pilani | Dubai | Goa | Hyderabad

CS1: Introduction and Evolution

Source Courtesy: Some of the contents of this PPT are sourced from materials provided by Publishers of T1
BITS Pilani, Pilani Campus
Content

• View of Middleware
• Forms of Middleware

BITS Pilani, Pilani Campus

What is Middleware?

⮚ Layer between OS and distributed applications

⮚ Hides complexity and heterogeneity of distributed system
⮚ Bridges gap between low-level OS communications and programming
language abstractions
⮚ Provides common programming abstraction and infrastructure for
distributed applications
⮚ Overview at: http://www.middleware.org

BITS Pilani, Pilani Campus

“Middleware is the software
that connects software
components or enterprise
applications in a distributed
system”.

Examples:
Enterprise Application
Integration software,
telecommunications
software, transaction monitors,
and messaging-and-queueing
software.

BITS Pilani, Pilani Campus

• Middleware provides support for (some of):
– Naming, Location, Service discovery, Replication
– Protocol handling, Communication faults, QoS
– Synchronization, Concurrency, Transactions, Storage
– Access control, Authentication

• Middleware dimensions:
– Request/Reply vs. Asynchronous Messaging
– Language-specific vs. Language-independent
– Proprietary vs. Standards-based
– Small-scale vs. Large-scale
– Tightly-coupled vs. Loosely-coupled components

BITS Pilani, Pilani Campus

Common Forms of MW

• Sockets
• Remote Procedure Calls
• Distributed Object Oriented Components (Ex: ORB)
• Message Oriented Middleware (Message Queues/
Enterprise Message Bus etc.)
• Service Oriented Architectures
• Web services (Arbitrary / RESTful)
• SQL-oriented data access
• Embedded middleware
• Cloud Computing

BITS Pilani, Pilani Campus

Socket

• Socket is an internal endpoint for sending/receiving data

within a node on network
• Berkeley/POSIX sockets defined API for Inter Process
Communication (IPC) within same host (BSD 4.2 – circa 1983)
• Early form of Middleware (limited to same host systems)
• Windows variant (WinSock) based on BSD Sockets.
• Treated similar to files in BSD/POSIX
• Maintained in File Descriptor table
• Supported protocols
– TCP/IP – IPv4, IPv6
– UDP

BITS Pilani, Pilani Campus

 Socket API
socket —creates a descriptor for use in recvmsg —receive the next incoming
network communications datagram (variation of recv)
connect —connect to a remote peer (client) recvfrom —receive the next incoming
write —send outgoing data across a datagram and record its source endpoint
connection address
read —acquire incoming data from a send —send an outgoing datagram
connection sendmsg —send an outgoing datagram
close —terminate communication and (variation of send)
deallocate a descriptor sendto —send an outgoing datagram, usually
bind —bind a local IP address and protocol to a prerecorded endpoint address
port to a socket
shutdown —terminate a TCP connection in
one or both directions
listen —set the socket listening on the given
getpeername —after a connection arrives,
address and port for connections from the
obtain the remote machine’s endpoint address
client and set the number of incoming
from a socket
connections from a client (backlog) that will
getsockopt —obtain the current options for a
be allowed in the listen queue at any one time
socket
accept —accept the next incoming setsockopt —change the options for a socket
connection (server)
recv —receive the next incoming datagram

BITS Pilani, Pilani Campus

Socket – Life Cycle

BITS Pilani, Pilani Campus

TCP Client – Server example

• #Server • #Client

Server.py Client.py

BITS Pilani, Pilani Campus

Remote Procedure Call (RPC)

⮚ Masks remote function calls as being local Client/server model

⮚ Request/reply paradigm usually implemented with message passing in
RPC service
⮚ Marshalling of function parameters and return value

BITS Pilani, Pilani Campus

Properties of RPC

Language-level pattern of function call

•easy to understand for programmer

Synchronous request/reply interaction

• natural from a programming language point-of-view
matches replies to requests
• built in synchronisation of requests and replies
Distribution transparency (in the no-failure case)
•hides the complexity of a distributed system

Various reliability guarantees

•deals with some distributed systems aspects of failure

BITS Pilani, Pilani Campus

Failure Modes of RPC

⮚ Invocation semantics supported by RPC in the light of:

network and/or server congestion,
client, network and/or server failure
note DS independent failure modes
⮚ RPC systems differ, many examples, local was Mayflower
⮚ May be or at most once ( RPC system tries
once)
⮚ Error return – programmer may retry
⮚ Exactly once (RPC system retries a few times)
• Hard error return – some failure most likely note that
“exactly once” cannot be guaranteed

BITS Pilani, Pilani Campus

Disadvantages of RPC

Synchronous request/reply interaction

• tight coupling between client and server
• client may block for a long time if server
fork(…)
loaded leads to multi-threaded
programming at client
• slow/failed clients may delay servers when remote call
replying multi-threading essential at
servers join(…)

Distribution Transparency
• Not possible to mask all problems

RPC paradigm is not object-oriented

• invoke functions on servers as opposed
to methods on objects

BITS Pilani, Pilani Campus

Object-Oriented Middleware
(OOM)
⮚Objects can be local or remote
⮚Object references can be local or remote
⮚Remote objects have visible remote interfaces
⮚Masks remote objects as being local using proxy objects

Remote method invocation

BITS Pilani, Pilani Campus

Properties of OOM

Support for object-oriented programming model

– objects, methods, interfaces, encapsulation…
– exceptions (were also in some RPC systems e.g. Mayflower)

Synchronous request/reply interaction

–same as RPC

Location Transparency
–system (ORB) maps object references to locations

Services comprising multiple servers are easier to build with OOM

–RPC programming is in terms of server-interface (operation)
–RPC system looks up server address in a location service

BITS Pilani, Pilani Campus

Java Remote Method Invocation (RMI)

public interface PrintService extends Remote {

int print(Vector printJob) throws RemoteException;
}

⮚ Distributed objects in Java

⮚ RMI compiler creates proxies and skeletons
⮚ RMI registry used for interface lookup
⮚ Entire system written in Java (single-language system)

BITS Pilani, Pilani Campus

CORBA

Common Object Request Broker Architecture

• Open standard by the OMG (Version 3.0)
• Language- and platform independent
Object Request Broker (ORB)
• General Inter-ORB Protocol (GIOP) for communication
• Interoperable Object References (IOR) contain object
location
• CORBA Interface Definition Language (IDL)
• Stubs (proxies) and skeletons created by IDL compiler
• Dynamic remote method invocation
Interface Repository
– Querying existing remote interfaces
Implementation Repository
– Activating remote objects on demand
BITS Pilani, Pilani Campus
 CORBA IDL
• Definition of language-independent remote interfaces
– Language mappings to C++, Java, Smalltalk, …
– Translation by IDL compiler
• Type system
typedef sequence<string> Files;
– basic types: long (32 bit),
interface PrintService : Server {
long long (64 bit), short, void print(in Files printJob);
float, char, boolean, };
octet, any, …
– constructed types: struct, union, sequence, array,
enum
– objects (common super type Object)
• Parameter passing
– in, out, inout
– basic & constructed types passed by value
– objects passed by reference
BITS Pilani, Pilani Campus
 CORBA Services (selection)

• Naming Service
Names 🡪 remote object references
• Trading Service
Attributes (properties) 🡪 remote object references
• Persistent Object Service
Implementation of persistent CORBA objects
• Transaction Service
Making object invocation part of transactions
• Event Service and Notification Service
• In response to applications‘ need for asynchronous
communication
• built above synchronous communication with push or pull
options
• not an integrated programming model with general IDL
messages BITS Pilani, Pilani Campus
Disadvantages of OOM

Synchronous request/reply interaction only

• So CORBA oneway semantics added and -
• Asynchronous Method Invocation (AMI)
• But implementations may not be loosely coupled

Distributed garbage collection

• Releasing memory for unused remote objects

OOM rather static and heavy-weight

• Bad for ubiquitous systems and embedded devices

BITS Pilani, Pilani Campus

Message-Oriented Middleware (MOM)
⮚ Communication using messages
⮚ Messages stored in message queues
⮚ message servers decouple client and server
⮚ Various assumptions about message content

BITS Pilani, Pilani Campus

Properties of MOM

⮚ Asynchronous interaction
– Client and server are only loosely coupled
– Messages are queued
– Good for application integration

⮚ Support for reliable delivery service

– Keep queues in persistent storage

⮚ Processing of messages by intermediate message

server(s)
– May do filtering, transforming, logging, …
– Networks of message servers

⮚ Natural for database integration

BITS Pilani, Pilani Campus

IBM MQSeries

One-to-one reliable message passing using queues

– Persistent and non-persistent messages
– Message priorities, message notification
Queue Managers
– Responsible for queues
– Transfer messages from input to output queues
– Keep routing tables
Message Channels
– Reliable connections between queue managers

MQopen Open a queue

Messaging API: MQclose Close a queue
MQput Put message into opened
queue MQgetGet message from local
queue BITS Pilani, Pilani Campus
Java Message Service (JMS)

• API specification to access MOM implementations

• Two modes of operation *specified*:
• Point-to-point
• one-to-one communication using queues
• Publish/Subscribe
• cf. Event-Based Middleware

• JMS Server implements JMS API

• JMS Clients connect to JMS servers
• Java objects can be serialised to JMS messages
• A JMS interface has been provided for MQ
• pub/sub (one-to-many) - just a specification?

BITS Pilani, Pilani Campus

Disadvantages of MOM

⮚ Poor programming abstraction (but has evolved)

• Rather low-level (cf. Packets)
• Request/reply more difficult to achieve, but can be done

⮚ Message formats originally unknown to middleware

• No type checking (JMS addresses this – implementation?)

⮚ Queue abstraction only gives one-to-one communication

• Limits scalability (JMS pub/sub – implementation?)

BITS Pilani, Pilani Campus

Web Services

⮚ Use well-known web standards for distributed computing

⮚ Communication
• Message content expressed in XML
• Simple Object Access Protocol (SOAP)
– Lightweight protocol for sync/async communication

⮚ Service Description
• Web Services Description Language (WSDL)
– Interface description for web services

⮚ Service Discovery
• Universal Description Discovery and Integration (UDDI)
– Directory with web service description in WSDL

BITS Pilani, Pilani Campus

Properties of Web Services
⮚ Language-independent and open standard

⮚ SOAP offers OOM and MOM-style communication:

• Synchronous request/reply like OOM
• Asynchronous messaging like MOM
• Supports internet transports (http, smtp, ...)
• Uses XML Schema for marshalling types to/from programming
language types

⮚ WSDL says how to use a web service

http://api.google.com/Google Search.wsdl

⮚ UDDI helps to find the right web service

• Exports SOAP API for access

BITS Pilani, Pilani Campus

Disadvantages of Web Services
Low-level abstraction
• leaves a lot to be implemented

Interaction patterns have to be built

• one-to-one and request-reply provided
• one-to-many?
• still synchronous service invocation, rather than notification
• No nested/grouped invocations, transactions, ...

No location transparency

BITS Pilani, Pilani Campus

What we lack, so far
• General interaction patterns
• we have one-to-one and request-reply
• one-to-many? many to many?
• notification?
• dynamic joining and leaving?

• Location transparency
• anonymity of communicating entities

• Support for pervasive computing

• data values from sensors
• lightweight software

BITS Pilani, Pilani Campus

Event-Based Middleware a.k.a. Publish/Subscribe

⮚ Publishers (advertise and) publish events (messages)

⮚ Subscribers express interest in events with subscriptions
⮚ Event Service notifies interested subscribers of published events
⮚ Events can have arbitrary content (typed) or name/value pairs

BITS Pilani, Pilani Campus

Topic-Based and Content-Based Pub/Sub

⮚ Event Service matches events against subscriptions

• What do subscriptions look like?

⮚ Topic-Based Publish/Subscribe
⮚Publishers publish events belonging to a topic or subject
⮚Subscribers subscribe to a topic
subscribe(PrintJobFinishedTopic, …)

⮚ (Topic and) Content-Based Publish/Subscribe

⮚Publishers publish events belonging to topics and
⮚Subscribers provide a filter based on content of events
subscribe(type=printjobfinished, printer=‘aspen’, …)

BITS Pilani, Pilani Campus

Properties of Publish/Subscribe
Asynchronous communication
• Publishers and subscribers are loosely coupled

Many-to-many interaction between pubs. and subs.

• Scalable scheme for large-scale systems
• Publishers do not need to know subscribers, and vice-versa
• Dynamic join and leave of pubs, subs,

(Topic and) Content-based pub/sub very expressive

• Filtered information delivered only to interested parties
• Efficient content-based routing through a broker network

BITS Pilani, Pilani Campus

Summary

⮚ Middleware is an important abstraction for

building distributed systems

1. Remote Procedure Call

2. Object-Oriented Middleware
3. Message-Oriented Middleware
4. Event-Based Middleware

⮚ Synchronous vs. asynchronous communication

⮚ Scalability, many-to-many communication
⮚ Language integration
⮚ Ubiquitous systems, mobile systems
BITS Pilani, Pilani Campus
Thank You
BITS Pilani, Pilani Campus
 Middleware Technologies (CSIZG524)
Dr. Rama Satish K V
BITS Pilani satishkvr@wilp.bits-pilani.ac.in
Pilani Campus
BITS Pilani, Pilani Campus
 IMP Note to Students
➢ It is important to know that just login to the session does not guarantee the
attendance.
➢ Once you join the session, continue till the end to consider you as present in
the class.
➢ IMPORTANTLY, you need to make the class more interactive by responding
to Professors queries in the session.
➢ Whenever Professor calls your number / name ,you need to respond,
otherwise it will be considered as ABSENT

BITS Pilani, Pilani Campus

Textbooks

T1: Letha Hughes Etzkorn - Introduction to middleware _ web

services, object components, and cloud computing-Chapman and
Hall_CRC (2017).

T2: William Grosso - Java RMI (Designing & Building Distributed

Applications)

R1: Gregor Hohpe, Bobby Woolf - Enterprise Integration Patterns_ Designing,

Building, and Deploying Messaging Solutions -Addison-Wesley Professional (2003)

R2: MongoDB in Action

Note: In order to broaden understanding of concepts as applied to Indian IT industry, students are advised to refer books of their choice and case-studies in their own organizations
BITS Pilani, Pilani Campus
Evaluation Components

Evaluation Name Type Weight Duration Schedule

Component
EC – 1 Quiz I & II Individual / 10% Pre/Post
Take-home Mid-Sem
EC – 2 Assignment/ Practical 10% TBA
Laboratory Exercises

EC – 3 Mid-Semester Closed Book 30% 2 Hrs. TBA

Examination

EC – 4 End-Semester Open Book 50% 3 Hrs. TBA

Examination

BITS Pilani, Pilani Campus

Modular Structure

No Title of the Module

M1 Introduction and Evolution

M2 Enterprise Middleware

M3 Middleware Design and Patterns

M4 Middleware for Web-based Application and Cloud-based Applications

M5 Specialized Middleware

BITS Pilani, Pilani Campus

 BITS Pilani
Pilani Campus

CS2:
CORBA - Common Object Request
Broker Architecture
Agenda

➢CORBA & VERSION

➢OMG
➢CORBA vs RMI
➢Inter-ORB Protocols
➢Object Bus and References
➢CORBA Naming Services
➢CORBA Object Services
➢ JAVA CORBA Facility

BITS Pilani, Pilani Campus

 CORBA -VERSION
3,4 February 2021 https://www.omg.org/spec/CORBA/3.4

3,3 October 2012 https://www.omg.org/spec/CORBA/3.3

3,2 October 2011 https://www.omg.org/spec/CORBA/3.2

3.1.1 July 2011 https://www.omg.org/spec/CORBA/3.1.1

3.0.3 March 2004 https://www.omg.org/spec/CORBA/3.0.3

2.6.1 June 2002 https://www.omg.org/spec/CORBA/2.6.1

2,5 September 2001 https://www.omg.org/spec/CORBA/2.5

2.4.2 October 2000 https://www.omg.org/spec/CORBA/2.4.2

2.3.1 October 1999 https://www.omg.org/spec/CORBA/2.3.1

2,2 July 1998 https://www.omg.org/spec/CORBA/2.2

2,1 September 1997 https://www.omg.org/spec/CORBA/2.1

2,0 February 1997 https://www.omg.org/spec/CORBA/2.0

BITS Pilani, Pilani Campus

CORBA

➢ CORBA (Common Object Request Broker Architecture) is a standard for

distributed object systems.
➢ It enables interoperability among distributed objects across different
languages and platforms.
➢ CORBA itself is a set of protocols, not a direct implementation.
➢ Systems that follow CORBA protocols are considered CORBA-compliant.
➢ CORBA-compliant objects can communicate across different facilities.
➢ The focus is on key CORBA concepts relevant to distributed objects.
➢ Java IDL is one example of a CORBA-based facility..
BITS Pilani, Pilani Campus
 OMG -Object Management Group

» OMG organisation to develop CORBA created by combined of multiple

organisations.
» As Defined by OMG, ORB provides mechanisms by which objects
transparently make request and receive response.
» ORB provides interoperability between applications on different machines
in heterogeneous distributed environments.
» Interconnects multiple object Systems.
» LIST OF ORGANIZATIONS ARE CONTRIBUTED DEVELOP CORBA
• Copyright © 1995; 1996 BNR Europe Ltd.
• Copyright © 1998 Borland International
• Copyright © 1991; 1992; 1995; 1996 Digital Equipment
• Copyright © 1995; 1996 Expersoft Corporation
• Copyright © 1996; 1997 Fujitsu
• Copyright © 1996 Genesis Development Corporation
• Copyright © 1989; 1990; 1991; 1992; 1995; 1996 Hewlett Packard
• Copyright © 1991; 1992; 1995; 1996 HyperDesk Corporation BITS Pilani, Pilani Campus
• Copyright © 1995; 1996 IONA Technologies Inc.
• Copyright © 1996; 1997 International Business Machines
• Copyright © 1996; 1997 International Computers Group
• Copyright © 1996; 1997 Micro Focus now ( OpenText Inc. )
• Copyright © 1991; 1992; 1995; 1996 NCR
• Copyright © 1995; 1996 Novell
• Copyright © 1991; 1992; 1995; 1996 Object Design
• Copyright © 1991; 1992; 1995; 1996 Object Management Group
• Copyright © 1996 Siemens Nixdorf Informationssysteme
• Copyright © 1991; 1992; 1995; 1996 Sun Microsystems
• Copyright © 1996 Sybase, Inc.
• Copyright © 1998 Telefonica I+D
• Copyright © 1996 Visual Edge Software

BITS Pilani, Pilani Campus

CORBA Architecture and Workflow
Architecture Basic
Workflow

BITS Pilani, Pilani Campus

CORBA vs RMI

➢ CORBA differs from the architecture of Java RMI in one significant aspect:
➢ RMI is a proprietary facility developed by Sun MicroSystems, Inc., and supports objects written in the Java
programming language only.
➢ CORBA is an architecture that was developed by the Object Management Group (OMG), an industrial
consortium.

BITS Pilani, Pilani Campus

CORBA Object Interface

➢ A distributed object is defined using a software file similar to the remote interface file in
Java RMI.
➢ Since CORBA is language independent, the interface is defined using a universal language
with a distinct syntax, known as the CORBA Interface Definition Language (IDL).
➢ The syntax of CORBA IDL is similar to Java and C++. However, object defined in a
CORBA IDL file can be implemented in a large number of diverse programming languages,
including C, C++, Java, COBOL, Smalltalk, Ada, Lisp, Python, and IDLScript.
➢ For each of these languages, OMG has a standardized mapping from CORBA IDL to the
programming language, so that a compiler can be used to process a CORBA interface to
generate the proxy files needed to interface with an object implementation or an object
client written in any of the CORBA-compatible languages.
BITS Pilani, Pilani Campus
Cross-language CORBA application

BITS Pilani, Pilani Campus

Inter-ORB Protocols

➢ To allow ORBs to be interoperable, the OMG specified a protocol known as the General
Inter-ORB Protocol (GIOP), a specification which “provides a general framework for
protocols to be built on top of specific transport layers.”
➢ A special case of the protocol is the Inter-ORB Protocol (IIOP), which is the GIOP applied
to the TCP/IP transport layer.
➢ The IIOP specification includes the following elements:
1. Transport management requirements: specifies the connection and disconnection
requirements, and the roles for the object client and object server in making and unmaking
connections.
2. Definition of common data representation: a coding scheme for marshalling and
unmarshalling data of each IDL data type
3. Message formats: different types of message format are defined. The messages allow clients to
send requests to object servers and receive replies. A client uses a Request message to invoke
a method declared in a CORBA interface for an object and receives a reply message from the
server.
BITS Pilani, Pilani Campus
Object Bus

➢ An ORB which adheres to the specifications of the IIOP may interoperate with any other
IIOP-compliant ORBs over the Internet. This gives rise to the term “object bus”, where the
Internet is seen as a bus that interconnects CORBA objects.
➢ There are a large number of proprietary as well as experimental ORBs available:
➢ (See CORBA Product Profiles, http://www.puder.org/corba/matrix/)
➢ Orbix IONA
➢ Borland Visibroker
➢ PrismTech’s OpenFusion
➢ Web Logic Enterprise from BEA
➢ Ada Broker from ENST
➢ Free ORB
BITS Pilani, Pilani Campus
Object Servers, Clients and Reference

➢ As in Java RMI, a CORBA distributed object is exported by an object server, similar to the
object server in RMI.
➢ An object client retrieves a reference to a distributed object from a naming or directory
service, to be described, and invokes the methods of the distributed object.
➢ As in Java RMI, a CORBA distributed object is located using an object reference. Since
CORBA is language-independent, a CORBA object reference is an abstract entity mapped
to a language-specific object reference by an ORB, in a representation chosen by the
developer of the ORB.
➢ For interoperability, OMG specifies a protocol for the abstract CORBA object reference
object, known as the Interoperable Object Reference (IOR) protocol.

BITS Pilani, Pilani Campus

Interoperable Object Reference (IOR)

➢ An ORB compatible with the IOR protocol will allow an object reference to be registered
with and retrieved from any IOR-compliant directory service. CORBA object references
represented in this protocol are called Interoperable Object References (IORs).
➢ An IOR is a string that contains encoding for the following information:
➢ The type of the object.
➢ The host where the object can be found.
➢ The port number of the server for that object.
➢ An object key, a string of bytes identifying the object.
➢ The object key is used by an object server to locate the object.
➢ The following is an example of the string representation of an IOR [5]:
IOR:000000000000000d49444c3a677269643a312e300000000000000001000000000000004c000100000000001
5756c472612e6475626c696e2e696f6e612e696500000963000000283a5c756c7472612e6475626c696e2e696f6e6
12e69653a677269643a303a3a49523a67726964003a
➢ The representation consists of the character prefix “IOR:” followed by a series of
hexadecimal numeric characters, each character representing 4 bits of binary data in the
IOR. BITS Pilani, Pilani Campus
CORBA Naming Service

➢ CORBA specifies a generic directory service. The Naming Service serves as a directory for
CORBA objects, and, as such, is platform independent and programming language
independent.
➢ The Naming Service permits ORB-based clients to obtain references to objects they wish to
use. It allows names to be associated with object references. Clients may query a naming
service using a predetermined name to obtain the associated object reference.
➢ To export a distributed object, a CORBA object server contacts a Naming Service to bind a
symbolic name to the object The Naming Service maintains a database of names and the
objects associated with them.
➢ To obtain a reference to the object, an object client requests the Naming Service to look up
the object associated with the name (This is known as resolving the object name.)
BITS Pilani, Pilani Campus
CORBA Naming Service

➢ The API for the Naming Service is specified

in interfaces defined in IDL, and includes
methods that allow servers to bind names
to objects and clients to resolve those
names.
➢ To be as general as possible, the CORBA
object naming scheme is necessary
complex. Since the name space is
universal, a standard naming hierarchy is
defined in a manner similar to the naming
hierarchy in a file directory
BITS Pilani, Pilani Campus
CORBA Naming Context

➢ A naming context correspond to a folder or directory in a file hierarchy, while object names
corresponds to a file.
➢ The full name of an object, including all the associated naming contexts, is known as a
compound name. The first component of a compound name gives the name of a naming
context, in which the second component is accessed. This process continues until the last
component of the compound name has been reached.
➢ Naming contexts and name bindings are created using methods provided in the Naming
Service interface.
➢ CORBA Object Name: The syntax for an object name is as follows:
<naming context > …<naming context><object name>
where the sequence of naming contexts leads to the object name.
BITS Pilani, Pilani Campus
Example of a naming hierarchy

➢ As shown, an object representing the men’s

clothing department is named
store.clothing.men, where store and clothing
are naming contexts, and men is an object
name.
➢ The Interoperable Naming Service (INS) is a
URL-based naming system based on the
CORBA Naming Service, it allows applications
to share a common initial naming context and
provide a URL to access a CORBA object.

BITS Pilani, Pilani Campus

CORBA Object Services

➢ CORBA specify services commonly needed in distributed applications, some of which are:
➢ Naming Service:
➢ Concurrency Service:
➢ Event Service: for event synchronization;
➢ Logging Service: for event logging;
➢ Scheduling Service: for event scheduling;
➢ Security Service: for security management;
➢ Trading Service: for locating a service by the type (instead of by name);
➢ Time Service: a service for time-related events;
➢ Notification Service: for events notification;
➢ Object Transaction Service: for transactional processing.

➢ Each service is defined in a standard IDL that can be implemented by a developer of the
service object, and whose methods can be invoked by a CORBA client.
BITS Pilani, Pilani Campus
Object Adapters

➢ In the basic architecture of CORBA, the implementation of a distributed object interfaces

with the skeleton to interact with the stub on the object client side. As the architecture
evolved, a software component in addition to the skeleton was needed on the server side:
an object adapter.
➢ An object adapter simplifies the responsibilities of an ORB by assisting an ORB in
delivering a client request to an object implementation.
➢ When an ORB receives a client’s request, it locates the object adapter
associated with the object and forwards the request to the adapter.
➢ The adapter interacts with the object implementation’s skeleton,
which performs data marshalling and invoke the appropriate method
in the object.
➢ There are different types of CORBA object adapters.
➢ The Portable Object Adapter, or POA, is a particular type of object
adapter that is defined by the CORBA specification. An object adapter
that is a POA allows an object implementation to function with
different ORBs, hence the word portable.
BITS Pilani, Pilani Campus
Java IDL – Java’s CORBA Facility

➢ IDL is part of the Java 2 Platform, Standard Edition (J2SE).

➢ The Java IDL facility includes a CORBA Object Request Broker (ORB), an IDL-to-Java
compiler, and a subset of CORBA standard services.
➢ In addition to the Java IDL, Java provides a number of CORBA-compliant facilities,
including RMI over IIOP, which allows a CORBA application to be written using the RMI
syntax and semantics.

➢ Key Java IDL Packages

➢ package org.omg.CORBA – contains interfaces and classes which provides the mapping of
the OMG CORBA APIs to the Java programming language
➢ package org.omg.CosNaming - contains interfaces and classes which provides the naming
service for Java IDL
➢ org.omg.CORBA.ORB - contains interfaces and classes which provides APIs for the Object
Request Broker.

BITS Pilani, Pilani Campus

Java IDL Tools

➢ Java IDL provides a set of tools needed for developing a CORBA application:
➢ idlj - the IDL-to-Java compiler (called idl2java in Java 1.2 and before)
➢ orbd - a server process which provides Naming Service and other services
➢ servertool – provides a command-line interface for application programmers to
register/unregister an object, and startup/shutdown a server.
➢ tnameserv – an olderTransient Java IDL Naming Service whose use is now
discouraged.

BITS Pilani, Pilani Campus

A Java IDL Application example
➢ Compiling the IDL file (using Java)
➢ The IDL file should be placed in a directory dedicated to the application. The file is
compiled using the compiler idlj using a command as follows:
idlj -fall Hello.idl
➢ The –fall command option is necessary for the compiler to generate all the files needed.
➢ In general, the files can be found in a subdirectory named <some name>App when an
interface file named <some name>.idl is compiled.
➢ If the compilation is successful, the following files can be found in a HelloApp subdirectory:
➢ HelloOperations.java
➢ The CORBA Interface file Hello.idl
➢ Hello.java
01. module HelloApp
➢ HelloHelper.java 02. {
➢ HelloHolder.java 03. interface Hello
➢ _HelloStub.java 04. {
05. string sayHello();
➢ HelloPOA.java
06. oneway void shutdown();
➢ These files require no modifications. 07. };
08. };
BITS Pilani, Pilani Campus
The *Operations.java file
➢ There is a file HelloOperations.java found in HelloApp/ after you compiled using idlj
➢ It is known as a Java operations interface in general
➢ It is a Java interface file that is equivalent to the CORBA IDL interface file (Hello.idl)
➢ You should look at this file to make sure that the method signatures correspond to what
you expect.
HelloApp/HelloOperations.java : The file contains the methods HelloApp/Hello.java: The signature interface file combines
specified in the original IDL file: in this case the methods the characteristics of the Java operations interface
sayHello( ) and shutdown(). (HelloOperations.java) with the characteristics of the
CORBA classes that it extends.
01. package HelloApp;
04. /** 01. package HelloApp;
05. * HelloApp/HelloOperations.java 03. /**
06. * Generated by the IDL-to-Java compiler (portable), 04. * HelloApp/Hello.java
07. * version "3.1" from Hello.idl 05. * Generated by the IDL-to-Java compiler (portable),
08. */ 06. * version "3.1" from Hello.idl
09. 07. */
10. public interface HelloOperations 09. public interface Hello extends HelloOperations,
11. { 10. org.omg.CORBA.Object,
12. String sayHello (); 11. org.omg.CORBA.portable.IDLEntity
13. void shutdown (); 12. {…
14. } // interface HelloOperations 13. } // interface Hello

BITS Pilani, Pilani Campus

The *Operations.java file
➢ HelloHelper.java, the Helper class:
➢ The Java class HelloHelper provides auxiliary functionality needed to support a CORBA
object in the context of the Java language.
➢ In particular, a method, narrow, allows a CORBA object reference to be cast to its
corresponding type in Java, so that a CORBA object may be operated on using syntax
for Java object.
➢ HelloHolder.java, the Holder class:
➢ The Java class called HelloHolder holds (contains) a reference to an object that
implements the Hello interface.
➢ The class is used to handle an out or an inout parameter in IDL in Java syntax
( In IDL, a parameter may be declared to be out if it is an output argument, and inout if
the parameter contains an input value as well as carries an output value.)
➢ _HelloStub.java:
➢ The Java class HelloStub is the stub file, the client-side proxy, which interfaces with
the client object.
➢ It extends org.omg.CORBA.portable.ObjectImpl and implements the Hello.java
interface.
BITS Pilani, Pilani Campus
The *Operations.java file
➢ HelloPOA.java, the server skeleton:
➢ The Java class HelloImplPOA is the skeleton, the server-side proxy, combined with the
portable object adapter.
➢ It extends org.omg.PortableServer.Servant, and implements the InvokeHandler
interface and the HelloOperations interface.
➢ Server Side classes:
➢ On the server side, two classes need to be provided: the servant and the server.
➢ The servant, HelloImpl, is the implementation of the Hello IDL interface; each Hello
object is an instantiation of this class.

BITS Pilani, Pilani Campus

The Application
The Servant - HelloApp/HelloImpl.java The Servant - HelloApp/HelloServer.java
// The servant -- object implementation -- for the
Hello example. Note that this is a subclass of public class HelloServer {
HelloPOA, whose source file is generated from the public static void main(String args[]) {
// compilation of Hello.idl using j2idl. try{
// create and initialize the ORB
import HelloApp.*; ORB orb = ORB.init(args, null);
import org.omg.CosNaming.*; // get reference to rootpoa & activate the POAManager
import java.util.Properties; … POA rootpoa =
class HelloImpl extends HelloPOA { (POA)orb.resolve_initial_references("RootPOA");
private ORB orb; rootpoa.the_POAManager().activate();
public void setORB(ORB orb_val) { // create servant and register it with the ORB
orb = orb_val; HelloImpl helloImpl = new HelloImpl();
} helloImpl.setORB(orb);
// implement sayHello() method // get object reference from the servant
public String sayHello() { org.omg.CORBA.Object ref =
return "\nHello world !!\n"; rootpoa.servant_to_reference(helloImpl);
} // and cast the reference to a CORBA reference
Hello href = HelloHelper.narrow(ref);
// implement shutdown() method
public void shutdown() {
orb.shutdown(false);
}
} //end class BITS Pilani, Pilani Campus
The Application
The Servant - HelloApp/HelloServer.java
// and cast the reference to a CORBA reference
Hello href = HelloHelper.narrow(ref);
// get the root naming context
// NameService invokes the transient name service
org.omg.CORBA.Object objRef = orb.resolve_initial_references("NameService");
// Use NamingContextExt, which is part of the
// Interoperable Naming Service (INS) specification.
NamingContextExt ncRef = NamingContextExtHelper.narrow(objRef);
// bind the Object Reference in Naming
String name = "Hello";
NameComponent path[] = ncRef.to_name( name );
ncRef.rebind(path, href);
System.out.println ("HelloServer ready and waiting ...");
// wait for invocations from clients
orb.run();

BITS Pilani, Pilani Campus

The Object Client Application
➢ A client program can be a Java application, an The Servant - HelloApp/HelloClient.java

applet, or a servlet. // A sample object client application.

import HelloApp.*;
➢ The client code is responsible for creating and import org.omg.CosNaming.*; …
public class HelloClient{
initializing the ORB, looking up the object using static Hello helloImpl;
the Interoperable Naming Service, invoking the public static void main(String args[]){
try{
narrow method of the Helper object to cast the ORB orb = ORB.init(args, null);
org.omg.CORBA.Object objRef =
object reference to a reference to a Hello object orb.resolve_initial_references("NameService");
implementation, and invoking remote methods NamingContextExt ncRef =
NamingContextExtHelper.narrow(objRef);
using the reference. The object’s sayHello method helloImpl =
HelloHelper.narrow(ncRef.resolve_str(“Hello”));
is invoked to receive a string, and the object’s System.out.println(helloImpl.sayHello());
shutdown method is invoked to deactivate the helloImpl.shutdown();

service.

BITS Pilani, Pilani Campus

Compiling and Running a Java IDL application
1. Create and compile the Hello.idl file on the server machine:
idlj -fall Hello.idl
2. Copy the directory containing Hello.idl (including the subdirectory generated by idlj) to the
client machine.
3. In the HelloApp directory on the client machine: create HelloClient.java. Compile the
*.java files, including the stubs and skeletons (which are in the directory HelloApp):
javac *.java HelloApp/*.java
4. In the HelloApp directory on the server machine:
1. Create HelloServer.java. Compile the .java files:
2. javac *.java HelloApp/*.java
5. On the server machine: Start the Java Object Request Broker Daemon, orbd, which
includes a Naming Service.
To do this on Unix: orbd -ORBInitialPort 1050 -ORBInitialHost servermachinename&
To do this on Windows:
start orbd -ORBInitialPort 1050 -ORBInitialHost servermachinename

BITS Pilani, Pilani Campus

 Compiling and Running a Java IDL application
5. On the server machine, start the Hello server, as follows:
java HelloServer –ORBInitialHost <nameserver host name> -ORBInitialPort 1050
6. On the client machine, run the Hello application client. From a DOS prompt or shell, type:
java HelloClient -ORBInitialHost nameserverhost -ORBInitialPort 1050
all on one line.
Note that nameserverhost is the host on which the IDL name server is running. In this
case, it is the server machine.
7. Kill or stop orbd when finished. The name server will continue to wait for invocations until
it is explicitly stopped.
8. Stop the object server

BITS Pilani, Pilani Campus

Summary
1. The key topics introduced with CORBA are:
• The basic CORBA architecture and its emphasis on object interoperability and platform independence
• Object Request Broker (ORB) and its functionalities
• The Inter-ORB Protocol (IIOP) and its significance
• CORBA object reference and the Interoperable Object Reference (IOR) protocol
• CORBA Naming Service and the Interoperable Naming Service (INS)
• Standard CORBA object services and how they are provided.
• Object adapters, portable object Adapters (POA) and their significance.
2. The key topics introduced with Java IDL are:
• It is part of the Java TM 2 Platform, Standard Edition (J2SE)
• Java packages are provided which contain interfaces and classes for CORBA support
• Tools provided for developing a CORBA application include idlj (the IDL compiler) and orbd (the ORB and
name server)
• An example application Hello
• Steps for compiling and running an application.
• Client callback is achievable.
3. CORBA toolkits and Java RMI are comparable and alternative technologies that provide distributed objects. An
application may be implemented using either technology. However, there are tradeoffs between the two.

BITS Pilani, Pilani Campus

Thank You

39
BITS Pilani, Pilani Campus
 Middleware Technologies (CSIZG524)
Dr. Rama Satish K V
satishkvr@wilp.bits-pilani.ac.in

BITS Pilani Enterprise Application Integration, Challenges, Patterns

Pilani Campus
BITS Pilani, Pilani Campus
 IMP Note to Students
 It is important to know that just login to the session does not guarantee the
attendance.
 Once you join the session, continue till the end to consider you as present in
the class.
 IMPORTANTLY, you need to make the class more interactive by responding
to Professors queries in the session.
 Whenever Professor calls your number / name ,you need to respond,
otherwise it will be considered as ABSENT

BITS Pilani, Pilani Campus

 Textbooks

T1: Letha Hughes Etzkorn - Introduction to middleware _

web services, object components, and cloud computing-
Chapman and Hall_CRC (2017).

T2: William Grosso - Java RMI (Designing & Building

Distributed Applications)

R1: Gregor Hohpe, Bobby Woolf - Enterprise Integration Patterns_ Designing,

Building, and Deploying Messaging Solutions -Addison-Wesley Professional
(2003)

R2: MongoDB in Action

Note: In order to broaden understanding of concepts as applied to Indian IT industry, students are advised to refer books of their choice and case-studies in their own organizations
BITS Pilani, Pilani Campus
Evaluation Components

Evaluation Name Type Weight Duration Schedule

Component
EC – 1 Quiz I & II Individual / 10% Pre/Post
Take-home Mid-Sem
EC – 2 Assignment/ Practical 10% TBA
Laboratory Exercises

EC – 3 Mid-Semester Closed Book 30% 2 Hrs. TBA

Examination

EC – 4 End-Semester Open Book 50% 3 Hrs. TBA

Examination
Modular Structure

No Title of the Module

M1 Introduction and Evolution

M2 Enterprise Middleware

M3 Middleware Design and Patterns

M4 Middleware for Web-based Application and Cloud-based Applications

M5 Specialized Middleware

BITS Pilani, Pilani Campus

 BITS Pilani
Pilani Campus

CS3: Enterprise Application Integration

Challenges, Patterns, Solutions
Agenda

 CORBA
 CORBA vs RMI
 Inter-ORB Protocols
 Object Bus and References
 CORBA Naming Services
 CORBA Object Services
 JAVA CORBA Facility

BITS Pilani, Pilani Campus

CORBA
 The Common Object Request Broker Architecture (CORBA) is a standard architecture for a
distributed objects system.
 CORBA is designed to allow distributed objects to interoperate in a heterogeneous
environment, where objects can be implemented in different programming language and/or
deployed on different platforms
 CORBA is not itself a distributed objects facility; instead, it is a set of protocols.
 A distributed object facility which adhere to these protocols is said to be CORBA-
compliant, and the distributed objects that the facility support can interoperate with
objects supported by other CORBA-compliant facilities.
 CORBA is a very rich set of protocols. We will instead focus on the key concepts of CORBA
related to the distributed objects paradigm. We will also study a facility based on CORBA:
the Java IDL. BITS Pilani, Pilani Campus
CORBA Architecture and Workflow
Architecture Basic Workflow
naming
lookup
object
naming service object client
implementation

stub skeleton

ORB ORB

network network

operating operating
system system

logical data flow

physical data flow

BITS Pilani, Pilani Campus

CORBA vs RMI
 CORBA differs from the architecture of Java RMI in one significant aspect:
 RMI is a proprietary facility developed by Sun MicroSystems, Inc., and supports objects written in the Java
programming language only.
 CORBA is an architecture that was developed by the Object Management Group (OMG), an industrial
consortium.

BITS Pilani, Pilani Campus

CORBA Object Interface

 A distributed object is defined using a software file similar to the remote interface file in
Java RMI.
 Since CORBA is language independent, the interface is defined using a universal language
with a distinct syntax, known as the CORBA Interface Definition Language (IDL).
 The syntax of CORBA IDL is similar to Java and C++. However, object defined in a
CORBA IDL file can be implemented in a large number of diverse programming languages,
including C, C++, Java, COBOL, Smalltalk, Ada, Lisp, Python, and IDLScript.
 For each of these languages, OMG has a standardized mapping from CORBA IDL to the
programming language, so that a compiler can be used to process a CORBA interface to
generate the proxy files needed to interface with an object implementation or an object
client written in any of the CORBA-compatible languages.

BITS Pilani, Pilani Campus

Inter-ORB Protocols

 To allow ORBs to be interoperable, the OMG specified a protocol known as the General
Inter-ORB Protocol (GIOP), a specification which “provides a general framework for
protocols to be built on top of specific transport layers.”
 A special case of the protocol is the Inter-ORB Protocol (IIOP), which is the GIOP applied to
the TCP/IP transport layer.
 The IIOP specification includes the following elements:
1. Transport management requirements: specifies the connection and disconnection requirements,
and the roles for the object client and object server in making and unmaking connections.
2. Definition of common data representation: a coding scheme for marshalling and unmarshalling
data of each IDL data type
3. Message formats: different types of message format are defined. The messages allow clients to
send requests to object servers and receive replies. A client uses a Request message to invoke a
method declared in a CORBA interface for an object and receives a reply message from the
server.
BITS Pilani, Pilani Campus
Object Bus
 An ORB which adheres to the specifications of the IIOP may interoperate with any other
IIOP-compliant ORBs over the Internet. This gives rise to the term “object bus”, where the
Internet is seen as a bus that interconnects CORBA objects.
 There are a large number of proprietary as well as experimental ORBs available:
 (See CORBA Product Profiles, http://www.puder.org/corba/matrix/)
 Orbix IONA
 Borland Visibroker CORBA
object
CORBA
object
CORBA
object

 PrismTech’s OpenFusion
 Web Logic Enterprise from BEA ORB ORB ... ORB

 Ada Broker from ENST

 Free ORB
The Internet
BITS Pilani, Pilani Campus
CORBA Naming Service
 CORBA specifies a generic directory service. The Naming Service serves as a directory for
CORBA objects, and, as such, is platform independent and programming language
independent.
 The Naming Service permits ORB-based clients to obtain references to objects they wish to
use. It allows names to be associated with object references. Clients may query a naming
service using a predetermined name to obtain the associated object reference.
 To export a distributed object, a CORBA object server contacts a Naming Service to bind a
symbolic name to the object The Naming Service maintains a database of names and the
objects associated with them.
 To obtain a reference to the object, an object client requests the Naming Service to look up
the object associated with the name (This is known as resolving the object name.)

BITS Pilani, Pilani Campus

CORBA Object Services
 CORBA specify services commonly needed in distributed applications, some
of which are:
 Naming Service:
 Concurrency Service:
 Event Service: for event synchronization;
 Logging Service: for event logging;
 Scheduling Service: for event scheduling;
 Security Service: for security management;
 Trading Service: for locating a service by the type (instead of by name);
 Time Service: a service for time-related events;
 Notification Service: for events notification;
 Object Transaction Service: for transactional processing.
 Each service is defined in a standard IDL that can be implemented by a
developer of the service object, and whose methods can be invoked by a
CORBA client. BITS Pilani, Pilani Campus
Java IDL – Java’s CORBA Facility
 IDL is part of the Java 2 Platform, Standard Edition (J2SE).
 The Java IDL facility includes a CORBA Object Request Broker (ORB), an
IDL-to-Java compiler, and a subset of CORBA standard services.
 In addition to the Java IDL, Java provides a number of CORBA-compliant
facilities, including RMI over IIOP, which allows a CORBA application to be
written using the RMI syntax and semantics.
 Key Java IDL Packages
 package org.omg.CORBA – contains interfaces and classes which provides
the mapping of the OMG CORBA APIs to the Java programming language
 package org.omg.CosNaming - contains interfaces and classes which
provides the naming service for Java IDL
 org.omg.CORBA.ORB - contains interfaces and classes which provides APIs
for the Object Request Broker.
BITS Pilani, Pilani Campus
 BITS Pilani
Pilani Campus

CS3: Enterprise Application Integration

Challenges, Patterns, Solutions
Agenda

 Integration Challenges
 Enterprise Information Systems
 Enterprise Application Integration

BITS Pilani, Pilani Campus

What is Information Systems?
 Enterprise information systems:
 Integrated ISs that support core BPs and functions.
 e.g Marketing, Accounting, Finance, Info security, HR, Compliance, Production,
Purchasing, and Logistics.
 Examples of Information Systems:
 Supply Chain Management − For managing suppliers, inventory and shipping, etc.
 Human Resource Management − For managing personnel, training and recruiting talents;
 Employee Health Care − For managing medical records and insurance details of employees;
 Customer Relationship Management − For managing current and potential customers;
 Business Intelligence Applications − For finding the patterns from existing data from
business operations.
 ERP: Enterprise resource planning
 CPFR: Collaborative planning, forecasting, and replenishment
 KM: Knowledge management

BITS Pilani, Pilani Campus

Need for Integration of Applications

 All these systems work as individual islands of automation. Most often these systems are
standalone and do not communicate with each other due to incompatibility issues such as
 Operating systems they are residing on;
 Database system used in the system;
 Legacy systems not supported anymore.
 Main EIS need is for data integration (data sharing/ exchange):
 e.g, ERP & SCM improve SC; KM & CRM for (un)profitable customers
 All these are facilitated by communication!

BITS Pilani, Pilani Campus

EAI : Enterprise Application Integration

 EAI is an integration framework, a middleware, made of a collection of technologies and

services
 EAI allows smooth integration of all such systems and applications throughout the
enterprise and enables data sharing and more automation of business processes.
 Characteristics of EAI:
 EAI is defined as "the unrestricted sharing of data and business processes among any
connected applications and data sources in the enterprise."
 Allows integration without any major changes to current infrastructure.
 Extends middleware capabilities to cope with application integration.
 Uses application logic layers of different middleware systems as building blocks.
 Keeps track of information related to the operations of the enterprise e.g. Inventory,
sales ledger and execute the core processes that create and manipulate this
information.
BITS Pilani, Pilani Campus
EAI : Enterprise Application Integration

 Challenges of EAI
 Need for EAI
 Hub and spoke architecture concentrates all of the
 Unrestricted sharing of data and processing into a single server/cluster.
business processes across an  Often became hard to maintain and evolve
organization. efficiently.
 Linkage between customers,  Hard to extend to integrate 3rd parties on other
suppliers and regulators. technology platforms.
 The linking of data, business  The canonical data model introduces an
processes and applications to intermediary step.
automate business processes.  Added complexity and additional processing effort.
 Ensure consistent qualities of  EAI products typified.
service (security, reliability etc.).  Heavy customization required to implement the
 Reduce the on-going cost of solution.
maintenance and reduce the cost  Lock-In − Often built using proprietary technology
of rolling out new systems. and required specialist skills.
 Lack of flexibility − Hard to extend or to integrate
with other EAI products!
 Requires organization to be EAI ready.

BITS Pilani, Pilani Campus

EAI : Enterprise Application Integration

Hub and spoke architecture

 Refers to a distribution method in which a centralized "hub" exists

 A network design where we have a central device (the hub) that is connected to multiple other devices (the
spokes)

 A cost-effective solution except the hub is a single point of failure

BITS Pilani, Pilani Campus

EAI Types
 There are three main types of EAI:
 Presentation Integration Model
 Data Integration Model
 Functional Integration Mode
 Data Level − Process, techniques and
technology of moving data between data stores.
 Application Interface Level − Leveraging of
interfaces exposed by custom or packaged
applications.
 Method Level − Sharing of the business logic.
 User Interface Level − Packaging applications
by using their user interface as a common
point of integration.

BITS Pilani, Pilani Campus

Integration Challenges
 Enterprise integration is no easy task
 Enterprise integration has to deal with multiple applications running on multiple
platforms in different locations
 EAI suites for Integration
 Cross-platform, cross-language integration as well as the ability to interface
 Enterprise integration requires a significant shift in corporate politics
 A failing or misbehaving integration solution can cost a business millions of Dollars in
lost orders, misrouted payments and disgruntled customers.
 One important constraint of developing integration solutions is the limited amount of
control the integration developers typically have over the participating applications.
 Despite the wide-spread need for integration solutions, only few standards have
established themselves in this domain
 The advent of XML, XSL and Web services certainly mark the most significant advance of
standards-based features in an integration solution.
 While developing an EAI solution is challenging in itself, operating and maintaining such
solution can be even more daunting.
BITS Pilani, Pilani Campus
ESB : Enterprise Service Bus

BITS Pilani, Pilani Campus

Integration Projects
 Information Portals
 Data Replication
 Shared Business Functions
 Service-Oriented Architectures
 Distributed Business Processes
 Business-to-Business Integration

 Information Portals:
 Many business users have to access more than one system to answer a
specific question.
 A customer service representative may have to access the order
management system on the mainframe plus log on to the system that
manages orders placed over the Web.

BITS Pilani, Pilani Campus

Integration Projects
 Data Replication:  Service Oriented Architecture:
 Many business systems require  Shared business functions are often referred to as
access to the same data. services.
 Many of these systems are  A service is a well-defined function that is
going to have their own data universally available and responds to requests
stores to store customer related from “service consumers”
information.  Assembles a collection of useful services,
managing the services becomes an important
function.

BITS Pilani, Pilani Campus

Integration Projects

 Distributed Business Process:  Business Integration:

 One of the key drivers of integration is  business functions may be available
the fact that a single business from outside suppliers or business
transaction is often spread across many partners
different systems.  business may use an outside provider to
 Business process management compute sales tax rates
component that manages the  above considerations apply equally to
execution of a business function across business-to-business integration
multiple existing systems.

BITS Pilani, Pilani Campus

Integration Solutions
Tightly Coupled Integration Solution
 Effects of tightly coupled dependencies.
 How to resolve them?
 Front-end Web application has to be integrated with the back-end financial
 Easiest way is to use TCP/IP connectivity
 Platform Technology – internal representations of numbers and objects
 Location – hard-coded machine addresses
 Time – all components have to be available at the same time
 Data Format – the list of parameters and their types must match

BITS Pilani, Pilani Campus

Integration Solutions
Loosely Coupled Integration
 Mechanisms such as a common data format, Loosely Coupled Integration
queuing channels, and transformers help turn
a tightly coupled solution into a loosely coupled
solution.
 The sender no longer has to depend on the
receiver's internal data format not its location
 computer is ready to accept requests or not
 Code Complexity compared to tightly coupled
system
Basic elements of Integration Solution

BITS Pilani, Pilani Campus

Patterns

 Proxy
 Design pattern: representative for remote access
 Factory
 Design pattern: object creation
 Wrapper
 Design pattern: interface transformation
 Interceptor
 Architectural pattern: adaptable service provision

 These patterns appear very frequently in middleware construction

BITS Pilani, Pilani Campus

Proxy

 Context
 Applications organized as a set of objects in a distributed environment; a client needs
access to the services provided by some possibly remote object (the “servant”)
 Problem
 Define an access mechanism that does not involve
 hard-coding the location of the servant into the client code
 deep knowledge of the communication protocols by the client
 Desirable properties
 Access should be efficient at run time and secure
 Programming should be simple for the client; ideally there should be no difference
between local and remote access
 Constraints
 Distributed environment (no single address space)

BITS Pilani, Pilani Campus

Proxy

 Solutions
 Use a local representative of the server on the client site (isolates client from servant
and communication system)
 Keep the same interface for the representative as for the servant
 Define a uniform proxy structure to facilitate automatic generation

BITS Pilani, Pilani Campus

Factory

 Context
 Applications organized as a set of objects in a distributed environment
 Problem
 Dynamically create multiple instances of a class
 Desirable properties
 Instances should be parameterized
 Evolution should be easy (no hard-coded decisions)
 Constraints
 Distributed environment (no single address space)

BITS Pilani, Pilani Campus

Factory
 Solutions
 Abstract Factory: defines a generic interface and organization for creating objects; the
actual creation is deferred to concrete factories that actually implement the creation
methods
 Abstract Factory may be implemented using Factory Methods (a creation method that
is redefined in a subclass)
 A further degree of flexibility is achieved by us

BITS Pilani, Pilani Campus

Wrapper (Adapters)

 Context
 Clients requesting services; servers providing services; services defined by interfaces

 Problem
 Reuse an existing server by modifying either its interface or some of its functions in
order to satisfy the needs of a client (or class of clients)
 Desirable properties: should be run-time efficient; should be adaptable because the
needs may change and may not be anticipated; should be itself reusable (generic)

BITS Pilani, Pilani Campus

Wrapper (Adapters)

 Solutions
 The wrapper screens the server by intercepting method calls to its interface. Each call
is prefixed by a prologue and followed by an epilogue in the wrapper.
 The parameters and results may need to be converted.

BITS Pilani, Pilani Campus

Interceptor

 Context
 Service provision (in a general setting)
 Client-server, Peer to peer, High-level to low-level
 May be uni- or bi-directional, synchronous or asynchronous
 Problem
 Transform the service (adding new functions), by different means
 Interposing a new layer of processing (like wrapper)
 Changing the destination (may be conditional)
 Constraints
 Services may be added/removed dynamically

BITS Pilani, Pilani Campus

Interceptor
 Solutions
 Create interposition objects (statically or dynamically). These objects
 Intercept calls (and/or returns) and insert specific processing, that may be based on
contents analysis
 May redirect call to a different target
 May use callbacks

BITS Pilani, Pilani Campus

Similarities and Differences between Patterns

 Wrapper vs Proxy
 Wrapper and Proxy have a similar structure
 Proxy preserves the interface; Wrapper transforms the interface
 Proxy often (not always) involves remote access; Wrapper is ususally on-site
 Wrapper vs Interceptor
 Wrapper and Interceptor have a similar function
 Wrapper transforms the interface
 Interceptor transforms the functionality (may even completely screen the target)

BITS Pilani, Pilani Campus

Integration Patterns
 Channel Patterns describe how messages are transported across a Message Channel.
These patterns are implemented by most commercial and open source messaging
systems.
 Message Construction Patterns describe the intent, form and content of the messages that
travel across the messaging system. The base pattern for this section is
the Message pattern.
 Routing Patterns discuss how messages are routed from a sender to the correct receiver.
Message routing patterns consume a message from one channel and republish it message,
usually without modification, to another channel based on a set of conditions. The
patterns presented in this section are specializations of the Message Router pattern.

BITS Pilani, Pilani Campus

Integration Patterns

 Transformation Patterns change the content of a message, for example to accommodate

different data formats used by the sending and the receiving system. Data may have to be
added, taken away or existing data may have to be rearranged. The base pattern for this
section is the Message Translator.
 Endpoint Patterns describe how messaging system clients produce or consume messages.
 System Management Patterns describe the tools to keep a complex message-based system
running, including dealing with error conditions, performance bottlenecks and changes in
the participating systems.

BITS Pilani, Pilani Campus

Messaging Patterns

BITS Pilani, Pilani Campus

Products Implement or Use EI Patterns?

 The patterns are not tied to a specific implementation. They help you design better
solutions, whether you use any of the following platforms:
 EAI and SOA platforms, such as IBM WebSphere MQ, TIBCO, Vitria, Oracle Service
Bus, WebMethods (now Software AG), Microsoft BizTalk, or Fiorano.
 Open source ESB's like Mule ESB, JBoss Fuse, Open ESB, WSo2, Spring Integration,
or Talend ESB
 Message Brokers like ActiveMQ, Apache Kafka, or RabbitMQ
 Web service- or REST-based integration, including Amazon Simple Queue Service
(SQS) or Google Cloud Pub/Sub
 JMS-based messaging systems
 Microsoft technologies like MSMQ or Windows Communication Foundation (WCF)
BITS Pilani, Pilani Campus
Class Summary

• CORBA
• CORBA vs RMI
• Inter-ORB Protocols
• Object Bus and References
• CORBA Naming Services
• CORBA Object Services
• JAVA CORBA Facility
• Integration Challenges
• Enterprise Information Systems
• Enterprise Application Integration
• Middleware Design patterns

BITS Pilani, Pilani Campus

Important note to self

Thank You!

BITS Pilani, Pilani Campus

 Middleware Technologies SSWT ZG589

Dr. Rama Satish K V

BITS Pilani
Pilani Campus
 BITS Pilani
Pilani Campus

CS4: Integration Styles and Messaging Systems

BITS Pilani, Pilani Campus
 IMP Note to Students
 It is important to know that just login to the session does not guarantee the
attendance.
 Once you join the session, continue till the end to consider you as present in
the class.
 IMPORTANTLY, you need to make the class more interactive by responding
to Professors queries in the session.
 Whenever Professor calls your number / name ,you need to respond,
otherwise it will be considered as ABSENT

BITS Pilani, Pilani Campus

Textbooks

T1: Letha Hughes Etzkorn - Introduction to middleware _

web services, object components, and cloud computing-
Chapman and Hall_CRC (2017).

T2: William Grosso - Java RMI (Designing & Building

Distributed Applications)

R1: Gregor Hohpe, Bobby Woolf - Enterprise Integration Patterns_ Designing,

Building, and Deploying Messaging Solutions -Addison-Wesley Professional
(2003)

R2: MongoDB in Action

Note: In order to broaden understanding of concepts as applied to Indian IT industry, students are advised to refer books of their choice and case-studies in their own organizations
BITS Pilani, Pilani Campus
Evaluation Components

Evaluation Name Type Weight Duration Schedule

Component
EC – 1 Quiz I & II Individual / 10% Pre/Post
Take-home Mid-Sem
EC – 2 Assignment/ Practical 10% TBA
Laboratory Exercises

EC – 3 Mid-Semester Closed Book 30% 2 Hrs. TBA

Examination

EC – 4 End-Semester Open Book 50% 3 Hrs. TBA

Examination
Modular Structure

No Title of the Module

M1 Introduction and Evolution

M2 Enterprise Middleware

M3 Middleware Design and Patterns

M4 Middleware for Web-based Application and Cloud-based Applications

M5 Specialized Middleware

BITS Pilani, Pilani Campus

Agenda

 Integration Challenges
 Enterprise Information Systems
 Enterprise Application Integration

BITS Pilani, Pilani Campus

What is Information Systems
 Enterprise information systems:
 Integrated ISs that support core BPs and functions.
 e.g. Marketing, Accounting, Finance, Info security, HR, Compliance, Production,
Purchasing, and Logistics.
 Examples of Information Systems:
 Supply Chain Management − For managing suppliers, inventory and shipping, etc.
 Human Resource Management − For managing personnel, training and recruiting talents;
 Employee Health Care − For managing medical records and insurance details of employees;
 Customer Relationship Management − For managing current and potential customers;
 Business Intelligence Applications − For finding the patterns from existing data from
business operations.
 ERP: Enterprise resource planning
 CPFR: Collaborative planning, forecasting, and replenishment
 KM: Knowledge management
BITS Pilani, Pilani Campus
Need for Integration of Applications
 All these systems work as individual islands of automation. Most often these systems are
standalone and do not communicate with each other due to incompatibility issues such as
 Operating systems they are residing on;
 Database system used in the system;
 Legacy systems not supported anymore.
 Main EIS need is for data integration (data sharing/ exchange):
 e.g, ERP & SCM improve SC; KM & CRM for (un)profitable customers
 All these are facilitated by communication!

BITS Pilani, Pilani Campus

EAI
 EAI is an integration framework, a middleware, made of a collection of technologies and
services
 EAI allows smooth integration of all such systems and applications throughout the
enterprise and enables data sharing and more automation of business processes.
 Characteristics of EAI:
 EAI is defined as "the unrestricted sharing of data and business processes among any
connected applications and data sources in the enterprise."
 Allows integration without any major changes to current infrastructure.
 Extends middleware capabilities to cope with application integration.
 Uses application logic layers of different middleware systems as building blocks.
 Keeps track of information related to the operations of the enterprise e.g. Inventory,
sales ledger and execute the core processes that create and manipulate this
information.
BITS Pilani, Pilani Campus
EAI
 Challenges of EAI
 Need for EAI
 Hub and spoke architecture concentrates all of the
 Unrestricted sharing of data and processing into a single server/cluster.
business processes across an  Often became hard to maintain and evolve
organization. efficiently.
 Linkage between customers,  Hard to extend to integrate 3rd parties on other
suppliers and regulators. technology platforms.
 The linking of data, business  The canonical data model introduces an
processes and applications to intermediary step.
automate business processes.  Added complexity and additional processing effort.
 Ensure consistent qualities of  EAI products typified.
service (security, reliability etc.).  Heavy customization required to implement the
 Reduce the on-going cost of solution.
maintenance and reduce the cost  Lock-In − Often built using proprietary technology
of rolling out new systems. and required specialist skills.
 Lack of flexibility − Hard to extend or to integrate
with other EAI products!
 Requires organization to be EAI ready.

BITS Pilani, Pilani Campus

EAI Types
 There are three main types of EAI:
 Presentation Integration Model
 Data Integration Model
 Functional Integration Mode
 Data Level − Process, techniques and
technology of moving data between data
stores.
 Application Interface Level − Leveraging of
interfaces exposed by custom or packaged
applications.
 Method Level − Sharing of the business logic.
 User Interface Level − Packaging applications
by using their user interface as a common
point of integration.

BITS Pilani, Pilani Campus

Integration Challenges
 Enterprise integration is no easy task
 Enterprise integration has to deal with multiple applications running on multiple platforms
in different locations
 EAI suites for Integration
 Cross-platform, cross-language integration as well as the ability to interface
 Enterprise integration requires a significant shift in corporate politics
 A failing or misbehaving integration solution can cost a business millions of Dollars in lost
orders, misrouted payments and disgruntled customers.
 One important constraint of developing integration solutions is the limited amount of
control the integration developers typically have over the participating applications.
 Despite the wide-spread need for integration solutions, only few standards have
established themselves in this domain
 The advent of XML, XSL and Web services certainly mark the most significant advance of
standards-based features in an integration solution.
 While developing an EAI solution is challenging in itself, operating and maintaining such
solution can be even more daunting.
BITS Pilani, Pilani Campus
Integration Projects
 Information Portals
 Data Replication
 Shared Business Functions
 Service-Oriented Architectures
 Distributed Business Processes
 Business-to-Business Integration
 Information Portals:
 Many business users have to access more than one system to
answer a specific question.
 A customer service representative may have to access the order
management system on the mainframe plus log on to the system
that manages orders placed over the Web.

BITS Pilani, Pilani Campus

Integration Solutions
 Tightly Coupled Integration Solution
 Loosely Coupled Integration Solution

 Tightly Coupled Integration:

 Effects of tightly coupled dependencies.
 How to resolve them?
 Front-end Web application has to be integrated with
the back-end financial
 Easiest way is to use TCP/IP connectivity
 Platform Technology – internal representations of
numbers and objects
 Location – hard-coded machine addresses
 Time – all components have to be available at the
same time
 Data Format – the list of parameters and their types
must match
BITS Pilani, Pilani Campus
Patterns
 Proxy
 Design pattern: representative for remote access
 Factory
 Design pattern: object creation
 Wrapper
 Design pattern: interface transformation
 Interceptor
 Architectural pattern: adaptable service provision

 These patterns appear very frequently in middleware construction

BITS Pilani, Pilani Campus

Integration Patterns
 Channel Patterns describe how messages are transported across a Message Channel.
These patterns are implemented by most commercial and open source messaging
systems.
 Message Construction Patterns describe the intent, form and content of the messages that
travel across the messaging system. The base pattern for this section is
the Message pattern.
 Routing Patterns discuss how messages are routed from a sender to the correct receiver.
Message routing patterns consume a message from one channel and republish it message,
usually without modification, to another channel based on a set of conditions. The
patterns presented in this section are specializations of the Message Router pattern.

BITS Pilani, Pilani Campus

Integration Patterns
 Transformation Patterns change the content of a message, for example to accommodate
different data formats used by the sending and the receiving system. Data may have to be
added, taken away or existing data may have to be rearranged. The base pattern for this
section is the Message Translator.
 Endpoint Patterns describe how messaging system clients produce or consume messages.
 System Management Patterns describe the tools to keep a complex message-based system
running, including dealing with error conditions, performance bottlenecks and changes in
the participating systems.

BITS Pilani, Pilani Campus

 BITS Pilani
Pilani Campus

CS4: Integration Styles and Messaging Systems

Application Integration Criteria
 If you can develop a single, stand-alone application that doesn’t need to collaborate with
any other applications
 We can avoid the whole integration issue entirely.
 Application coupling
 Integration simplicity
 Integration technology
 Data format
 Data timeliness
 Data or functionality
 Asynchronicity

BITS Pilani, Pilani Campus

Application Integration Criteria
Application Coupling
 Integrated applications should minimize their dependencies on each other so that each
can evolve without causing problems for the others.
 Tightly coupled applications make numerous assumptions about how the other
applications work
 The interface for integrating applications should be specific enough to implement
useful functionality
Integration Simplicity
 Integrating an application into an enterprise, developers should strive to minimize
changing the application and minimize the amount of integration code needed.
 To provide good integration functionality and the approaches with the least impact on
the application may not provide the best integration into the enterprise.
BITS Pilani, Pilani Campus
Application Integration Criteria
Integration Technology
 Different integration techniques require varying amounts of specialized software and
hardware.
 These special tools can be expensive, can lead to vendor lock-in, and increase the
burden on developers to understand how to use the tools to integrate applications.
Data Format
 Integrated applications must agree on the format of the data they exchange, or must
have an intermediate translator to unify applications that insist on different data
formats.
 A related issue is data format evolution and extensibility—how the format can change
over time and how that will affect the applications.
Data Timeliness
 Integration should minimize the length of time between when one application decides
to share some data and other applications have that data.
 Data should be exchanged frequently in small chunks, rather than waiting to
exchange a large set of unrelated items.
 Applications should be informed as soon as shared data is ready for consumption.
BITS Pilani, Pilani Campus
Application Integration Criteria
Data Functionality
 Integrated applications may not want to simply share data, they may wish to share
functionality such that each application can invoke the functionality in the others.
 Invoking functionality remotely can be difficult to achieve, and even though it may
seem the same as invoking local functionality, it works quite differently, with
significant consequences for how well the integration works.
Asynchronocity
 Where the remote application may not be running or the network may be
unavailable—the source application may wish to simply make shared data available or
log a request for a sub procedure call– Asynchronicity.

BITS Pilani, Pilani Campus

Application Integration Options
 File Transfer — Have each application produce files of shared data for others to consume,
and consume files that others have produced.
 Shared Database — Have the applications store the data they wish to share in a common
database.
 Remote Procedure Invocation — Have each application expose some of its procedures so
that they can be invoked remotely, and have applications invoke those to run behaviour
and exchange data.
 Messaging — Have each application connect to a common messaging system, and
exchange data and invoke behavior using messages.

BITS Pilani, Pilani Campus

File Transfer
 Have each application produce files containing information that other applications need to
consume.
 Integrators take the responsibility of transforming files into different formats.
 Produce the files at regular intervals according to the nature of the business.
 An important decision with files is what format to use. Very rarely will the output of one
application be exactly what's needed for another, so you'll have to do a fair bit of
processing of files along the way.
 File Transfer simple is that no extra tools
or integration packages are needed, but that
also means that developers have to do a lot
of the work themselves.
BITS Pilani, Pilani Campus
Shared Database
 File Transfer enables applications to share data, but can lack timeliness, yet timeliness of
integration is often a critical issue. File Transfer also may not enforce data format
sufficiently.
 Shared Database is made much easier by the widespread use of SQL-based relational
databases. Biggest difficulties with Shared Database is coming up with a suitable design
 Multiple applications using a Shared Database to frequently read and modify the same
data can cause performance bottlenecks and even deadlocks as each application locks
others out of the data

BITS Pilani, Pilani Campus

Remote Procedure Invocation
 Remote Procedure Invocation applies  Messaging: Asynchronous messaging is
the principle of encapsulation to fundamentally a pragmatic reaction to
integrating applications. the problems of distributed systems.
 There are a number of Remote  The high frequency of messages in
Procedure Call (RPC) approaches: Messaging reduces many of the
CORBA, COM, .NET Remoting, Java inconsistency problems
RMI, etc.

BITS Pilani, Pilani Campus

Messaging Systems
 Message Structure is well defined by the messaging system used. It
usually contains header and body sections.
 Channels are the mediums where messages are produced. They are the
usual queue and topics
 Pipe and Filters pattern is useful when one needs to process messages
before they are delivered to consumer applications.
 Message Router: When the sender application does not know on which
channel the receiver is subscribed to. A router is used in between for
delivering messages on the correct channel. It has routing rules that
decides where the messages shall be delivered.

BITS Pilani, Pilani Campus

Messaging Systems
 Translators are used to change the format of message. The sender
application might send CSV data while the receiver application
understands XML, in that case we need to have a translator before the
receiving application that does the job of CSV to XML conversion.

 Endpoint is a component that helps an application interact with

messaging systems. They have the protocol built in for communication
with the messaging systems. They are the message producers and the
consumers.

BITS Pilani, Pilani Campus

Some Pattern examples

BITS Pilani, Pilani Campus

Integration Framework – Apache Camel
 Powerful open source integration framework based on EIP
 Java framework for integration and mediation
 rules in multiple DSLs:
 Fluent Java
 Spring XML
 Blueprint XML
 Scala
 over > 150 out-of-box components
 bean binding and integration with popular frameworks
 active community ~ 900 messages/month

BITS Pilani, Pilani Campus

Camel Architecture

Image from: (Camel in Action)

BITS Pilani, Pilani Campus
Camel Basics
 CamelContext object represents the Camel runtime system.
 Registry: Holds all the bean necessary for routings.
 Exchange: contains message related headers properties.
 TypeConverter: Type conversion utility for converting types.
 Component is a factory for creating Endpoint instances.
 create a CamelContext
 optionally configure components or endpoints Producer Template
 add routing rules
 start the context

BITS Pilani, Pilani Campus

Apache Camel
 Apache CAMEL
• Apache Camel is an integration library for Java.
• It’s a set of Java APIs that help you integrate and process data between different computer
systems.
• Camel is like glue between different applications.
• Camel comes with components that let you integrate lots of different applications –
everything from web services, to reading and writing files on disk.
• You can even connect Camel with web apps like Twitter and Salesforce.
• You can think of Camel as a plumbing toolkit for Java.
• Just like real plumbing pipes, Camel takes data from one point, and pipes it to another.
• Along the way, the data can be changed, transformed, or sent through other pipes.
• Included in your plumbing toolkit is a range of adaptors that fit all sorts of different apps and
software systems.
• With a full range of tools at your disposal, it’s then up to you how you choose to build the
plumbing.
• Camel was inspired by the book Enterprise Integration Patterns, which is an academic
textbook about integrating software systems.
 Apache CAMEL
• Camel facts :
• It’s built in Java
• The entire code is completely open source
• It’s not just for web services – it’s a general integration framework
• It comes with a huge library of components
• It’s mature – Apache Camel is at the foundation of some commercial integration products,
like Red Hat Fuse and Talend ESB.

• What is Camel used for?

• Almost any time you need to move data from A to B, you can probably use Camel.
• Any of the following scenarios could be implemented using Camel:
• Picking up invoices from an FTP server and emailing them to your Accounts department
• Taking files from a folder and pushing them into Google Drive
• Taking messages from a JMS queue and using them to invoke a web service
• Making a web service that allows users to retrieve customer details from a database
Apache CAMEL
 Apache CAMEL
• Route

• The basic concept in Camel is the route.

• Routes are objects which you configure in Camel, which move your data from A to B.

• To use the plumbing example from earlier, a route is a pipe that moves data from one place to
another.
• It moves data between things called endpoints.

• You can create routes in Camel either using a Java syntax, or using an XML syntax.
• Here’s a very simple Java route:

• // This is a complete Camel route definition!

• from("file:home/customers/new")
• .to("file:home/customers/old")
 Apache CAMEL
• Endpoints

• In Camel, an endpoint represents any other external system to Camel.

• For an example, at the start of a route, Camel receives a message from an endpoint.
• Then, the message might be processed in some way – perhaps by an EIP – before being sent
to another destination endpoint.

• Components

• To allow Camel to connect to an endpoint, it comes with a library of components.

• A component is simply like a plug that allows you to connect to an external system (such as a
file on disk, a mailbox, or an app like Dropbox, Twitter, etc).
• You can also think of it like an adaptor.
 Apache CAMEL
• Camel components are reusable, open source, and you can even contribute your own.
• Here are some of the most common components, and how you might reference them in an
endpoint. • As you can see, each component can
Component Purpose Endpoint URI usually read and write:

for creating or consuming • A component that is configured

HTTP http:
web sites to write something is called
a producer – for example, writing to a
for reading and writing file on disk, or writing to a message
File file:
files queue.
• A component that is configured
for reading and writing to
JMS jms: to read something is called
messaging queues
a consumer – for example, reading a
for joining your Camel file from disk, or receiving a REST
Direct direct: request.
routes together
• Between each endpoint, the data can
for getting data in and also be transformed or modified, either
Salesforce salesforce:
out of Salesforce by passing the data through another
endpoint, or by using an EIP.
 Apache CAMEL - Enterprise Integration Patterns (EIPs)
• EIPs are another important part of Camel. They do special processing on messages.
• When you want to perform some common activities on a message, such as transformation,
splitting and logging, you’ll use an EIP.
EIP name What it does Java syntax

Splitter Splits a message into multiple parts .split()

Combines several messages into

Aggregator .aggregate()
one message

Log Writes a simple log message .log()

Converts an object into a text or

Marshal .marshal()
binary format
Receives a message from an
From* .from()
endpoint

To* Sends a message to an endpoint .to()

 Apache CAMEL - Camel Context

• Finally, to run and manage your routes,

• Camel has a container called the Camel Context.

• Your routes run inside this engine.

• You could think of it almost like a mini application server.

• When Camel starts, it reads your route definitions (in Java or XML), creates the routes, adds
them to a Camel Context, and starts the Camel Context.

• When Camel terminates, it shuts down your routes, and closes the Camel Context.
 Apache CAMEL - Route
• What does a Route look like?
• Camel create routes that move data between endpoints, using components.
• Although Camel is a library for Java, it can be configured using one of two languages -
either Java or XML.
• In Camel-speak, these are known as DSLs (Domain Specific Languages).
• Each route starts with a from, configured with a URI, that defines the endpoint where the data
is coming from.
• A route can consist of multiple steps – such as transforming the data, or logging it.
• But a route usually ends with a to step, which describes where the data will be delivered to.

• A really simple route in Camel’s Java DSL could look something like this:

// This is a complete Camel route definition!

from("file:home/customers/new")
.to("file:home/customers/old")
 Apache CAMEL - Route
• That same route above could be expressed in Camel’s XML DSL like this:

<route>
<from uri="file:home/customers/new"/>
<to uri="file:home/customers/old"/>
</route>

• But… what if we wanted to add another step in our route?

• Let’s say we want to log a message when we’ve received a file.
• Then we simply need to add our new step in between the existing steps. Like this:

<route>
<from uri="file:home/customers/new"/>
<log message="Received a new customer!"/>
<to uri="file:home/customers/old"/>
</route>
• In the code above, a message is moved from the new folder to the old folder. In the middle, we use the Enterprise
Integration Pattern (EIP) called log, which writes a simple Log message to the console
 Apache CAMEL - What does data look like in Camel?
• Camel treats data as individual messages – like letters flowing through a post office.

• Each message is an individual object.

• A message can be huge, or it can be very small. Camel has an object to represent messages,
and helpfully it’s called Message.

• A Message has a body, where the message content lives.

• It also has headers, which can be used to hold values associated with the message.
• The Message object is then passed along a route.

• A Message is part of a Camel object called an Exchange.

• You’ll often see the term Exchange mentioned in Camel documentation.
• An Exchange is simply a message or interaction currently taking place inside your Camel route.
 Apache CAMEL - What does data look like in Camel?
• The important thing to understand about Camel’s message model is that the message body
can contain almost any kind of Java object, such as a List, Map or String
• The body doesn’t have to be a String, like JSON or XML.

• The real power of Camel becomes clear when you start using these different types of objects.
• Camel has good built-in support for converting between different object types.
• In fact, for many common file types, you might barely even have to write any conversion code.
 Apache CAMEL – Case Study - Assignment
• Consider a situation where a large online grocery store in your town such as the
Bigbasket in India invites you to design an IT solution for them.
• The stable and scalable solution will help them overcome the software maintenance problems
they are facing today.
• This online store has been running its business for the last decade.
• The store accepts online orders for different categories of products from their customers and
distributes those to the respective suppliers.
• For example, suppose you order some soaps, oil and milk; these three items will be
distributed to the three respective suppliers.
• The three suppliers will then send their supplies to a common distribution point from where
the entire order will be fulfilled by the delivery center.
• Now, let us look at the problem they are facing today.
 Apache CAMEL – Case Study - Assignment
• When this store started its business, it was accepting orders in a comma-separated plain
text file.
• Over a period of time, the store switched to message-driven order placement.
• Later, some software developer suggested an XML based order placement.
• Eventually, the store even adapted a web service interface.
• Now, here comes the real problem. The orders now come in different formats.
• At the same time, as the business kept on growing, the store periodically added new suppliers
to its repertoire.
• Each such supplier had its own protocol for accepting orders.
• Once again, we face the integration issue; our application architecture must be scalable to
accommodate new suppliers with their unique order placement mechanism
Assignment – Design an Integration Solution
Thank You

51
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
 IMP Note to Students
 It is important to know that just login to the session does not guarantee the
attendance.
 Once you join the session, continue till the end to consider you as present in
the class.
 IMPORTANTLY, you need to make the class more interactive by responding
to Professors queries in the session.
 Whenever Professor calls your number / name ,you need to respond,
otherwise it will be considered as ABSENT

BITS Pilani, Pilani Campus

 Course Name :
Middleware Technologies
SSWT ZG589
BITS Pilani
Pilani Campus
Textbooks

T1: Letha Hughes Etzkorn - Introduction to middleware _

web services, object components, and cloud computing-
Chapman and Hall_CRC (2017).

T2: William Grosso - Java RMI (Designing & Building

Distributed Applications)

R1: Gregor Hohpe, Bobby Woolf - Enterprise Integration Patterns_ Designing,

Building, and Deploying Messaging Solutions -Addison-Wesley Professional
(2003)

R2: MongoDB in Action

Note: In order to broaden understanding of concepts as applied to Indian IT industry, students are advised to refer books of their choice and case-studies in their own organizations
BITS Pilani, Pilani Campus
 BITS Pilani
Pilani Campus

CS5: Message Channels and Construction

Agenda

 Introduction
 Application Integration Criteria
 Application Integration Options

BITS Pilani, Pilani Campus

Application Integration Criteria
 If you can develop a single, stand-alone application that doesn’t need to collaborate with
any other applications
 We can avoid the whole integration issue entirely.
 Application coupling
 Integration simplicity
 Integration technology
 Data format
 Data timeliness
 Data or functionality
 Asynchronicity

BITS Pilani, Pilani Campus

Application Integration Options
 File Transfer — Have each application produce files of shared data for others to consume,
and consume files that others have produced.
 Shared Database — Have the applications store the data they wish to share in a common
database.
 Remote Procedure Invocation — Have each application expose some of its procedures so
that they can be invoked remotely, and have applications invoke those to run behaviour
and exchange data.
 Messaging — Have each application connect to a common messaging system, and
exchange data and invoke behavior using messages.

BITS Pilani, Pilani Campus

Integration Framework – Apache Camel
 Powerful open source integration framework based on EIP
 java framework for integration and mediation
 rules in multiple DSLs:
 Fluent Java
 Spring XML
 Blueprint XML
 Scala
 over > 150 out-of-box components
 bean binding and integration with popular frameworks
 active community ~ 900 messages/month

BITS Pilani, Pilani Campus

Camel Architecture

Image from: (Camel in Action)

BITS Pilani, Pilani Campus
Camel Basics
 CamelContext object represents the Camel runtime system.
 Registry: Holds all the bean necessary for routings.
 Exchange: contains message related headers properties.
 TypeConverter: Type conversion utility for converting types.
 Component is a factory for creating Endpoint instances.
 create a CamelContext
 optionally configure components or endpoints Producer Template
 add routing rules
 start the context

BITS Pilani, Pilani Campus

 BITS Pilani
Pilani Campus

CS5: Message Channels and Construction

Agenda

 Message Channels
 Point to Point messaging
 Publish-subscribe model
 Message Construction

BITS Pilani, Pilani Campus

Message Channels
 Messaging system has different Message Channels for different types of information the
applications want to communicate.
 When two applications wish to exchange data, they do so by sending the data through a
channel that connects the two.
 The application sending the data may not know which application will receive the data
 But by selecting a particular channel to send the data on, the sender knows that the
receiver will be one that is looking for that sort of data by looking for it on that channel.
 In this way, the applications that produce shared data have a way to communicate with
those that wish to consume it.

BITS Pilani, Pilani Campus

Message Channels Themes
 Deciding to use a Message Channel is the simple part.
 The challenge is knowing what channels your applications will need and what to use
them for. Let’s review the available options.
 Fixed set of channels:
 The set of Message Channels available to an application tends to be static.
 When designing an application, a developer has to know
 Where to put what types of data so to share the data with other applications, and
 Where to look for what types of data coming from other applications.
 These paths of communication cannot be dynamically created and discovered at runtime,
they need to be agreed upon at design time so that the application knows where its data is
coming from and where the data is going to.
BITS Pilani, Pilani Campus
Message Channels Themes
 Determining the set of channels

 A related issue is: Who decides what Message Channels are available, the messaging
system or the applications?
 Does the messaging system define certain channels and require the applications to make
due with those?
 Or the applications determine what channels they need and require the messaging
system to provide them?
 There is no simple answer; designing the needed set of channels is iterative.
 First the applications determine the channels the messaging system will need to provide.
 Subsequent applications will try to design their communication around the channels that
are available
 Applications already use a certain set of channels, and new applications wish to join in,
they too will use the existing set of channels.
 When existing applications add new functionality, they may require new channels.

BITS Pilani, Pilani Campus

Message Channels Themes
 Unidirectional channels
 Another common source of confusion is whether a Message Channel is unidirectional or
bidirectional.
 Technically, it’s neither;
 A channel is more like a bucket that some applications add data to and other
applications take data from.
 But because the data is in messages that travel from one application to another, that
gives the channel direction, making it unidirectional.
 So for all practical purposes, channels are unidirectional.
 As a consequence, for two applications to have a two-way conversation,
 They will need two channels, one in each direction Request-Reply.
BITS Pilani, Pilani Campus
Message Channels Decisions
 Now that we understand what Message Channels are, let’s consider the decisions
involved in using them:

 One-to-one or one-to-many
 What type of data
 Invalid and dead messages
 Crash proof
 Non-messaging clients
 Communications backbone

BITS Pilani, Pilani Campus

Message Channels Decisions
 One-to-one or one-to-many
 When your application shares a piece of data, do you want to share it with just one other
application or with any other application that is interested?
 To send the data to a single application, use a "Point-to-Point Channel".
 This does ensure that any one piece of data will only be received by one of the
applications.
 If you want all of the receiver applications to be able to receive the data, use a "Publish-
Subscribe Channel".
 When you send a piece of data this way, the channel effectively copies the data for each of
the receivers.

BITS Pilani, Pilani Campus

Message Channels Decisions
 What type of data — "Datatype Channel“
 Any data in any computer memory has to conform to some sort of type,
 A known format or expected structure with an agreed upon meaning.
 Messaging systems work much the same way, the message contents must conform to
some type so that the receiver understands the data’s structure.

 "Datatype Channel" is the principle that all of the data on a channel has to be of the
same type.
 This is the main reason why messaging systems need lots of channels
 If the data could be of any type, the messaging system would only need one channel (in
each direction) between any two applications.
BITS Pilani, Pilani Campus
Message Channels Decisions
Invalid and dead messages — "Dead Letter Channel“
 The message system can ensure that a message is delivered properly,
 But it cannot guarantee that the receiver will know what to do with it.
 The receiver has expectations about the data’s type and meaning;
 When it receives a message that doesn’t meet these expectations, there’s not much it can
do.

 What it can do, though, it put the strange message on a specially designated Invalid
Message Channel, in hopes that some utility monitoring the channel will pick up the
message and figure out what to do with it.

 Many messaging systems have a similar built-in feature, a "Dead Letter Channel" for
messages which are successfully sent but ultimately cannot be successfully delivered.

BITS Pilani, Pilani Campus

Message Channels Decisions
Crash proof — "Guaranteed Delivery“
 If the messaging system crashes or is shut down for maintenance, what happens to its
messages?
 When it is back up and running, will its messages still be in its channels?
 By default, no; channels store their messages in memory.
 However, "Guaranteed Delivery" makes channels persistent so that their messages are
stored on disk.
 This hurts performance but makes messaging more reliable, even when the messaging
system isn’t.

BITS Pilani, Pilani Campus

Message Channels Decisions
Non-messaging clients — "Channel Adapter“
 What if an application cannot connect to a messaging system but still wants to
participate in messaging?
 Normally it would be out of luck;
 But if the messaging system can connect to the application through its user interface,
API, its database, or through a network connection such as TCP/IP or HTTP—
 Then a "Channel Adapter" on the messaging system can be used to connect a channel (or
set of channels) to the application.
 Channel Adapter helps to connect without having to modify the application and perhaps
without having to have a messaging client running on the same machine as the
application.
BITS Pilani, Pilani Campus
Message Channels Decisions
Communications backbone — "Message Bus“

 As enterprise’s applications connect to the messaging system and make their

functionality available through messaging,
 The messaging system becomes a centralized point of one-stop-shopping for functionality
in the enterprise.
 A new application simply needs to know which channels to use to request functionality
and which others to listen on for the results.
 The messaging system itself essentially becomes a "Message Bus" a backbone providing
access to all of the enterprise’s various and ever-changing applications and functionality.

BITS Pilani, Pilani Campus

Point to Point Channel
 An application is using Messaging to make remote procedure calls (RPC’s) or transfer
documents. How can the caller be sure that exactly one receiver will receive the document
or perform the call?
 RPC is invoked on a single remote process, so the receiver performs the procedure.
 And since the receiver was only called once, it only performs the procedure once.
 But with messaging, once a call is packaged as a Message and placed on a Message
Channel, potentially many receivers could see it on the channel and decide to perform the
procedure.
 The messaging system could prevent more than one receiver from monitoring a single
channel, but this would unnecessarily limit callers that wish to transmit data to multiple
receivers.
BITS Pilani, Pilani Campus
Point to Point Channel
 All of the receivers on a channel could coordinate to ensure that only one of them actually
performs the procedure, but that would be complex, create a lot of communications
overhead.
 Multiple receivers on a single channel may be desirable so that multiple messages can be
consumed concurrently, but any one receiver should consume any single message.
 Sending the message on a Point-to-Point Channel, which ensures that only one receiver
will receive a particular message.

BITS Pilani, Pilani Campus

Point to Point Channel
 A Point-to-Point Channel ensures that only one receiver consumes any given message.
 If the channel has multiple receivers, only one of them can successfully consume a
particular message.
 If multiple receivers try to consume a single message, the channel ensures that only one of
them succeeds.
 Multiple receivers to consume multiple messages concurrently, but only a single receiver
consumes any one message.
 When a Point-to-Point Channel only has one consumer, the fact that a message only gets
consumed once is not surprising.
 When the channel has multiple consumers, then they become Competing Consumers, and
the channel ensures that only one of the consumers receives each message.
 The effort to consume messages is highly scalable because that work can be load-balanced
across multiple consumers running in multiple applications on multiple computers.
BITS Pilani, Pilani Campus
Point to Point Channel
 Whereas a Point-to-Point Channel sends a message to only one of the available receivers,
to send a message to all available receivers, use a Publish-Subscribe Channel.
 To implement RPC's using messaging, use Request-Reply with a pair of Point-to-Point
Channels.
 The call is a Command Message whereas the reply is a Document Message.
 Example: Stock Trading
 In a stock trading system, the request to make a particular trade is a message that should
be consumed and performed by exactly one receiver, so the message should be placed on a
Point-to-Point Channel.
 Example: JMS Queue (Java Messaging Service)
 In JMS, a point-to-point channel implements the Queue interface.
 The sender uses a QueueSender to send messages; each receiver uses its own
QueueReceiver to receive messages

BITS Pilani, Pilani Campus

Enterprise Application Integration (EAI)
 Message Channels – Example (JMS)

BITS Pilani, Pilani Campus

Enterprise Application Integration (EAI)

JMS Message Producer

The message producer is an object which is created by a
session and is used for sending messages to the destination.
This implements the MessageProducer interface.
We use Session to create a MessageProducer for the
destination, queue or topic object.

JMS Message Consumer

The message consumer is an object which is created

by a session and is used to receive messages sent at a
destination.
It will implement the MessageConsumer interface. We
use a session to create a MessageConsumer for a
destination, queue or topic object.

BITS Pilani, Pilani Campus

Enterprise Application Integration (EAI)
 Message Channels – Example (JMS)

 Configuring Message channels (Queues and Topics)

 Accessing Message channels

BITS Pilani, Pilani Campus

Enterprise Application Integration (EAI)
 Point to point channel – Example (JMS)
 Sending message

 Receiving message

BITS Pilani, Pilani Campus

Publish Subscribe Channel
 An application is using Messaging to announce events.
 How can the sender broadcast an event to all interested receivers?
 There are well-established patterns for implementing broadcasting.
 The Observer pattern provides event notification to all interested observers no matter how
many observers there are (even none).
 The Publisher-Subscriber pattern [POSA] expands upon Observer by adding the notion of
an event channel for communicating event notifications.
 That’s the theory, but how does it work with messaging?

 Next Slide answers this ……..

BITS Pilani, Pilani Campus

Publish Subscribe Channel
 The event can be packaged as a Message so
that messaging will reliably communicate the
event to the observers (subscribers). Then the
event channel is a Message Channel.
 Each subscriber needs to be notified of a
particular event once, but should not be
notified repeatedly of the same event. The
event cannot be considered consumed until all
of the subscribers have been notified.
 But once all of the subscribers have been
notified, the event can be considered
consumed and should disappear from the
channel.
 Send the event on a Publish-Subscribe
Channel, which delivers a copy of a particular
event to each receiver.
 Pictorial representation follows …..
BITS Pilani, Pilani Campus
Publish Subscribe Channel
 A Publish-Subscribe Channel works like this:
 It has one input channel that splits into multiple output channels, one for each
subscriber.
 When an event is published into the channel, the Publish-Subscribe Channel delivers a
copy of the message to each of the output channels.
 Each output channel has only one subscriber, which is only allowed to consume a
message once.
 In this way, each subscriber only gets the message once and consumed copies disappear
from their channels.

 A Publish-Subscribe Channel can be a useful debugging tool.

 Even though a message is destined to only a single receiver, using a Publish-Subscribe
Channel allows you to eavesdrop on a message channel without disturbing the existing
message flow.
 Monitoring all traffic on a channel can be tremendously helpful when debugging
messaging applications.
BITS Pilani, Pilani Campus
Publish Subscribe Channel
 The ability to eavesdrop on a Publish-Subscribe Channel can also turn into a
disadvantage.
 If your messaging solution transmits payroll data between the payroll system and the
accounting system, you may not want to allow anyone to write a simple program to listen
to the message traffic.
 Point-to-Point Channels alleviate the problem to some extent because the eavesdropper
would consume the messages and the situation would be detected very quickly.
 So subscribing to a Message Channel is an operation that should be restricted by security
policies.
 Creating a monitoring tool that logs active subscribers to Message Channels can be a
useful systems management tool.
 An Event Message is usually sent on a Publish-Subscribe Channel because multiple
dependents are often interested in an event.
 A subscriber can be durable or non-durable.
 If notifications should be acknowledged by the subscribers, use Request-Reply,
 where the notification is the request and the acknowledgement is the reply.
BITS Pilani, Pilani Campus
Publish Subscribe Channel
 Publish - subscribe channel
 Used for broadcasting messages to each
recipient in the channel
 Ensures that each recipient gets only one copy
of the message
 Helps in debugging channel without
disturbing traffic
 JMS Implementation uses ‘Topic’ interface to
implement Point to point channel
 TopicPublisher – send messages
 TopicSubscriber – receive messages
BITS Pilani, Pilani Campus
Publish Subscribe Channel
 Publish - subscribe channel – Example (JMS)
 Sending message

 Receiving message

BITS Pilani, Pilani Campus

Channel Adapter
 Many enterprises use Messaging to integrate multiple,
disparate applications.
 How can you connect an application to the messaging system
so that it can send and receive messages?
 Use a Channel Adapter that can access the application's API
or data and publish messages on a channel based on this
data, and
 Likewise can receive messages and invoke functionality
inside the application.
 The adapter acts as a messaging client to the messaging
system and invokes applications functions via an
application-supplied interface.
 This way, any application can connect to the messaging
system and be integrated with other applications as long as it
has a proper Channel Adapter.
 The Channel Adapter can connect to different layers of the
application's architecture, depending on that architecture
and the data the messaging system needs to access.
BITS Pilani, Pilani Campus
Message Bridge
 Messaging Bridge
 An enterprise is using Messaging to enable applications to
communicate.
 However, the enterprise uses more than one messaging
system, which confuses the issue of which messaging system
an application should connect to.

 How can multiple messaging systems be connected so that

messages available on one are also available on the others?
 What is needed is a way for messages on one messaging
system that are of interest to applications on another Use a message bridge, a connection
messaging system to be made available on the second between messaging systems to
messaging system as well. replicate messages between
systems
 Use a Messaging Bridge, a connection between messaging
systems, to replicate messages between systems.

BITS Pilani, Pilani Campus

Message Bus
 An enterprise contains several existing systems that must be
able to share data and operate in a unified manner in
response to a set of common business requests.
 What is an architecture that enables separate applications to
work together, but in a decoupled fashion such that
applications can be easily added or removed without affecting
the others?
 An enterprise often contains a variety of applications that
operate independently, yet need to work together in a unified
manner.
 Enterprise Application Integration (EAI) describes a solution
to this problem but doesn’t describe how to accomplish it.
 For example, consider an insurance company that sells
different kinds of insurance products (life, health, auto,
home, etc.).
 As a result of corporate mergers, and of the varying winds of
change in IT development, the enterprise consists of a
number of separate applications for managing the company’s
various products.
BITS Pilani, Pilani Campus
Message Bus
 An insurance agent trying to sell a customer several different
types of policies must log into a separate system for each
policy, wasting effort and increasing the opportunity for
mistakes.
 The agent needs a single, unified application for selling
customers a portfolio of policies

 What is needed is an integration architecture that enables

the product applications to coordinate in a loosely coupled
way, and for user applications to be able to integrate with
them.
 Structure the connecting middleware between these
applications as a Message Bus that enables them to work
together using messaging.

 A Message Bus is a combination of a common data model, a

common command set, and a messaging infrastructure to
allow different systems to communicate through a shared set
of interfaces.
BITS Pilani, Pilani Campus
Message Construction - Introduction
 When two applications wish to exchange a piece of data, they do so by wrapping it in a
message.
 Whereas a Message Channel cannot transmit raw data per se, it can transmit the data
wrapped in a message.
 Deciding to create a Message and send it raises several other issues:
 Message intent — Messages are ultimately just bundles of data, but the sender can have
different intentions for what it expects the receiver to do with the message.
 It can send a "Command Message, Document Message, Event Message, Request-Reply
Message".
 Returning a response — When an application sends a message, it often expects a response
confirming that the message has been processed and providing the result.
 Huge amounts of data — Sometimes applications want to transfer a really large data
structure, one that may not fit comfortably in a single message. In this case, break the
data into more manageable chunks and send them as a "Message Sequence".
 Slow messages — A concern with messaging is that the sender often does not know how
long it will take for the receiver to receive the message.
BITS Pilani, Pilani Campus
Message Construction - Introduction
 Yet the message contents may be time-sensitive, such that if the message isn’t received by
a deadline, it should just be ignored and discarded.
 In this situation, the sender can use "Message Expiration" to specify an expiration date.
 Now let us go into the details of different types Message construction :

1. Command Message
2. Document Message
3. Event Message
4. Request-Reply
5. Return Address
6. Correlation Identifier
7. Message Sequence
8. Format Identifier
9. Message Expiration

BITS Pilani, Pilani Campus

Message Construction – Command Message
 How can messaging be used to invoke a procedure in another
application?
 An application needs to invoke functionality provided by
other applications.
 The advantage of Remote Procedure Invocation is that it’s
synchronous, if the network is down then the call doesn’t
work.
 If the call were asynchronous, it could keep trying until the
procedure in the remote application is successfully invoked.
 Messaging is asynchronous.
 How to make a procedure call into a message.
 There’s a well-established pattern for how to encapsulate a
request as an object. Use a command message to reliably
 The Command pattern shows how to turn a request into an invoke a procedure in another
object that can be stored and passed around. application
 The object could be stored in and passed around through a
Message Channel.
 Use a Command Message to reliably invoke a procedure in
another application. BITS Pilani, Pilani Campus
Message Construction – Document Message
 How can reliably Transfer a data structure between
applications?
 A Document Message just passes data and lets the receiver
decide what, if anything, to do with the data.
 The important part of a Document Message is its content, the
document.
 Successfully transferring the document is important; the
timing is less important.
 Guaranteed Delivery may be a consideration.
 Message Expiration probably is not a consideration.
 A Document Message can be any kind of message in the
messaging system.
 In JMS, the document message may be an ObjectMessage Use a Document message to
containing a Serializable data object for the document, or reliably transfer a data structure
 A TextMessage containing the data in XML form. between applications
 In .NET, a document message is a Message with the data
stored in it.
 A Simple Object Access Protocol (SOAP) reply message is a
document message. BITS Pilani, Pilani Campus
Message Construction – Event Message
 How can you achieve reliable asynchronous event
modification between application?
 For example, in a B2B system, one business may need to
notify others of price changes or a whole new product
catalog. It would be better to send the event notification
asynchronously, as a Message. When a subject has an event
to announce, it will create an event object, wrap it in a
message, and send it on a channel.
 The observer will receive the event message, get the event,
and process it. Messaging does not change the event
notification, just makes sure that the notification gets to the
observer.
 A Event Message can be any kind of message in the
messaging system. Our B2B example could use Event Use an Event message to reliable,
Messages, Document Messages, or a combination of both asynchronous event notification
between applications
 If a message says that the price for computer disk drives has
changed, that’s an event. If the message provided information
about the disk drive, including its new price, that’s a
document being sent as an event.
BITS Pilani, Pilani Campus
Message Construction – Event Message
 The Event Messages can be designed using either a Push or Pull

 Push model – The message is a combined document/event message;

 The message’s delivery announces that the state has occurred and the message’s contents are the
new state.
 This is more efficient if all observers want these details, but otherwise can be the worst.

 Pull model – There are three messages:

 Update – An Event Message that notifies the observer of the event.

 State Request – A Command Message an interested observer uses to request details from the
subject.
 State Reply – A Document Message the subject uses to send the details to the observer.

 The advantage of the pull model is that the update messages are small, only interested observers
request details.

BITS Pilani, Pilani Campus

Message Construction – Request Reply
 When two applications communicate via Messaging, the
communication is one-way.
 The applications may want a two-way conversation.
 When an application sends a message, how can it get a
response from the receiver?
 Messaging provides one-way communication between
applications.
 Messages travel on a Message Channel in one direction, from
the sender to the receiver.
 How can messaging be two-way?
 Perhaps a sender and receiver could share a message
simultaneously.
 A message is first sent, then received, such that the sender
and receiver cannot both access the message at the same
Send a pair of request reply messages,
time. each on its own channel.
 A Message Channel transmits messages in one direction.
 What is needed is a two-way message on a two-way channel.
 Send a pair of Request-Reply messages, each on its own
channel.
BITS Pilani, Pilani Campus
Message Construction – Request Reply
 Request Reply has two participants
 Requestor : Sends a request message and waits for reply message
 Replier: Receives a request message and responds with a reply message

 The request channel can be a Point-to-Point Channel or a Publish-Subscribe Channel.

 The reply channel, on the other hand, is almost always point-to-point because broadcast replies
should only be returned to the requestor.
 Two applications sending requests and replies interests us with what the two messages represent.
 The request is like a method call.
 The reply is one of three possibilities:
 1. Void – Simply notifies the caller that the method has finished so that the caller can proceed.
 2. Result value – A single object that is the method's return value.
 3. Exception – A single exception object indicating that the method aborted before completing
successfully, and indicating why.

 The request should contain a Return Address to tell the replier where to send the reply.
 The reply should contain a Correlation Identifier that specifies which request this reply is for.
BITS Pilani, Pilani Campus
Message Construction – Return Address
 My application is using Messaging to perform a Request-Reply.

 How does a replier know where to send the reply?

 Messages are often thought of as completely independent, such that any sender sends a message on
any channel whenever it likes.
 Messages are often Request-Reply pairs, two messages where the reply message has a one-to-one
correspondence with the request message that caused it.
 Thus the replier that processes the request message must send it on the channel the requestor
expects the reply on.
 Hard coding channels makes the software less flexible and more difficult to maintain.
 A requestor potentially may not want a reply sent back to itself.
 The requestor could have multiple callback processors such that replies for different requests from
the same requestor should be sent to different p
 The reply channel will not necessarily transmit replies back to the requestor

BITS Pilani, Pilani Campus

Message Construction – Return Address
 It’s easier for the request to explicitly specify  A message’s Return Address is analogous to
which reply channel to use. the reply-to field in an e-mail message.
 What is needed is a way for the requestor to tell  When the reply is sent back the channel
the replier where and how to send back a reply. indicated by the Return Address, it may also
 The request message should contain a Return need a Correlation Identifier.
Address that indicates where to send the reply  The Return Address tells the receiver what
message. channel to put the reply message on; the
 This way, the replier does not need to know correlation identifier tells the sender which
where to send the reply, it can just ask the request a reply is for.
request.

BITS Pilani, Pilani Campus

Message Construction – Correlation Identifier
 My application is using Messaging to perform a  There are a couple of approaches the caller
Request-Reply and has received a reply message. can use to avoid this confusion.
 How does a requestor that has received a reply  What the caller needs is for the reply message
know which request this is the reply for? to have a pointer or reference to the request
 Messaging is asynchronous, it makes the call, message.
then sometime later a result appears.  A unique identifier could be used to identify
 The caller may not even remember making the the message from other messages, clients that
request, or may have made so many requests use the message, etc.
that it no longer knows which one this is the  Each reply message should contain a
result for. Correlation Identifier, a unique identifier that
 When the caller finally gets the result, it may not indicates which request message this reply is
know what to do with it, which defeats the for.
purpose of making the call.

BITS Pilani, Pilani Campus

Message Construction – Correlation Identifier
 There are six parts to Correlation Identifier:
 1. Requestor — An application that performs a business task by sending a request and waiting for a
reply.
 2. Replier — Another application that receives the request, fulfills it, then sends the reply. It gets the
request ID from the request and stores it as the correlation ID in the reply.
 3. Request — A Message sent from the requestor to the replier containing a request ID.
 4. Reply — A Message sent from the replier to the requestor containing a correlation ID.
 5. Request ID — A token in the request that uniquely identifies the request.
 6. Correlation ID — A token in the reply that has the same value as the request ID in the request.

BITS Pilani, Pilani Campus

Message Construction – Message Sequence
 My application needs to send a huge amount of data to another process, more than may fit in a
single message.

 How can messaging transmit an arbitrarily large amount of data?

 One approach is if the large amount of data is the result of a request, the caller could issue multiple
requests, one for each result chunk, but that assumes the caller even knows how many result
chunks will be needed.
 The receiver could listen for data chunks until there are not anymore , then try to figure out how to
reassemble the chunks into the original, large piece of data, but that would be error-prone.
 Inspiration comes from the way a mail order company sometimes ships an order in multiple boxes.
 If there are three boxes, the shipper will mark them as “1 of 3,” “2 of 3,” and “3 of 3” so that the
receiver will know which ones he’s received and whether he has all of them.
 The trick is to apply the same technique to messaging.

BITS Pilani, Pilani Campus

Message Construction – Message Sequence
 Whenever a large set of data may need to be broken into message-size chunks, send the data as a
Message Sequence and mark each message with sequence identification fields.
 The three Message Sequence identification fields are:

 1. Sequence identifier – Distinguishes this cluster of messages from others.

 2. Position identifier – Uniquely identifies and sequentially orders each message in a sequence.
 3. Size or End indicator – Specifies the number of messages in the cluster, or marks the last message
in the cluster (whose position identifier then specifies the size of the cluster).

 Design the sequences such that each message indicates whether it is the last message in that
sequence.

BITS Pilani, Pilani Campus

Message Construction – Message Expiration
 My application is using Messaging. If a Message’s data is not received by a certain time, it should be
ignored.

 How can a sender indicate when a message should be considered stale and thus shouldn’t be
processed?

 Messaging practically guarantees that the Message will eventually be delivered to the receiver.
 What it cannot guarantee is how long the delivery may take.
 Messaging is highly reliable, even when the participants sender, network, and receiver are not
 But messages can take a very long time to transmit in unreliable circumstances.
 Often, a message’s contents have a practical limit for how long they’re useful.
 A caller issuing a stock quote request will lose interest if it does not receive an answer within a
minute.
 That means the request should not take more than a minute to transmit
 A stock quote reply more than a minute or two old is probably too old and therefore unreliable.

BITS Pilani, Pilani Campus

Message Construction – Message Expiration

 How does the receiver know which

messages to reject?
 What is needed is a way for the
sender to specify the message’s
lifetime.
 Set the Message Expiration to
specify a time limit how long the
message is viable

BITS Pilani, Pilani Campus

Message Construction – Format Indicator
 Several applications are communicating via Messages that follow an agreed upon data format,
perhaps an enterprise-wide Canonical Data Model.
 However, that format may need to change over time.
 How can a message’s data format be designed to allow for possible future changes?
 Even when you design a data format that works for all participating applications, future
requirements may change.
 New applications may be added that have new format requirements, new data may need to be added
to the messages, or developers may find better ways to structure the same data.
 Design a data format that includes a Format Indicator, so that the message specifies what format it
is using.
 The format indicator enables the sender to tell the receiver the format of the message.
 This way, a receiver expecting several possible formats knows the format of the message’s contents.
 The format indicator enables the sender to tell the
receiver the format of the message
 This handles versioning and future changes
 Format indicator can be implemented as:
 Version Number
 Foreign Key
 Format Document
BITS Pilani, Pilani Campus
Case Study

BITS Pilani, Pilani Campus

Widget-Gadget Corp -- An Example
 The best way to understand message-based integration
solutions is by walking through a concrete example.
 Let’s consider Widgets & Gadgets ‘R Us (WGRUS), an
on-line retailer that buys widgets and gadgets from
manufacturers and resells them to customers.
 The solution needs to support the following
requirements.
 Take Orders: Customers can place orders via Web,
phone or fax
 Process Orders: Processing an order involves multiple
steps, including verifying inventory, shipping the goods
and invoicing the customer
 Check Status: Customers can check the order status
 Change Address: Customers can use a Web front-end
to change their billing and shipping address
 New Catalog: The suppliers update their catalog
periodically. WGRUS needs to update its pricing and
availability based in the new catalogs.
BITS Pilani, Pilani Campus
Widget-Gadget Corp -- An Example
 Announcements: Customers can subscribe to selective
announcements from WGRUS.
 Testing and Monitoring: The operations staff needs to
be able to monitor all individual components and the
message flow between them.
 We will tackle each of these requirements separately and
describe the solution alternatives and trade-offs using
the pattern language
 Like in most integration scenarios, WGRUS is not a so-
called “green field” implementation, but rather the
integration of an existing IT infrastructure comprised of
a variety of packaged and custom applications.
 The fact that we have to work with existing applications
often makes integration work challenging. In our
example WGRUS runs the following systems

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Taking order
 The first function we want to implement is taking orders.
 The first step to streamlining order processing is to unify taking orders.

 A customer can place orders over one of three channels:

 Web site, call center or fax.
 Each system is based on a different technology and stores incoming orders in a different data
format.
 The call center system is a packaged application while the Web site is a custom J2EE application.
 The inbound fax system requires manual data entry into a small Microsoft Access application.
 We want to treat all orders equally, regardless of their source

 The packaged call center application and the inbound fax application was not developed with
integration in mind, we connect it to the messaging system using a Channel Adapter.
 A Channel Adapter is a component that can attach to an application and publish messages to a
Message Channel whenever an event occurs inside the application
 Because the Web application is custom-built we implement the Endpoint code inside the
application.
BITS Pilani, Pilani Campus
Widget-Gadget Corp – Taking order
 Because each system uses a different data format for the incoming orders we use three Message
 Translators to convert the different data formats into a common New Order message that follows a
Canonical Data Model.
 A Canonical Data Model defines message formats that are independent from any specific
application so that all applications can communicate with each other in this common format
 The NEW_ORDER Message Channel is a Datatype Channel
because it carries messages of only one type, i.e. new
orders. This makes it easy for message consumers to know
what type of message to expect.
 We name application-specific Message Channels starting
with the name of the application
 e.g. WEB_NEW_ORDER.
 Channels carrying canonical messages are named after the
intent of the message without any prefix,
 e.g. NEW_ORDER.
 We connect each Channel Adapter to the Message
Translator via a Point-to-Point Channel because we want to
be sure that each order message is consumed only once
BITS Pilani, Pilani Campus
Widget-Gadget Corp – Processing orders
 Now that we have a consistent order message that is independent from the message source we need
to process orders.
 In order to fulfill an order we need to complete the following steps:
 Verify the customer’s credit standing. If the customer has outstanding bills, we want to reject the
new order.
 Verify inventory. We can’t fulfill orders for items that are not in stock.
 If the customer is in good standing and we have inventory we want to shop the goods and bill the
customer.
 We can express this sequence of events using a UML activity diagram.

 The activity diagram executes the “Check Inventory” task and the “Verify
Customer Standing” task in parallel.
 The join bar waits until both activities are completed before it allows the next
activity to start.
 The next activity verifies the results of both steps – do we have inventory and
is the customer in good standing?
 If both conditions are fulfilled, the process goes on to fulfill the order.
 Otherwise, we transition to an exception handling activity. For example, we
may call the customer to remind them to pay the last invoice or send an e-
mail letting him or her know that the order will be delayed.

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Processing orders
 To convert the logical activity diagram into an integration design, we use a
 Publish-Subscribe Channel to implement the fork action and an Aggregator to implement the join
action.
 A Publish-Subscribe Channel sends a message to all active consumers while an Aggregator receives
multiple incoming messages and combines them into a single outgoing message

The Aggregator combines the results from both messages and

passes the resulting message to a Content-Based Router.

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Processing orders
 WGRUS has two inventory systems, one for widgets and one for gadgets.
 As a result, we have to route the request for inventory to the correct system.
 As we want to hide the peculiarities of the inventory systems we insert a Content-Based Router that
routes the message to the correct system based on the type of item ordered
 For example, all incoming messages with an item number starting with ‘W’ are routed to the widget
inventory system and all orders with an item number starting with ‘G’ are routed to the gadget
inventory system.

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Checking Status
 In order to ascertain the status of the order in the sequence of steps we would have to know the
“last” message related to this order.
 One of the advantages of a Publish-Subscribe Channel is that we can add additional subscribers
without disturbing the flow of messages.
 We can use this property to listen in to new and validated orders and store them into a Message
Store.
 We could then query the Message Store database for the status

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Change Address
 WGRUS needs to deal with a number of addresses.
 For example, the invoice has to be sent to the customer’s billing address while the goods are
shipped to the shipping address.
 We want to allow the customer to maintain all these addresses through the Web Interface to
eliminate unnecessary manual steps.

 We can choose between two basic approaches to get the correct billing and shipping addresses to
the billing and shipping systems:
 Include address data with the New Order message
 Replicate address data to other systems

 We update the address directly into the system database using a database Channel Adapter.
 Sending the goods or producing an invoice has to invoke the applications’ business logic.
 Because we are dealing with multiple types of addresses (shipping and billing addresses) we need to
make sure that only the right type of address is stored in each system.
 We accomplish this by using Message Filters that only pass messages matching certain criteria

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Change Address
 We use Message Translators to translate the generic Address Change message into the specific
message format used by the applications.
 Both the shipping and the billing system store addresses in a relational database so that we use a
database Channel Adapter to update the data in each system.

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Updating Catalog

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Sending Announcements

BITS Pilani, Pilani Campus

Thank You

73
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
 Middleware Technologies SSWT ZG589

Dr. Rama Satish K V

BITS Pilani Guest Faculty
Pilani Campus
BITS Pilani, Pilani Campus
 IMP Note to Students
 It is important to know that just login to the session does not guarantee the
attendance.
 Once you join the session, continue till the end to consider you as present in
the class.
 IMPORTANTLY, you need to make the class more interactive by responding
to Professors queries in the session.
 Whenever Professor calls your number / name ,you need to respond,
otherwise it will be considered as ABSENT

BITS Pilani, Pilani Campus

Textbooks

T1: Letha Hughes Etzkorn - Introduction to middleware _

web services, object components, and cloud computing-
Chapman and Hall_CRC (2017).

T2: William Grosso - Java RMI (Designing & Building

Distributed Applications)

R1: Gregor Hohpe, Bobby Woolf - Enterprise Integration Patterns_ Designing,

Building, and Deploying Messaging Solutions -Addison-Wesley Professional
(2003)

R2: MongoDB in Action

Note: In order to broaden understanding of concepts as applied to Indian IT industry, students are advised to refer books of their choice and case-studies in their own organizations
BITS Pilani, Pilani Campus
Modular Structure

No Title of the Module

M1 Introduction and Evolution

M2 Enterprise Middleware

M3 Middleware Design and Patterns

M4 Middleware for Web-based Application and Cloud-based Applications

M5 Specialized Middleware

BITS Pilani, Pilani Campus

 BITS Pilani
Pilani Campus

CS6: Message Routing and Message Transformations

Agenda

 Message Routers
 Message Transformations

BITS Pilani, Pilani Campus

 Message Routers
Below are list of routers types available

 Content-Based Router
 Message Filter
 Dynamic Router
 Recipient List
 Splitter
 Aggregator
 Re sequencer
 Composed Message Processor
 Scatter-Gather
 Routing Slip
 Process Manager
 Message Broker

BITS Pilani, Pilani Campus

Message Routers
 Content and Dynamic Router
 Content Routers inspects the content of the message and passes on the message to
successive stage if the content of the message is correct. A message filter filters out
messages with wrong content.
 Usage of Dynamic routers with Content Routers improves the flexibility of transmission
Dynamic Routers allows Routing logic to be varied during transmission A recipient list is a
Content Router which can send the information to more then one destination.

 Composed Router
 Combines multiple Message Router Variants to obtain more comprehensive Solutions.
 Routing Slip specifies the path the message should take.
 Message Broker Architectural Pattern can be used to obtain solution to architect Message
Routers.

BITS Pilani, Pilani Campus

Message Routers
 Right Router for Correct purpose

BITS Pilani, Pilani Campus

Message Routers

 Content Based Router

BITS Pilani, Pilani Campus

Message Routers - Publishing and
Subscribing to TIBCO Rendezvous Messages
 Rendezvous programs use messages as the
common currency to exchange data. The
Rendezvous palette includes activities that
allow you to easily setup subjects to send and
receive messages. This sample shows how to
use Rendezvous activities
 Prerequisites
 TIBCO Rendezvous must be running on the
machine
 To run this sample you can either use TIBCO
ActiveMatrix BusinessWorks 6.x client or
TIBCO ActiveMatrix BusinessWorks 5.x as a
client
 Refer to Configure the Client in TIBCO TIBCO Message Broker
Designer section to configure client in TIBCO
ActiveMatrix BusinessWorks 5.x
BITS Pilani, Pilani Campus
Message Routers
 Message Filter

BITS Pilani, Pilani Campus

Message Routers
 Message Filter with Broadcast Channel

BITS Pilani, Pilani Campus

Message Routers
 Dynamic Router

BITS Pilani, Pilani Campus

Message Routers
 Recipient List

BITS Pilani, Pilani Campus

Message Routers
 Recipient List with Bus

BITS Pilani, Pilani Campus

Message Routers
 Splitter

BITS Pilani, Pilani Campus

 Message Routers
 Aggregator

BITS Pilani, Pilani Campus

Message Routers
 Aggregator with channels

BITS Pilani, Pilani Campus

 Message Routers
 Aggregator other options

BITS Pilani, Pilani Campus

Message Routers
 Composed Message Processor

BITS Pilani, Pilani Campus

Message Routers
 Composed Message Processor

BITS Pilani, Pilani Campus

Message Routers
 Scatter – Gather Router

BITS Pilani, Pilani Campus

Message Routers
 Routing Slip

BITS Pilani, Pilani Campus

Message Routers
 Central Process Unit, a  Message Broker
process manager to maintain
the state of the sequence and
determine the next processing
step based on intermediate
results

BITS Pilani, Pilani Campus

Message Transformations
Below are list of transformations types available
 Envelope Wrapper
 Content Enricher
 Content Filter
 Claim Check
 Normalizer
 Canonical Data Model

BITS Pilani, Pilani Campus

Message Transformations
 Metadata Integration

BITS Pilani, Pilani Campus

Message Transformations

 Wrapping and unwrapping

BITS Pilani, Pilani Campus

Message Transformations
 Envelope structure

BITS Pilani, Pilani Campus

Message Transformations - Example

BITS Pilani, Pilani Campus

Message Transformations - Example

BITS Pilani, Pilani Campus

Message Transformations

 Content Enricher

BITS Pilani, Pilani Campus

Message Transformations
 Adapter

BITS Pilani, Pilani Campus

Message Transformations
 Claim Check

BITS Pilani, Pilani Campus

 Message Transformations
 Process Manager with Claim Check

BITS Pilani, Pilani Campus

Message Transformations
 Normalizer with Message Translator

BITS Pilani, Pilani Campus

Message Transformations

 Canonical Data Model

BITS Pilani, Pilani Campus

Message Transformations

 Canonical Data Model

BITS Pilani, Pilani Campus

Case Study

BITS Pilani, Pilani Campus

Widget-Gadget Corp -- An Example

 The best way to understand message-based integration

solutions is by walking through a concrete example.
 Let’s consider Widgets & Gadgets ‘R Us (WGRUS), an
on-line retailer that buys widgets and gadgets from
manufacturers and resells them to customers.
 The solution needs to support the following
requirements.
 Take Orders: Customers can place orders via Web,
phone or fax
 Process Orders: Processing an order involves multiple
steps, including verifying inventory, shipping the goods
and invoicing the customer
 Check Status: Customers can check the order status
 Change Address: Customers can use a Web front-end
to change their billing and shipping address
 New Catalog: The suppliers update their catalog
periodically. WGRUS needs to update its pricing and
availability based in the new catalogs.
BITS Pilani, Pilani Campus
Widget-Gadget Corp -- An Example
 Announcements: Customers can subscribe to selective
announcements from WGRUS.
 Testing and Monitoring: The operations staff needs to
be able to monitor all individual components and the
message flow between them.
 We will tackle each of these requirements separately and
describe the solution alternatives and trade-offs using
the pattern language
 Like in most integration scenarios, WGRUS is not a so-
called “green field” implementation, but rather the
integration of an existing IT infrastructure comprised of
a variety of packaged and custom applications.
 The fact that we have to work with existing applications
often makes integration work challenging. In our
example WGRUS runs the following systems

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Taking order
 The first function we want to implement is taking orders.
 The first step to streamlining order processing is to unify taking orders.

 A customer can place orders over one of three channels:

 Web site, call center or fax.
 Each system is based on a different technology and stores incoming orders in a different data
format.
 The call center system is a packaged application while the Web site is a custom J2EE application.
 The inbound fax system requires manual data entry into a small Microsoft Access application.
 We want to treat all orders equally, regardless of their source

 The packaged call center application and the inbound fax application was not developed with
integration in mind, we connect it to the messaging system using a Channel Adapter.
 A Channel Adapter is a component that can attach to an application and publish messages to a
Message Channel whenever an event occurs inside the application
 Because the Web application is custom-built we implement the Endpoint code inside the
application.
BITS Pilani, Pilani Campus
Widget-Gadget Corp – Taking order
 Because each system uses a different data format for the incoming orders we use three Message
 Translators to convert the different data formats into a common New Order message that follows a
Canonical Data Model.
 A Canonical Data Model defines message formats that are independent from any specific
application so that all applications can communicate with each other in this common format
 The NEW_ORDER Message Channel is a Datatype Channel
because it carries messages of only one type, i.e. new
orders. This makes it easy for message consumers to know
what type of message to expect.
 We name application-specific Message Channels starting
with the name of the application
 e.g. WEB_NEW_ORDER.
 Channels carrying canonical messages are named after the
intent of the message without any prefix,
 e.g. NEW_ORDER.
 We connect each Channel Adapter to the Message
Translator via a Point-to-Point Channel because we want to
be sure that each order message is consumed only once
BITS Pilani, Pilani Campus
Widget-Gadget Corp – Processing orders
 Now that we have a consistent order message that is independent from the message source we need
to process orders.
 In order to fulfill an order we need to complete the following steps:
 Verify the customer’s credit standing. If the customer has outstanding bills, we want to reject the
new order.
 Verify inventory. We can’t fulfill orders for items that are not in stock.
 If the customer is in good standing and we have inventory we want to shop the goods and bill the
customer.
 We can express this sequence of events using a UML activity diagram.

 The activity diagram executes the “Check Inventory” task and the “Verify
Customer Standing” task in parallel.
 The join bar waits until both activities are completed before it allows the next
activity to start.
 The next activity verifies the results of both steps – do we have inventory and
is the customer in good standing?
 If both conditions are fulfilled, the process goes on to fulfill the order.
 Otherwise, we transition to an exception handling activity. For example, we
may call the customer to remind them to pay the last invoice or send an e-
mail letting him or her know that the order will be delayed.

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Processing orders
 To convert the logical activity diagram into an integration design, we use a
 Publish-Subscribe Channel to implement the fork action and an Aggregator to implement the join
action.
 A Publish-Subscribe Channel sends a message to all active consumers while an Aggregator receives
multiple incoming messages and combines them into a single outgoing message

The Aggregator combines the results from both messages and

passes the resulting message to a Content-Based Router.

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Processing orders
 WGRUS has two inventory systems, one for widgets and one for gadgets.
 As a result, we have to route the request for inventory to the correct system.
 As we want to hide the peculiarities of the inventory systems we insert a Content-Based Router that
routes the message to the correct system based on the type of item ordered
 For example, all incoming messages with an item number starting with ‘W’ are routed to the widget
inventory system and all orders with an item number starting with ‘G’ are routed to the gadget
inventory system.

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Checking Status
 In order to ascertain the status of the order in the sequence of steps we would have to know the
“last” message related to this order.
 One of the advantages of a Publish-Subscribe Channel is that we can add additional subscribers
without disturbing the flow of messages.
 We can use this property to listen in to new and validated orders and store them into a Message
Store.
 We could then query the Message Store database for the status

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Change Address
 WGRUS needs to deal with a number of addresses.
 For example, the invoice has to be sent to the customer’s billing address while the goods are
shipped to the shipping address.
 We want to allow the customer to maintain all these addresses through the Web Interface to
eliminate unnecessary manual steps.

 We can choose between two basic approaches to get the correct billing and shipping addresses to
the billing and shipping systems:
 Include address data with the New Order message
 Replicate address data to other systems

 We update the address directly into the system database using a database Channel Adapter.
 Sending the goods or producing an invoice has to invoke the applications’ business logic.
 Because we are dealing with multiple types of addresses (shipping and billing addresses) we need to
make sure that only the right type of address is stored in each system.
 We accomplish this by using Message Filters that only pass messages matching certain criteria

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Change Address
 We use Message Translators to translate the generic Address Change message into the specific
message format used by the applications.
 Both the shipping and the billing system store addresses in a relational database so that we use a
database Channel Adapter to update the data in each system.

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Updating Catalog

BITS Pilani, Pilani Campus

Widget-Gadget Corp – Sending Announcements

BITS Pilani, Pilani Campus

Summary

• Message Routers
• Message Transformations
• Widget-Gadget Corp -- Case Study

BITS Pilani, Pilani Campus

Thank You

54
BITS Pilani, Pilani Campus
BITS Pilani, Pilani Campus
 Middleware Technologies-CSIZG524
Contact Session-07
22nd 23rd 24th August 2025
BITS Pilani
Pilani Campus
Topic: Messaging Systems
 Middleware Technologies SSWT ZG589

Dr. Rama Satish K V

Guest Faculty
BITS Pilani
Pilani Campus satishkvr@wilp.bits-pilani.ac.in
IMP Note to Self

BITS Pilani, Pilani Campus

 IMP Note to Students
 It is important to know that just login to the session does not guarantee the
attendance.
 Once you join the session, continue till the end to consider you as present in
the class.
 IMPORTANTLY, you need to make the class more interactive by responding
to Professors queries in the session.
 Whenever Professor calls your number / name ,you need to respond,
otherwise it will be considered as ABSENT

BITS Pilani, Pilani Campus

Evaluation Components

Evaluation Name Type Weight Duration Schedule

Component
EC – 1 Quiz I, II & Individual / 10% & Pre/Post
Quiz III Contact session 10% Mid-Sem
EC – 2 Mid-Semester Closed Book 30% 2 Hrs. TBA
Examination

EC – 3 End-Semester Open Book 50% 3 Hrs. TBA

Examination

BITS Pilani, Pilani Campus

• The slides presented here are obtained from the authors of the books and from
various other contributors.

• I hereby acknowledge all the contributors for their material and inputs.

• I have added and modified a few slides to suit the requirements of the course.

BITS Pilani, Pilani Campus

Textbooks

T1: Letha Hughes Etzkorn - Introduction to middleware _

web services, object components, and cloud computing-
Chapman and Hall_CRC (2017).

T2: William Grosso - Java RMI (Designing & Building

Distributed Applications)

R1: Gregor Hohpe, Bobby Woolf - Enterprise Integration Patterns_ Designing,

Building, and Deploying Messaging Solutions -Addison-Wesley Professional
(2003)

R2: MongoDB in Action

Note: In order to broaden understanding of concepts as applied to Indian IT industry, students are advised to refer books of their choice and case-studies in their own organizations
BITS Pilani, Pilani Campus
Modular Structure

No Title of the Module

M1 Introduction and Evolution

M2 Enterprise Middleware

M3 Middleware Design and Patterns

M4 Middleware for Web-based Application and Cloud-based Applications

M5 Specialized Middleware

BITS Pilani, Pilani Campus

 BITS Pilani
Pilani Campus

CS7: EAI: Simple and Composed Messaging

Agenda

 Messaging Endpoints.
 Simple Messaging.
 Composed Messaging.
 How to compose routing and transformation patterns into a larger
solution.
 Overview of Messaging Patterns.

BITS Pilani, Pilani Campus

Message Endpoints
 Connector – Links an application with a messaging channel.
 Translator – Converts application commands/data into messages and
vice versa.
 Custom Code – Depends on both the application logic and the
messaging system’s API.
 Responsibilities – Sends, receives, encodes, and decodes messages
meaningfully.
 Related Patterns – Channel Adapter, Messaging Gateway, Service
Activator, Event-Driven Consumer, Idempotent Receiver.

BITS Pilani, Pilani Campus

Integrate Applications using Messaging
 So far, we’ve introduced a lot of patterns. We’ve seen the basic Messaging
Components, such as Message Channel, Message, and Message Endpoint.
We’ve also seen detailed patterns for Messaging Channels and for Message
Construction.
 So how do all of these patterns fit together? How does a developer integrate
applications using these patterns? What does the code look like, and how does
it work?
 Simple Messaging with illustrative Patterns
 This is a simple but powerful example, transmitting a request and transmitting
back a reply. It consists of two main classes:
 Requestor — The object that sends the request message and expects to receive
the reply message.
 Replier — The object that receives the request message and sends a reply
message in response.
 Message Channel and Point-to-Point Channel — One channel for transmitting
the requests, another for transmitting the replies.
BITS Pilani, Pilani Campus
Simple Messaging with illustrative Patterns

 Document Message — The default type of message, used as both the request and the reply.
 Request-Reply — A pair of messages sent over a pair of channels, allowing the two
applications to have a two-way conversation.
 Return Address — The channel to send the response on.
 Correlation Identifier — The ID of the request that caused this response.
 Datatype Channel — All of the messages on each channel should be of the same type.
 Invalid Message Channel — What happens to messages that aren’t of the right type.
 Publish/Subscribe — Explores how to use a JMS Topic to implement the Observer pattern.
 JMS Request-Reply Example
 jms/RequestQueue — The Queue the Requestor uses to send
the request message to the Replier.
 jms/ReplyQueue — The Queue the Replier uses to send the
reply message to the Requestor.
 jms/InvalidMessages — The Queue that the Requestor and
Replier move a message to when they receive a message that
they cannot interpret.
 Here's how the example works. When the Requestor is
started in a command-line window, it starts and prints
output like this:
BITS Pilani, Pilani Campus
JMS Request Reply Example
 When the Replier is started in another command-line window, it starts
and prints output like this:

 This shows that the Replier received the request message and sent a
reply message.
 Observe the following:
 MessageID same hence same message.
 TimeStamp in Receiver is 30270 later.
 Contents Same.(Hello world.) BITS Pilani, Pilani Campus
JMS Request Reply Java Code.
( Initialization)

BITS Pilani, Pilani Campus

JMS Request Reply Java Code.
(Send/Receive)

BITS Pilani, Pilani Campus

JMS Publish/Subscriber Example
Definition – A Publish–Subscribe Channel allows a sender (publisher) to broadcast an
event/message once, and multiple receivers (subscribers) can get it simultaneously.
Efficiency – The publisher sends the event only once, regardless of the number of
subscribers.
Flexibility – Multiple subscriber applications can process the same event independently.
Relation to Observer Pattern – It’s like the Observer pattern, but distributed across
multiple applications rather than within one program.
Alternative Strategies – Variations exist for how event details are communicated to
subscribers (e.g., full event, filtered, or just references).

BITS Pilani, Pilani Campus

JMS Publish/Subscriber Observer Pattern

 Observer provides two ways to get the new state from the subject to the observer:
 the push model and the pull model.
 With the push model, the Update call to each observer contains the new state as a
parameter. Thus, interested observers can avoid having to call GetState(), but effort is
wasted passing data to uninterested observers.
 The opposite approach is the pull model, where the subject sends basic notification, and
each observer requests the new state from the subject. Thus, each observer can request
the exact details it wants, even none, but the subject often has to serve multiple requests
for the same data.
 The push model requires a single, one-way communication—the Subject pushes the data
to an Observer as part of the update.
 The pull model requires three one-way communications—the Subject notifies an Observer,
the Observer requests the current state from the Subject, and the Subject sends the
current state to the Observer. As we'll see, the number of one-way communications affects
both the design-time complexity and the runtime performance of the notification.

BITS Pilani, Pilani Campus

 JMS Publish/Subscriber Java Code

 A Publish-Subscribe Channel implements the Observer pattern, making the pattern much
easier to use amongst distributed applications. The pattern is implemented in three steps:

 The messaging system administrator creates a Publish-Subscribe Channel. (This will be

represented in Java applications as a JMS Topic.)

 The application acting as the Subject creates a TopicPublisher (a type of MessageProducer)

to send messages on the channel.

 Each of the applications acting as an Observer (e.g., a dependent) creates a

TopicSubscriber (a type of MessageConsumer) to receive messages on the channel. (This is
analogous to calling the Attach(Observer) method in the Observer pattern.)

 This establishes a connection between the subject and the observers through the channel.
Now, whenever the subject has a change to announce, it does so by sending a message.
The channel will ensure that each of the observers receives a copy of this message.
BITS Pilani, Pilani Campus
JMS Publish/Subscriber Java Code.
(Init)

BITS Pilani, Pilani Campus

JMS Publish/Subscriber
(Receive the change notification)

BITS Pilani, Pilani Campus

JMS Publish/Subscriber Java Code

 SubjectGateway is a Messaging Gateway between the Subject (not shown) and the
messaging system. The subject creates the gateway and then uses it to broadcast
notifications.
 Essentially, the subject's Notify() method is implemented to
call SubjectGateway.notify(String). The gateway then announces the change by sending a
message on the update channel.
 Observer Gateway
 Observer Gateway is another Messaging Gateway, this time between the Observer (not
shown) and the messaging system. The observer creates the gateway, then uses attach() to
start the Connection (which is analogous to calling the Attach(Observer) method in the
Observer pattern).
 These two classes implement the push model version of Observer. With the notification
message sent by SubjectGateway.notify(String), the existance of the message tells the
Observer that a change has occurred, but it is the contents of the message that tell the
Observer what the Subject's new state is. The new state is being pushed from the Subject
to the Observer. As we'll see later, there's another way to implement all this using the pull
model.
BITS Pilani, Pilani Campus
JMS Publish/Subscriber Observer Gateway

 The gateway is an Event-Driven Consumer, so it implements the MessageListener

interface, which requires the onMessage method. In this way, when an update is
received, the gateway processes the message to get the new state, and calls is own
update(String) method which calls the corresponding message in the observer.

BITS Pilani, Pilani Campus

Comparisons

 For distributed notification between applications, the Publish-Subscribe (e.g., messaging)

approach has several advantages over the traditional, synchronous (e.g., RPC) approach
of implementing Observer:
 Simplifies Notification — The Subject's implementation of Notify() becomes incredibly
simple; the code just has to send a message on a channel. Likewise, Observer.Update()
just has to receive a message.
 Simplifies Attach/Detach — Rather than attach to and detach from the Subject, an
Observer needs to subscribe to and unsubscribe from the channel. The Subject does not
need to implement Attach(Observer) or Detach(Observer) (although the Observer may
implement these methods to encapsulate the subscribe and unsubscribe behavior).
 Simplifies Concurrent Threading — The Subject only needs one thread to update all
Observers concurrently—the channel delivers the notification message to the Observers
concurrently—and each Observer handles the update in its own thread. This simplifies
the Subject's implementation, and because each Observer uses its own thread, what one
does in its update thread does not affect the others.

BITS Pilani, Pilani Campus

Pub/Sub Advantages

 Publish-Subscribe (e.g., messaging) approach has several advantages over the traditional,
synchronous (e.g., RPC) approach of implementing Observer:
 Simplifies Notification — The Subject's implementation of Notify() becomes incredibly simple; the
code just has to send a message on a channel. Likewise, Observer.Update() just has to receive a
message.
 Simplifies Attach/Detach — Rather than attach to and detach from the Subject, an Observer needs
to subscribe to and unsubscribe from the channel. The Subject does not need to implement
Attach(Observer) or Detach(Observer) (although the Observer may implement these methods to
encapsulate the subscribe and unsubscribe behavior).
 Simplifies Concurrent Threading — The Subject only needs one thread to update all Observers
concurrently—the channel delivers the notification message to the Observers concurrently—and
each Observer handles the update in its own thread. This simplifies the Subject's implementation,
and because each Observer uses its own thread, what one does in its update thread does not affect
the others.
 Increases Reliability — Because the channel uses messaging, notifications will be queued until the
Observer can process them, which also enables the Observer to throttle the notifications. If an
Observer wants to receive notifications that are sent while that Observer is disconnected, it should
make itself a Durable Subscriber.
BITS Pilani, Pilani Campus
Composed Messaging Example

 Design a loan broker system using integration patterns studied.

 List the individual tasks that the loan broker needs to perform.

 Patterns used to implement Loan Broker

 How Broker receives the incoming request.
 We cover this topic in much more detail in the following chapter on endpoint patterns. So
for now, let's skip over this step and assume the message is somehow received by the
broker.
 Now, the broker has to retrieve some additional information, i.e. the customer's credit
score. A Content Enricher sounds like the ideal
choice for this task.
 Once the broker has the complete information,
the broker has to determine the appropriate
banks to route the request message to. We can
accomplish this with another Content
Enricher that computes the list of recipients for
the request. BITS Pilani, Pilani Campus
Simple Loan Broker Design

 Sending a request message to multiple recipients and re-combining the responses back
into a single message is the specialty of the Scatter-Gather.
 The Scatter-Gather can use a Publish-Subscribe Channel or a Recipient List to send the
request to the banks. Once the banks reply with their rate quotes, the Scatter-
Gather aggregates the individual rate quotes into a single quote for the consumer using
an Aggregator.

BITS Pilani, Pilani Campus

Complete Loan Broker Design
 Each bank may use a slightly different message format for the loan request and response.
Since we want to separate the routing and aggregation logic from the banks' proprietary
formats, we need to insert Message Translators into the communication lines between the
broker and the banks. We can use a Normalizer to translate the individual responses into
a common format:

BITS Pilani, Pilani Campus

Synchronous vs Asynchronous

 So far, we have described the flow of the messages and the routing and transformation
patterns we can use to describe the design of the loan broker component. We have not yet
discussed the timing of the broker operation. We have two primary choices:
 Synchronous (Sequential): The broker asks one bank for the quote and waits for a
response.
 Asynchronous (Parallel): The broker sends all quote requests at once and waits for the
answers to come back.

 Synchronous, Sequential Processing of Loan Requests

 UML sequence diagrams to illustrate the two options.
 The synchronous option implies a sequential processing of all loan requests (see picture).
 This solution has the advantage of being simpler to manage because we do not have to
deal with any concurrency issues or threads.
 However, it is an inefficient solution because we do not take advantage of the fact that
each bank possesses independent processing power and could be executing requests
simultaneously. As a result, the consumer might have to wait a long time for an answer..
BITS Pilani, Pilani Campus
Synchronous, Sequential Processing of
Loan Requests

BITS Pilani, Pilani Campus

Asynchronous, Parallel Processing of
Loan Requests
 Another significant advantage of using asynchronous invocation over a message queue is
the ability to create more than one instance of a service.
 What are the observations from the above Sequence Diagram ?
 How can the design be made more performant ?

 For example, if it turns out that the credit bureau is a bottleneck, we could decide to run
two instances of the credit bureau component. Because the loan broker sends the request
message to a queue instead of directly to the credit bureau component it does not matter
which component instance processes the message as long as the response is put back
onto the response channel..

 However, it is an inefficient solution because we do not take advantage of the fact that
each bank possesses independent processing power and could be executing requests
simultaneously. As a result, the consumer might have to wait a long time for an answer..

BITS Pilani, Pilani Campus

Asynchronous, Parallel Processing of
Loan Requests

BITS Pilani, Pilani Campus

Addressing: Distribution vs Auction
 Using a Scatter-Gather to obtain the best quote allows us to choose from two addressing
mechanisms, a Recipient List or a Publish-Subscribe Channel. The decision primarily
depends on how much control do we want to exert over the banks who are allowed to
participate in a specific loan request?

 Again, we have a number of choices:

 Fixed: The list of banks is hard-coded. Each loan request goes to the same set of banks.

 Distribution: The broker maintains criteria on which banks are a good match for a
specific request. For example, it would not send a quote request for a customer with a
poor credit history to a bank that specializes in premier customers.

BITS Pilani, Pilani Campus

Products with Enterprise Integration Patterns
 The patterns are not tied to a specific implementation. They help you design better
solutions, whether you use any of the following platforms:

 EAI and SOA platforms, such as IBM WebSphere MQ, TIBCO, Vitria, Oracle Service Bus,
WebMethods (now Software AG), Microsoft BizTalk, or Fiorano.

 Open source ESB's like Mule ESB, JBoss Fuse, Open ESB, WSo2, Spring Integration, or
Talend ESB

 Message Brokers like ActiveMQ, Apache Kafka, or RabbitMQ

 Web service- or REST-based integration, including Amazon Simple Queue Service (SQS)
or Google Cloud Pub/Sub

 JMS-based messaging systems

 Microsoft technologies like MSMQ or Windows Communication Foundation (WCF)
BITS Pilani, Pilani Campus
 Agenda
⮚Challenges faced
⮚ Development
⮚ Testing

⮚Monitoring and Controlling

⮚ Control Bus
⮚ Detour

⮚Observing and Analyzing Message Traffic

⮚ Wire Tap
⮚ Message History
⮚ Message Store
⮚ Smart Proxy

⮚Testing and Debugging

⮚ Test Message Injection
⮚ Channel purger 35
BITS Pilani, Pilani Campus
 Architect's dream, Developer's nightmare
symptom.
⮚ A message-based integration solution may produce, route and transform thousands or even
millions of messages in a day.
⮚ One has to deal with exceptions, performance bottlenecks and changes in the participating
systems.
⮚ Components are distributed across many platforms and machines that can reside at
multiple locations.
⮚ Architectural benefits of loose coupling actually make testing and debugging a system
harder.
⮚ Assumptions made about each other are less in such system and therefore provide flexibility.
⮚ Complexity for Testers: Testing a system where a message producer is not aware of who
the consumers of this message are, can be challenging.
⮚ Asynchronous nature of messaging make things even more complicated.
⮚ For example, the messaging solution may not even be designed for the message producer
to receive a reply message from the recipient(s).
⮚ Likewise, the messaging infrastructure typically guarantees the delivery of the message
but not the delivery time.
⮚ This makes it hard to develop test cases that rely on the results of the message delivery.
36
BITS Pilani, Pilani Campus
Message Transformations

Below are list of transformations types available

⮚Envelope Wrapper
⮚Content Enricher
⮚Content Filter
⮚Claim Check
⮚Normalizer
⮚Canonical Data Model

37
BITS Pilani, Pilani Campus
 Message Transformations

⮚ Metadata Integration

38
BITS Pilani, Pilani Campus
Message Transformations

⮚ Wrapping and unwrapping

39
BITS Pilani, Pilani Campus
Message Transformations

⮚ Envelope structure

40
BITS Pilani, Pilani Campus
Message Transformations - Example

41
BITS Pilani, Pilani Campus
Message Transformations - Example

42
BITS Pilani, Pilani Campus
Message Transformations - Example

43
BITS Pilani, Pilani Campus
Message Transformations

⮚ Content Enricher • Use a specialized transformer, a Content Enricher, to

access an external data source in order to augment a
message with missing information.

• The Content Enricher uses information inside the incoming

message (e.g. key fields) to retrieve data from an external
source.

• After the Content Enricher retrieves the required data from

the resource, it appends the data to the message.

• The original information from the incoming message may

be carried over into the resulting message or may no
longer be needed, depending on the specific needs of the
receiving application.

44
BITS Pilani, Pilani Campus
Message Transformations

⮚ Content Enricher

45
BITS Pilani, Pilani Campus
Message Transformations

⮚ Adapter

46
BITS Pilani, Pilani Campus
Message Transformations
Store message data in a persistent store and pass a Claim Check to
⮚ Claim Check subsequent components. These components can use the Claim Check to
retrieve the stored information.
The Claim Check pattern consists of the following steps:
1.A message with data arrives.
2.The "Check Luggage" component generates a unique key for the
information. This key will be used later as the Claim Check
3.The Check Luggage component extracts the data from the message and
stores it in a persistent store, e.g. a file or a database. It associates the stored
data with the key .
4.It removes the persisted data from the message and adds the Claim Check.
5.Another component can use a Content Enricher to retrieve the data based
on the Claim Check.
•This process is analogous to a luggage check at the airport. If you do not
want to carry all your luggage with you, you simply check it with the airline
counter.
•In return you receive a sticker on your ticket that has a reference number
that uniquely identifies each piece of luggage you checked.
•Once you reach your final destination, you can retrieve your luggage.
47
BITS Pilani, Pilani Campus
Message Transformations

⮚ Process Manager with Claim Check

48
BITS Pilani, Pilani Campus
Message Transformations
⮚ Normalizer with Message Translator

49
BITS Pilani, Pilani Campus
Message Transformations
⮚ Canonical Data Model

50
BITS Pilani, Pilani Campus
Monitoring and Controlling

⮚ Messages can be tracked at two different levels of Abstraction.

⮚ System Management - Monitors how many messages are being sent or
how long it took a message to be processed.
⮚ These monitoring solutions do not inspect the message data except
maybe for some fields in the message header such as the message
identifier or the Message History.
⮚ Business Activity Monitoring (BAM) - Focus on the payload data
contained in the message, for example the Dollar value of all orders
placed in the last hour.
51
BITS Pilani, Pilani Campus
 Control Bus

⮚ Use a Control Bus to manage an enterprise integration system.

⮚ The Control Bus uses the same messaging mechanism used by the application data, but
uses separate channels to transmit data that is relevant to the management of
components involved in the message flow.
⮚ One can use additional steps, such as validation or logging for troubleshooting.

52
BITS Pilani, Pilani Campus
Detour (Validation, Testing or
Debugging)
⮚ The Detour uses a simple context-based router with two output channels. One output
channel passes the unmodified message to the original destination.
⮚ When instructed by the Control Bus, the Detour routes messages to a different channel.
This channel sends the message to additional components that can inspect and/or modify
the message. Ultimately, these components route the message to the same destination.

53
BITS Pilani, Pilani Campus
 Wire Tap

⮚ The Wire Tap is a fixed Recipient List with two output channels. It consumes messages off
the input channel and publishes the unmodified message to both output channels.
⮚ To insert the Wire Tap into a channel, you need to create an additional channel and
change the destination receiver to consume of the second channel

54
BITS Pilani, Pilani Campus
 Message History

⮚ Message History maintains a list of all components that the message passed through and
its origination.
⮚ Every component that processes the message (including the originator) adds one entry to
the list.
⮚ Message History should be part of the message header and not application data because it
contains system-specific control information.

55
BITS Pilani, Pilani Campus
Message Store

56
BITS Pilani, Pilani Campus
Smart Proxy

⮚ In order to track messages sent to request-reply services, a Smart

Proxy is inserted into the message stream.
⮚ Smart Proxy stores the Return Address supplied by the original
requestor and replace it with the address of the Smart Proxy. When the
service sends the reply message route it to the original Return Address.

BITS Pilani, Pilani Campus

Testing and Debugging

⮚ Testing a messaging system before deploying it into production is a good idea. But testing
should not stop there. You should be actively verifying that the running messaging system
is functioning properly.
⮚ You can do this by periodically injecting a Test Message into the system and verifying the
results.

58
BITS Pilani, Pilani Campus
Testing and Debugging
⮚ Test Message: Troubleshooting message which a component is actively processing
messages, but garbles outgoing messages due to an internal fault can be handled using a
Test Message.
⮚ Channel Purger: When a component fails or misbehaves, it is easy to end up with
unwanted messages on a channel. During testing it is very useful to remove all remaining
messages from a channel so that the components under test do not receive 'leftover'
messages. A Channel Purger does that for us.Use a Channel Purger to remove unwanted
messages from a channel.

59
BITS Pilani, Pilani Campus
Case Study: Online Order Processing
System
⮚ Test Message: Case Study: Online Order Processing System

⮚ An e-commerce company, "OrderMax," processes thousands of orders daily.

⮚ Their system integrates with multiple suppliers, shipping providers, and payment
gateways.

⮚ To ensure seamless communication between these systems, OrderMax uses various

message transformation middleware technologies.

⮚ 1. Envelope Wrapper

⮚ Problem: Suppliers require different formats for order headers (e.g., order ID, date, and
customer info).

60
BITS Pilani, Pilani Campus
Case Study: Online Order Processing
System
⮚ Solution: OrderMax uses an Envelope Wrapper to wrap the order payload with the required
headers for each supplier.
⮚ Original Message:

⮚ { "orderId": "123",
⮚ "customerName": "John Doe",
⮚ "items": [...] }

⮚ Wrapped Message (for Supplier A):

⮚ { "header": {
⮚ "orderId": "123",
⮚ "date": "2023-02-15"
⮚ },
⮚ "payload": {
⮚ "customerName": "John Doe", 61
⮚ "items": [...] } } BITS Pilani, Pilani Campus
Case Study: Online Order Processing
System
⮚ 2. Content Enricher

⮚ Problem: Shipping providers require additional customer information (e.g., phone number,
email) not present in the original order message.

⮚ Solution: OrderMax uses a Content Enricher to fetch the required information from the
customer database and add it to the message.

⮚ Example:
⮚ Original Message:

⮚ {
⮚ "orderId": "123",
⮚ "customerId": "456",
⮚ "items": [...]
⮚ } 62
BITS Pilani, Pilani Campus
Case Study: Online Order Processing
System
⮚ Enriched Message:

⮚ {
⮚ "orderId": "123",
⮚ "customerId": "456",
⮚ "customerPhone": "123-456-7890",
⮚ "customerEmail": "johndoe@example.com",
⮚ "items": [...]
⮚ }

⮚ 3. Content Filter

⮚ Problem: Payment gateways only require a subset of order information (e.g., order ID,
amount, and customer ID).
⮚ Solution: OrderMax uses a Content Filter to remove unnecessary information from the
message. 63
BITS Pilani, Pilani Campus
Case Study: Online Order Processing
System
⮚ Example: Original Message:

⮚ {
⮚ "orderId": "123",
⮚ "customerName": "John Doe",
⮚ "items": [...],
⮚ "amount": 100.00
⮚ }

⮚ Filtered Message:

⮚ {
⮚ "orderId": "123",
⮚ "customerId": "456",
⮚ "amount": 100.00
⮚ } 64
BITS Pilani, Pilani Campus
Case Study: Online Order Processing
System
⮚ 4. Claim Check

⮚ Problem: Large order messages with many items exceed the message size limit for some
suppliers.
⮚ Solution: OrderMax uses a Claim Check to store the large message payload in a repository
and include a claim check (reference ID) in the message.

⮚ Example:

⮚ Original Message:
⮚ {
⮚ "orderId": "123",
⮚ "items": [...] // large payload
⮚ }

65
BITS Pilani, Pilani Campus
Case Study: Online Order Processing
System
⮚ Transformed Message:

⮚ {
⮚ "orderId": "123",
⮚ "claimCheckId": "CCI-123"
⮚ }

⮚ The supplier can then retrieve the payload using the claim check ID.

⮚ 5. Canonical Model

⮚ Problem: Different suppliers and shipping providers use varying formats for order
messages.
⮚ Solution: OrderMax uses a Canonical Model to define a standard order format, which is
then transformed to match each external system's requirements.
66
BITS Pilani, Pilani Campus
Case Study: Online Order Processing
System
⮚ Example: - Canonical Order Model:

⮚ {
⮚ "orderId": "",
⮚ "customer": {
⮚ "name": "",
⮚ "email": "",
⮚ "phone": ""
⮚ },
⮚ "items": [...] }

⮚ Transformed messages for each supplier/shipping provider would adhere to this canonical
model, making it easier to integrate and maintain multiple systems.
⮚ By applying these message transformation technologies, OrderMax ensures seamless
integration with its suppliers, shipping providers, and payment gateways, reducing errors
and improving overall system efficiency. 67
BITS Pilani, Pilani Campus
 Chanel Purger Amazon-
SQS.
⮚ Many hosted messaging systems have a built-in function to purge channels. For example,
with Amazon's Simple Queuing Service (SQS), you can purge a queue simply from the web
user interface.

68
BITS Pilani, Pilani Campus
 Chanel Purger Amazon-
SQS.
⮚ SQS stands for Simple Queue Service.
⮚ SQS was the first service available in AWS.
⮚ Amazon SQS is a web service that gives you access to a message queue that can be used to store messages
while waiting for a computer to process them.
⮚ Amazon SQS is a distributed queue system that enables web service applications to quickly and reliably
queue messages that one component in the application generates to be consumed by another component
where a queue is a temporary repository for messages that are awaiting processing.
⮚ With the help of SQS, you can send, store and receive messages between software components at any
volume without losing messages.
⮚ Using Amazon sqs, you can separate the components of an application so that they can run independently,
easing message management between components. 69
BITS Pilani, Pilani Campus
Chanel Purger Amazon-
SQS.

70
BITS Pilani, Pilani Campus
 Chanel Purger Amazon-
SQS.
⮚ Suppose the user wants to look for a package holiday and wants to look at the best possible flight.
⮚ User types a query in a browser, it then hits the EC2 instance.
⮚ An EC2 instance looks "What the user is looking for?", it then puts the message in a queue to the SQS.
⮚ An EC2 instance pulls queue. An EC2 instance continuously pulling the queue and looking for the jobs to do.
⮚ Once it gets the job, it then processes it.
⮚ It interrogates the Airline service to get all the best possible flights.
⮚ It sends the result to the web server, and the web server sends back the result to the user.
⮚ User then selects the best flight according to his or her budget.

71
BITS Pilani, Pilani Campus
 Chanel Purger Amazon-
SQS.
⮚ Here's a real-life case study on using a Chained Purger with Amazon SQS:

⮚ Case Study: Order Processing System with Amazon SQS and Chained Purger

⮚ A large e-commerce company, "ShopZone," uses Amazon SQS to handle their order processing workflow.

⮚ ShopZone's system consists of multiple microservices, each responsible for a specific task, such as order
validation, inventory management, and payment processing.

⮚ Problem:

⮚ ShopZone's order processing system was experiencing issues with:

⮚ Message duplication: Duplicate order messages were being processed, causing inconsistencies in their inventory
and payment systems.
72
BITS Pilani, Pilani Campus
 Chanel Purger Amazon-
SQS.
⮚ Message ordering: Orders were not being processed in the correct sequence, leading to errors and retries.

⮚ Solution:

⮚ ShopZone implemented a Chained Purger pattern using Amazon SQS to address these issues.

⮚ Here's how they did it:

⮚ Architecture:

⮚ Order Queue: ShopZone created an SQS queue (Order Queue) where orders are sent by their web application.

⮚ Deduplication Queue: A second SQS queue (Deduplication Queue) is used to store unique order messages.

⮚ Chained Purger: A Lambda function (Chained Purger) is triggered by the Order Queue. It checks for duplicate
orders and purges them, ensuring only unique orders are sent to the Deduplication Queue.
73
BITS Pilani, Pilani Campus
 Chanel Purger Amazon-
SQS.
⮚ Chained Purger Logic:

⮚ Message Receipt: The Chained Purger Lambda function receives order messages from the Order Queue.

⮚ Duplicate Check: It checks if the order message is a duplicate by verifying the order ID against a cache (e.g.,
Redis) or a database.

⮚ Purge Duplicates: If a duplicate order is found, the Chained Purger deletes the message from the Order Queue.

⮚ Send Unique Messages: If the order is unique, the Chained Purger sends the message to the Deduplication
Queue.

74
BITS Pilani, Pilani Campus
 Chanel Purger Amazon-
SQS.
⮚ Benefits:

⮚ Eliminated Duplicate Processing: By purging duplicate orders, ShopZone eliminated inconsistencies in their
inventory and payment systems.

⮚ Ensured Correct Order Processing: The Chained Purger ensured that orders are processed in the correct
sequence, reducing errors and retries.

⮚ Improved System Efficiency: By removing duplicates and ensuring correct order processing, ShopZone's system
efficiency improved, and they experienced fewer errors

75
BITS Pilani, Pilani Campus
 Chanel Purger Amazon-
SQS.
⮚ const AWS = require('aws-sdk');
⮚ const redis = require('redis');

⮚ exports.handler = async (event) => {

⮚ const sqs = new AWS.SQS({ region: 'your-region' });
⮚ const redisClient = redis.createClient({ host: 'your-redis-host' });

⮚ for (const record of event.Records) {

⮚ const orderId = JSON.parse(record.body).orderId;

⮚ // Check for duplicate orders in Redis

⮚ const isDuplicate = await redisClient.exists(orderId);

76
BITS Pilani, Pilani Campus
 Chanel Purger Amazon-
SQS.
⮚ if (isDuplicate) {
⮚ // Delete duplicate message from Order Queue
⮚ await sqs.deleteMessage({

⮚ QueueUrl: 'https://sqs.your-region.amazonaws.com/your-account-id/OrderQueue',
⮚ ReceiptHandle: record.receiptHandle,
⮚ }).promise();

77
BITS Pilani, Pilani Campus
 Chanel Purger Amazon-
SQS.
⮚ } else {
⮚ // Send unique message to Deduplication Queue
⮚ await sqs.sendMessage({
⮚ QueueUrl: 'https://sqs.your-region.amazonaws.com/your-account-id/DeduplicationQueue',
⮚ MessageBody: record.body,
⮚ }).promise();

⮚ // Set order ID in Redis with a TTL (e.g., 1 hour)

⮚ await redisClient.set(orderId, 'processed', 'EX', 3600);
⮚ }
⮚ }

⮚ return { statusCode: 200 };

⮚ };

78
BITS Pilani, Pilani Campus
Thank You

79
BITS Pilani, Pilani Campus