[go: up one dir, main page]

0% found this document useful (0 votes)
21 views26 pages

IIB-Handout

The document outlines the architecture and features of IBM Integration Bus (IIB) version 10, highlighting the optional use of MQ for Integration Nodes on distributed platforms and the introduction of flexible topology. It details the message flow processing, logical trees, and various parsing methods, as well as the use of ESQL for managing logical trees and message transformations. Additionally, it describes the packaging of resources in the toolkit and the functionality of different message flow nodes.

Uploaded by

shamanthck.work
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
21 views26 pages

IIB-Handout

The document outlines the architecture and features of IBM Integration Bus (IIB) version 10, highlighting the optional use of MQ for Integration Nodes on distributed platforms and the introduction of flexible topology. It details the message flow processing, logical trees, and various parsing methods, as well as the use of ESQL for managing logical trees and message transformations. Additionally, it describes the packaging of resources in the toolkit and the functionality of different message flow nodes.

Uploaded by

shamanthck.work
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 26

IBM INTEGRATION BUS ARCHITECTURE

IBM INTEGRATION BUS V10 – FLEXIBLE TOPOLOGY FOR MQ

Note: From v10 of IIB, MQ has been made op�onal for IIB Run�me on distributed pla�orms like
Windows, Linux, Unix, although its s�ll mandatory for IIB Run�me on Mainframe (zOS).

• Each Integra�on Node can be created without any Queue Manager associated
• Each Integra�on Node can be associated with one default Queue Manager.
• Integra�on Node can connect with 1 or more Queue Managers running locally or remotely.
• Queue Managers can be shared across mul�ple Integra�on Nodes.
IBM INTEGRATION BUS V9 & BEFORE – MQ TOPOLOGY

Note: Till v9 of IBM Integra�on Bus, MQ was a mandatory component for se�ng up IIB Run�me.
Every Integra�on Node must be associated with a dedicated Queue Manager.

• Each Integra�on Node connects only to the Queue Manager associated with it.
• No sharing of Queue Managers between Integra�on Nodes.

IBM INTEGRATION BUS RUNTIME PROCESS OVERVIEW


IBM INTEGRATION BUS PROCESS & THREADS OVERVIEW

INTEGRATION SOLUTION RESOURCES


MESSAGE MODEL

Approaches to creating Message Models

 mqsicreatemsgdefs: command line utility for creating message models by


importing C header files, COBOL copybooks, XML DTD files, or XML Schema files
 mqsicreatemsgdefsfromwsdl: command line utility for creating message
models by importing wsdl definitions.
 Using Toolkit
MESSAGE FLOWS

Message Flow Illustration

 Sequence of processing instruction provided to runtime for integrating source


and target applications.
o Processing Instructions / Steps: specified by using message flow
nodes.
o Sequence: determined by the connecting between the message flow
nodes
 Each message flow as set of predefined terminals, which define the entry &
exit points.
o Out terminal: For sending the processed data (output logical tree
instances) in case of success.
o Failure terminal:
 For catching exceptions occurring in that node
 Input logical tree instances & ExceptionList tree capturing the
details of error are propagated via this terminal, when an error
occurs in the node.
 Like handling the exception locally in each Java method by using
try-catch method.
o Catch terminal:
 For catching exceptions occurring elsewhere in the downstream
processing nodes
 Like handling exception commonly in the main Java method
using try-catch block, for catching exception occurring in other
methods. Methods keep throwing the exception back to the
calling method until it reaches the method with try-catch block.
 Input Nodes are the starting point of the processing logic.
o Configured to specify the Message Domain, based on the input
message format, to determine the Parsed used for parsing the input bit
stream to Logical Tree.
o Also specifies the Model definition to be used by Model-driven parsers
for parsing.
 Each Message flow runs as a thread within the Integration Server process in
runtime.
o By default, they are single threaded, but can be made multi-threaded
by specifying Additional Instances for the Message Flow in the BAR
File
 Each Message flow is one unit-of work. Transaction starts when the message
flow starts processing a message and the transaction is committed only at the
message flow processing or rolled-back in case of any exceptions occurring in
the processing.
o Transaction behaviour of the message flow can be controlled by
specifying the Transaction Mode property in the Input nodes.
o Transaction behaviour can be explicitly overridden in the Output nodes.
CONTAINERS FOR PACKAGING RESOURCES IN TOOLKIT

 Integration Project : unstructured container in which you can develop


message flows and related resources. When resources from an integration
project are packaged for deployment, you can choose exactly which resources
should be packaged together.
 Applications : container for the resources that are required to create a
solution, and that should be deployed and managed together. Applications
provide encapsulation, and resources within an application are packaged and
deployed as a single unit and can be managed as a single unit.
 Libraries - logical grouping of related code, data, or both. A library typically
contains reusable helper routines and resources.
o Shared Library (introduced in v10) :
 For maximum reusability, to deploy and manage just one copy of
those common resources.
 Deployed directly to the Integration Server
 On update to Shared library resource, you need to deploy only
the Share Library and the changes are immediately picked up by
all referencing application in that Integration Server
 To be deployed before or along with Applications referencing
them
 Can refer to other Shared Libraries but not Static Libraries
o Static Library
 Each Application referring the Static Library, takes a private copy
of the Static Library within itself when packaged into BAR file and
deployed to runtime.
 If you update a static library, you must repackage and redeploy
each application that references that library.

LOGICAL TREES
 Internal representation of a message
 Message Tree:
o Contains the Application related message, either received from Source
Application or to be sent to the Target Application
o Top-most element: Root
 Properties: First child of Root element
 Body: Last child of Root element and is typically named after the
Message Domain and contains the application payload

 Environment Tree:
o For storing information as variables and passing between nodes for
internal processing
o Top-most element: Environment
o Only 1 instance for entire message processing
o Information stored in Environment tree are persisted even during
exception scenarios
o Similar to Class variable in Java

 LocalEnvironment Tree:
o Purposes
 Storing information and passing to downstream nodes of
message flow information [LocalEnvironment.Variables]
 For storing information for routing purpose to be used by output
nodes [LocalEnvironment.Destination]
 For storing information related to messages sent out by output
nodes and hence created by output nodes
[LocalEnvironment.WrittenDestination]
 For storing information related to incoming message for certain
protocols [e.g. LocalEnvironment.File, LocalEnvironment.REST
etc.]
o Top-most element: LocalEnvironment
o Unlike Environment tree, each Node creates a new instance to modify
this tree and in case of any exception, changes to this tree are reverted
on rollback.

 ExceptionList Tree:
o For storing error details and are generated only when message flow
fails. Like Exception object in Java
o Contains information about exceptions that occur when a message is
processed when an exception is caught on the local Failure path or
Catch path
o Can be accessed, modified, and traced
o Top-most element: ExceptionList
PARSERS

Input Message Parsing


Serializing Output Logical Tree to bitstreams

BODY PARSERS / TYPE OF PARSERS

 MRM Parser
o Message Repository Manager (MRM)
o Parser for parsing message definitions created as Message Sets.
o Handles all text, binary & XML data
 CWF – Custom Wire Format for handling binary, fixed-width and
packed data formats
 TDS – Tagged Delimited Separator format for handling delimited
and tagged data formats
 XML – for handling XML message formats

 Parsers for XML messages


o XML parser  for parsing simple XML data without any namespaces
o XMLNS parser  for parsing XML data with namespaces
o XMLNSC parser  Compact parse for XML data with namespaces.
Recommended parser for parsing XML Data
 reduces the amount of memory used by logical tree by discarding
discard non-significant white space and mixed content,
comments, processing instructions from the XML data

 DFDL (Data Format Description Language)


o Open Standard for modelling text & binary data
o Standard maintained by Open Grid Forum (OGF)
o Stores the definition in schema file (.xsd) with DFDL annotations.

 Which Parser or Domain to be used –

Application Data Format Parser or Domain


SOAP-based web services SOAP
REST Services / API with JSON data JSON
XML Data XMLNSC
C or COBOL application or fixed-width DFDL (or)
data MRM with CWF format
Text with delimiters or tags DFDL (or)
MRM with TDS format

Note: Message Model was introduced from WebSphere Message Broker v8 onwards. Prior to
that Message Sets was used to create definitions for messages. Message Models replaced
Message Sets from v8 onwards. While Message Set was proprietary to WMB / IIB for creating
message definitions, Message Models adopt open standards like DFDL, XSD for creating
message definitions.

Parser used to parse Message Sets: MRM Parser

Types of Message Format in Message Sets


• CWF – Custom Wire Format: Typically used for creating legacy data format and for
binary data
• TDS – Tagged and Delimited String Format: Typically used for industry standard
formats, including SWIFT, EDI, ACORD, AL3, and TLOG. Also, used for any delimited
or tagged text data and binary data too.
• XML – Typically used for creating definition for XML data

MESSAGE MODELING OVERVIEW


ESQL

 ESQL: Extended Structured Query Language


 Purpose:
o One of the Interface provided in IIB to work with Logical Tree – to read
/ write
o Can work with Databases too
o Based on SQL language
 Nodes using ESQL
o Compute Node
o Filter Node
o Database Node
o DatabaseInput Node
o Throw Node – for specifying Patterns in format ${}. For e.g.
${CURRENT_TIMESTAMP}
 Every node using ESQL logic, refers to corresponding ESQL module with MAIN
function.
o Multiple nodes can refer to same ESQL module too for reusability.
 ESQL logics / statements are saved separately in a .esql file.
o Single ESQL file can contain 1 or more ESQL modules that are referred
by the message flow nodes
 ESQL Variables:
o Normal:
DECLARE <variable-name> <variable-datatype> <initial expression>
e.g. DECLARE status CHARACTER

o Shared: for caching purpose and make variable shared between


multiple threads
 Declared used keyword SHARED in DECLARE statement outside
the module
 When the flow or integration server stops, SHARED variable
lifetime ends
 ATOMIC blocks can serialize access to SHARED variables for
modifying variable value
DECLARE <variable-name> SHARED <variable-datatype> <initial-expression>
e.g. DECLARE status SHARED CHARACTER

o External: For creating environment specific custom properties for


configuring message flow
 Created as User Defined Properties (UDP) in the Message Flow
Editor
 Accessed in ESQL by declaring the UDP as EXTERNAL variable
 Declared used keyword EXTERNAL in DECLARE statement
outside the module
DECLARE <variable-name> EXTERNAL <variable-datatype> <initial-expression>
e.g. DECLARE status EXTERNAL CHARACTER

 ESQL Functions:
o THE – Returns the first element from the array object
o CARDINALITY – Returns the size (number of elements) of the array
object
o EXISTS – Checks if array has any elements or not. Returns TRUE if array
size > 0 otherwise FALSE
o LENGTH – Returns the length of string value
o CONTAINS – Checks if search string is present in the string value or not.
Returns TRUE is the search string is present in the string value
otherwise FALSE
o CAST – To convert from one datatype to another
o PROPAGATE – To send the output logical trees via a specified terminal
or by default out terminal of Compute Node.
o EXTRACT – To extract the date fields from the date variable like DAY or
MONTH or Year etc as INTEGER
 Difference between PROPAGATE & RETURN in compute node:
o When RETURN TRUE, compute node sends the output logical tree
instances via the out terminal and once sent out control does not come
back to the compute node.
o Wheresas when PROPAGATE statement is used, the output logical tree
instances are sent out via the specified terminal or default out terminal.
And once the processing path, that was connected to the terminal via
which output logical tree instances was sent out, is completed the
control comes back the ESQL module of the compute node and starts
executing the statement after the PROPAGATE statement.
 For nodes working with multiple instances of the logical tree instances that is
coming as input and that will be sent out as output, e.g. Compute Node, to
identify the logical tree instance referred in ESQL statements, we additionally
correlate as InputRoot or OutputRoot etc, where
o Root refers the top-most element of the message tree
o Input or Output are correlation used to identify the instance of the
message tree.

Note: Only Environment tree does not have correlation as only 1 instance of the
Environment tree is maintained for the entire message processing.
FEW MESSAGE FLOW NODES

 FILTER Node:
o Primary Purpose: For routing based on simple if-else logic
o Interface used: ESQL
o Does not modify the logical trees and hence propagates the same
instance that is received as input. Hence no need for additional
correlation when accessing the logic trees. Simply access the logical
tress as Root, LocalEnvironment etc.
o Terminals
 FAILURE: for exception handling of errors occurring in this node
 TRUE: Message is sent out via this terminal, when ESQL module
returns TRUE based on condition check
 FALSE: Message is sent out via this terminal, when ESQL module
returns FALSE based on condition check
 UNKNOWN: Message is sent out via this terminal, when the
condition check expression evaluates to null or is empty or
cannot be determined as TRUE or FALSE.
 ROUTE Node:
o Primary Purpose: For routing based on filtering conditions
o Interface used: XPATH
o One of the node in which custom output terminals can be added and
used for routing.
o Does not modify the logical trees and hence propagates the same
instance that is received as input. Hence no need for additional
correlation when accessing the logic trees. Simply access the logical
tress as $Root, $LocalEnvironment etc.
o Multiple Filter patterns can be specified and mapped to the terminal
that is to be used for routing of the pattern matches.
o Distribution Mode: Determines how the filter patterns specified must
be evaluated -
 FIRST: The multiple filter patterns specified are evaluated as IF-
ELSEIF block. If any pattern evaluates to TRUE no further
patterns are checked.
 ALL: The multiple filter patterns specified are evaluated as
separate IF blocks. Irrespective of any pattern evaluating to
TRUE, all patterns are checked and handled accordingly.
 RouteToLabel & Label Node:
o Best suited for handling multiple conditional path
o Like GoTo <Label> and Label block in programming languages
o Always used together.
o Each Label identifies separate processing path to be execution for a
condition / scenario.
o RouteToLabel is like GoTo statement, when is used to route the
processing control to the appropriate label block identified by label
name specified.
 Label Names to be routed to are specified in the
LocalEnvironment tree based on condition checks in a node
prior to RouteToLable node either in Compute / JavaCompute /
Mapping node etc.
o No physical connection exists between RouteToLabel & Label Node

 Compute Node
o Primary Purpose: General purpose node primarily used for
transformation of logical trees. Hence works with 2 instances of the
logical trees – input and output instances. Additionally can also be used
for
 Content-based Routing
 Business process logics
 Database operations
 Creating & distributing the output logical trees for different target
applications
o Terminals: 5 out terminals are there in addition to Failure terminal. By
default output logical trees are sent out via out terminal.
o Refers to ESQL module specified in ESQL Module property
o Transaction Mode: Determines whether the compute node commits
the database transactions performed within the node immediately or
after the message flow processing end completely
 AUTOMATIC: All database transactions performed in the node
are committed only at the end of message flow processing. Any
error occurs in the message flow processing, all transactions are
rolledback along with message flow.
 COMMIT: All database transactions performed in the node are
committed immediately when node completes the processing
irrespective of the processing status of the message flow.
o DSN: specifies the default DSN (ODBC Data Source Name) used by the
ESQL statements working with the database in the node, if not DSN was
specified at the statement level.
o Compute Mode: Identifies the logical trees for which the Compute node
creates output logical tree instances or writable instances. Any changes
only to the specifies logical tree names are considered and sent out as
Output. For all the other logical trees, the same instances received as
input will be sent out as it is.

 JavaCompute Node
o Primary Purpose: General purpose node primarily used for
transformation of logical trees. Hence works with 2 instances of the
logical trees – input and output instances. Additionally, can also be
used for
 Content-based Routing
 Business process logics
 Database operations
 Creating & distributing the output logical trees for different target
applications
o Uses Java Programming language and hence refers to a Java Class
o Entry point: evaluate method of the specified Java class
o Input to evaluate method: Message Assembly object which contains all
4 logical trees.
 MbMessageAssembly  refers to the Assembly object which is
bundle of all 4 logical trees
 MbMessage  refers to the individual Logical Trees like Message
Tree, Environment, LocalEnvironment or ExceptionList
 MbElement  refers to each field or element of a Logical Tree
o Terminals: 2 out terminals (Out & Alternate) are there in addition to
Failure terminal. By default, output logical trees are sent out via out
terminal.

 Mapping Node:
o Primary Purpose: Graphically perform transformation of logical trees.
Additionally, can also be used for mapping.
 LocalEnvironment & Environment trees
 Database operations
 Working with Global Cache
Creating & distributing the output logical trees for different target
applications
o Uses XPath functions to perform transformation and refers to a
Mapping routing stored in .map file

 MQGet Node:
o To retrieve message from MQ queue in-between the message flow
processing logic
 FileRead Node:
o To read the file in-between the message flow processing logic

 MQInput Node:
o For starting the message flow by reading the messages from the input
queue
o Terminals:
 out
 failure
 catch
 MQOutput Node:
o For sending message to a MQ queue
o Options for specifying the queue name for writing messages by setting
the Destination Mode property
 Queue Name: uses the name of the queue specified in the
MQOutput node property for delivering the message
 DestinationList: uses the name of the queue from the
LocalEnvironment
 ReplyToQueue: uses the name of the queue specified in the
ReplyToQ file of MQMD header
WORKING WITH DATABASES

 Database connections for ODBC:


o Connections established per DSN for each message flow thread.
 Connection reused across nodes within the thread (under same
transaction)
 Connection created for different transaction mode

Command for Creating Security Identity or associating DSN to Integration Node

Note:

 ODBC connectivity is required for nodes using ESQL – Compute, Filter,


Database, DatabaseInput Nodes
 JDBC Connectivity is used by following nodes – Mapping, JavaCompute,
DatabaseRetrieve, DatabaseRoute Nodes
JDBC Connection Management for Pooled Connections – for each thread

JDBC Connection Management for Non-Pooled Connections – for each thread


ERROR HANDLING
 Options for Preventing Message from going to DLQ of QM
o Connect the Failure terminal
o Connect the Catch terminal
o Configure Backout Queue for the Input Queue

 Transactional status of Message Flow is determined using Transaction Mode


property of Input Node

 Backout Queue: Associated to Input Queue property


 Dead Letter Queue: Associated at Queue Manager level
o Per Queue Manager only 1 DLQ configurable

 Options for catching exceptions:


o Connect the Failure terminal of the node to a sequence of nodes that
processes the node's internal exception
o Connect the Catch terminal of the node to a sequence of nodes that
processes exceptions that are generated beyond it
o Insert one or more TryCatch nodes at specific points in the message
flow to catch and process exceptions
o Include a Throw node to generate an exception
o Ensure that all the messages received by an MQInput node are:
 Either processed in a transaction, or are not processed in a
transaction
 Either persistent, or are not persistent
IBM INTEGRATION BUS COMMANDS:

 mqsistart: To start the integration node


 mqsistop: to stop the integration node
 mqsilist: to list all integration nodes with their status that have been created
in the system
 mqsisetdbparms:
o to create security identity alias for storing username & password that
are required to connect to external systems like databases, ftp server,
email server etc.
o For ODBC connectivity, command associates the DSN to the Integration
Node along with specifying the username and password for database
connection.
 mqsichangeproperties: to modify integration node properties and properties
of integration node resources
 mqsichangetrace: to enable / disable userTrace or serviceTrace or trace
nodes
o -u: to work with user Trace
o -t: to work with Service Trace
o -n: to work with Trace nodes
 mqisreadlog: to retrieve the user trace or service trace records stored in
binary format and convert to XML format into an XML file.
 mqsicreatemsgdefs: command line utility for creating message models by
importing C header files, COBOL copybooks, XML DTD files, or XML Schema
files
 mqsicreatemsgdefsfromwsdl: command line utility for creating message
models by importing wsdl definitions.
 mqsiformatlog: to process the XML log created by mqsireadlog as part of
userTrace or serviceTrace into normal text output
ACRONYMS

SOA: Service Oriented Architecture

API: Application Programming Interface

ESB: Enterprise Service Bus

EAI: Enterprise Application Integration

JSON: JavaScript Object Notation

REST: Representational State Transfer

WSDL: Web Services Description Language

SOAP: Simple Object Access Protocol

BAR: Broker Archive File

YAML: Yet Another Markup Language

HTML: HyperText Markup Language

XSD: XML Schema Definition

MQMD: Message Queue Message Descriptor

DFDL: Data Format Description Language

JDBC: Java Database Connectivity

ODBC: Open Database Connectivity

MQ: Message Queue

XML: Extensible Markup Language

ESQL: Extended Structured Query Language

MRM: Message Repository Manager

CWF: Custom Wire Format

TDS: Tagged and Delimited String

You might also like