CS542
CS542 Topics
Topics in
in
Distributed
Distributed Systems
Systems
Diganta Goswami
RMI/RPC
RMI/RPC -- Motivation
Motivation
• You write a program where objects call each other
• Works well if the program runs on one process
• What if you split your objects across multiple
processes?
• Can Object1 still call [Link]()?
• Why (not)?
• Solution
– RMIs: Remote Method Invocations (Object-based)
– RPCs: Remote Procedure Calls (non-Object-based)
Access libraries of reusable code across hosts
Pros
Supports code reuse
Standard interface, independent of applications and OS’s
Middleware
Middleware
Layers
Layers
Applications
RPCs and RMIs, e.g., CORBA
Middleware
Request reply protocol layers=
Provide
External data representation support to the
application
Operating System
Run at all servers
@user level
RPC = Remote Procedure Call
RMI=Remote Method Invocation
CORBA=Common Object Request Brokerage Architecture
Local
Local
Objects
Objects
• Within one process’ address space
• Object
– consists of a set of data and a set of methods.
– E.g., C++ object, Java object.
• Object reference
– an identifier via which objects can be accessed.
– i.e., a pointer (e.g., virtual memory address within process)
• Interface
– provides a definition of the signatures of a set of methods (i.e.,
the types of their arguments, return values, and exceptions)
without specifying their implementation.
Remote
Remote
Objects
Objects
• May cross multiple process’ address spaces
• Remote method invocation
– method invocations between objects in different processes
(processes may be on the same or different host).
– Remote Procedure Call (RPC): procedure call between
functions on different processes in non-object-based system
• Remote objects
– objects that can receive remote invocations.
• Remote object reference
– an identifier that can be used globally throughout a distributed
system to refer to a particular unique remote object.
• Remote interface
– Every remote object has a remote interface that specifies
which of its methods can be invoked remotely. E.g., CORBA
interface definition language (IDL).
A
A Remote
Remote Object
Object and
and Its
Its Remote
Remote Interface
Interface
remoteobject
Data
remote
interface
m1 m4
{
implementation m5
m2
of methods m6
m3
Example Remote Object reference=(IP,port,objectnumber,signature,time)
Remote
Remote and
and Local
Local Method
Method
Invocations
Invocations
Object Process
Process Process
Host A
remote local C
E
invocation invocation local
remote
invocation invocation F
A B local
invocation D
Host B
Local invocation=between objects on same process.
Has exactly once semantics
Remote invocation=between objects on different processes.
Ideally also want exactly once semantics for remote invocations
But difficult (why?)
Failure
Failure Modes
Modes of
of RMI/RPC
RMI/RPC
Request
correct lost
Execute request
function
Reply
Request Channel
Execute fails
during
Execute, crash reply
Crash before Reply
reply
Request Client
machine
Execute fails
crash before
Crash before receiving
execution Reply
reply
(and if request is received more than once?)
Transparency=remote invocation has same behavior as
Invocation
Invocation local invocation
[Birrell and Nelson, inventors of RPC, 1984]
Semantics
Semantics
Very difficult to implement in asynchronous network…
whether to keep a
history of result
Whether or not to
when retransmissions messages to enable
retransmit the request
are used, whether to lost results to be
message until either
filter out duplicate retransmitted without
a reply is received or
requests at the server. re-executing the
the server is assumed
operations
to be failed
Fault tolerance measures Invocation
semantics
Retransmit request Duplicate Re-execute procedure
message filtering or retransmit reply
CORBA No Not applicable Not applicable Maybe
(ok for idempotent operations)
Sun RPC Yes No Re-execute procedure At-least-once
Java RMI,
CORBA Yes Yes Retransmit old reply At-most-once
Idempotent=same result if applied repeatedly, w/o side effects
Proxy
Proxy and
and Skeleton
Skeleton in
in Remote
Remote Method
Method
Invocation
InvocationProcess P1 Process P2
client server
remote
object A proxy for B skeleton
object B
Request & dispatcher
for B’s class
Reply
Remote Communication Communication Remote reference
reference module module module module
MIDDLEWARE
Proxy
Proxy and
and Skeleton
Skeleton in
in Remote
Remote Method
Method
Invocation
InvocationProcess P1 (“client”) Process P2 (“server”)
client server
remote
object A proxy for B skeleton
object B
Request & dispatcher
for B’s class
Reply
Remote Communication Communication Remote reference
reference module module module module
Prox
Prox
yy
• Is responsible for making RMI transparent to
clients by behaving like a local object to the
invoker.
– The proxy implements (Java term, not literally) the methods in
the interface of the remote object that it represents. But,…
• Instead of executing an invocation, the proxy
forwards it to a remote object.
– On invocation, a method of the proxy marshals the following
into a request message: (i) a reference to the target object, (ii)
its own method id and (iii) the argument values. Request
message is sent to the target, then proxy awaits the reply
message, un-marshals it and returns the results to the invoker.
– Invoked object unmarshals arguments from request message,
and when done marshals return values into reply message.
Marshalling
Marshalling &
& Unmarshalling
Unmarshalling
A x86 (Windows) client sends an RMI to a PowerPC (e.g.,
Unix/Mac) server
won’t work because x86is little endian while PowerPC is big-endian
Big endian: 1234 is stored as 1234
External data representation: an agreed, platform-
independent, standard for the representation of data
structures and primitive values.
CORBA Common Data Representation (CDR)
Allows Windows client (little endian) to interact with Unix server or Mac
server (big endian).
Marshalling: the act of taking a collection of data items
(platform dependent) and assembling them into the external
data representation (platform independent).
Unmarshalling: the process of disassembling data that is in
external data representation form, into a locally interpretable
form.
Remote
Remote Reference
Reference
Module
Module
• Is responsible for translating between local and remote
object references and for creating remote object references.
• Has a remote object table
– An entry for each remote object held by any process. E.g., B at P2.
– An entry for each local proxy. E.g., proxy-B at P1.
• When a new remote object is seen by the remote reference
module, it creates a remote object reference and adds it to
the table.
• When a remote object reference arrives in a request or reply
message, the remote reference module is asked for the
corresponding local object reference, which may refer to
either to a local proxy or a remote object.
• In case the remote object reference is not in the table, the
RMI software creates a new proxy and asks the remote
reference module to add it to the table.
Proxy
Proxy and
and Skeleton
Skeleton in
in Remote
Remote Method
Method
Invocation
InvocationProcess P1 (“client”) Process P2 (“server”)
client server
remote
object A proxy for B skeleton
object B
Request & dispatcher
for B’s class
Reply
Remote Communication Communication Remote reference
reference module module module module
What
What about
about Server
Server Side?
Side?
Dispatcher
Dispatcher and and
Skeleton
Skeleton
• Each process has one dispatcher. And a skeleton
for each local object (actually, for the class).
• The dispatcher receives all request messages
from the communication module.
– For the request message, it uses the method id to select the
appropriate method in the appropriate skeleton, passing on the
request message.
• Skeleton “implements” the methods in the remote
interface.
– A skeleton method un-marshals the arguments in the request
message and invokes the corresponding method in the local
object (the actual object).
– It waits for the invocation to complete and marshals the result,
together with any exceptions, into a reply message.
Summary
Summary of
of Remote
Remote Method
Method Invocation
Invocation (RMI)
(RMI)
Client Proxy Proxy object is a hollow
Process Object container of Method
B names.
Object A Remote
Reference Remote Reference
Module Module translates
Comm. between local and
Module remote object
references.
Server Dispatcher sends the
Process Comm. Remote request to Skeleton
Module Reference
Module Object
Skeleton unmarshals
Dispatcher parameters, sends it
Skeleton Object to the object, &
for B’s B
marshals the results
Class for return
MIDDLEWARE
Generation
Generation of
of Proxies,
Proxies, Dispatchers
Dispatchers and
and
Skeletons
Skeletons
• Programmer only writes object implementations
and interfaces
• Proxies, Dispatchers and Skeletons generated
automatically from the specified interfaces
• In CORBA, programmer specifies interfaces of
remote objects in CORBA IDL; then, the interface
compiler automatically generates code for
proxies, dispatchers and skeletons.
• In Java RMI
– The programmer defines the set of methods offered by a
remote object as a Java interface implemented in the remote
object.
– The Java RMI compiler generates the proxy, dispatcher and
skeleton classes from the class of the remote object.
Binder
Binder and
and Activator
Activator
• Binder: A separate service that maintains a table containing
mappings from textual names to remote object references.
(sort of like DNS, but for the specific middleware)
– Used by servers to register their remote objects by name. Used by
clients to look them up. E.g., Java RMI Registry, CORBA Naming Svc.
• Activation of remote objects
– A remote object is active when it is available for invocation within a
running process.
– A passive object consists of (i) implementation of its methods; and (ii)
its state in the marshalled form (a form that is shippable).
– Activation creates a new instance of the class of a passive object and
initializes its instance variables. It is called on-demand.
– An activator is responsible for
» Registering passive objects at the Binder
» Starting named server processes and activating remote objects in
them.
» Keeping track of the locations of the servers for remote objects it
has already activated
– E.g., Activator=Inetd, Passive Object/service=FTP (invoked on demand)
Etc
Etc
..
• Persistent Object = an object that survives
between simultaneous invocation of a process.
E.g., Persistent Java, PerDIS, Khazana.
• If objects migrate, may not be a good idea to have
remote object reference=(IP,port,…)
– Location service= maps a remote object reference to its likely
current location
– Allows the object to migrate from host to host, without
changing remote object reference
– Example: Akamai is a location service for web objects. It
“migrates” web objects using the DNS location service
Remote
Remote Procedure
Procedure Call
Call (RPC)
(RPC)
Similar to RMIs, but for non-OO/non-object-based
scenarios
Procedure call that crosses process boundary
Client process calls for invocation of a procedure
at the server process.
Semantics are similar to RMIs – at least once, at most once,
maybe
Format of the message is standard (marshaled), uses request-
reply
Client
Client and
and Server
Server Stub
Stub Procedures
Procedures in
in
RPC
RPC
client process server process
Request
Reply
client stub server stub
procedure procedure
client service
procedure Communication Communication procedure
module module dispatcher
Stubs
Stubs
Stubs are generated automatically from interface
specifications.
Stubs hide details of (un)marshalling from
application programmer & library code developer.
Client Stubs perform marshalling into request
messages and unmarshalling from reply messages
Server Stubs perform unmarshalling from request
messages and marshalling into reply messages
Stubs also take care of invocation
The
The Stub
Stub Generation
Generation Process
Process
Compiler / Linker gcc
Server
.o, .exe Program
.c Server .c
Server
Stub Source
.h
Interface Stub Common RPC
RPC
Specification Generator Header LIBRARY
LIBRARY
e.g., in SUN XDR e.g., rpcgen
Client Client .c
.c
Stub Source
.o, .exe Client
Program
Compiler / Linker gcc
Summary
Summary
• Local objects vs. Remote objects
• RPCs and RMIs
• RMI: invocation, proxies, skeletons, dispatchers
• Binder, Activator, Persistent Object, Location
Service
Available with most Sun systems, and NFS
Files
Files Interface
Interface in
in Sun
Sun
XDR
XDR
const MAX = 1000;
typedef int FileIdentifier;
typedef int FilePointer;
typedef int Length; struct readargs {
struct Data { FileIdentifier f;
int length; FilePointer position;
char buffer[MAX]; Length length;
Only one argument allowed
}; }; Can specify as struct
struct writeargs {
FileIdentifier f; program FILEREADWRITE {
FilePointer position; version VERSION {
Data data; void WRITE(writeargs)=1; 1
}; Data READ(readargs)=2; 2
}=2; Version number
} = 9999; Program number
Finding
Finding RPCs
RPCs
Finding An RPC:
CLIENT
Client
RPCs live on specific hosts
Stub at specific ports.
Client
Program Port mapper on the host
Comm.
Module maps from RPC name to
port#
When a server process is
initialized, it registers its
RPCs (handle) with the port
Comm. SERVER mapper on the server
Module
A client first connects to
port mapper (daemon on
Dispatcher
Server standard port) to get this
procedure handle
Server
Stub The call to RPC is then
made by connecting to the
corresponding port
Dealing
Dealing Room
Room
[Publish-Subscribe System]
e.g, stock market
System
System External
Dealer’s computer Dealer’s computer
source
Dealer Notification Notification Dealer
Notification Information
provider Notification
Notification
Notification
Notification
Dealer’s computer Dealer’s computer
Notification
Information
provider
Notification
At each dealer: Dealer Notification
One object per Dealer
stock type
of interest External
source
Architecture
Architecture for
for Distributed
Distributed Event
Event
Notification
Notification
Event service
object of interest subscriber
1. notification
object of interest observer subscriber
2. notification notification
object of interest observer subscriber
3. notification
