BCAM051

CLOUD COMPUTING

UNIT 2 Cloud Enabling

Technologies Service
Oriented Architecture

SYLLABUS

Cloud Enabling Technologies Service Oriented Architecture: REST

and Systems of Systems – Web Services – Publish, Subscribe Model –
Basics of Virtualization – Types of Virtualization – Implementation
Levels of Virtualization – Virtualization Structures – Tools and
Mechanisms – Virtualization of CPU – Memory – I/O Devices –
Virtualization Support and Disaster Recovery.
 Service-Oriented Architecture
Service-Oriented Architecture (SOA) is a stage in the evolution of application development
and/or integration. It defines a way to make software components reusable using the
interfaces.

Formally, SOA is an architectural approach in which applications make use of services available
in the network. In this architecture, services are provided to form applications, through a
network call over the internet. It uses common communication standards to speed up and
streamline the service integrations in applications. Each service in SOA is a complete business
function in itself. The services are published in such a way that it makes it easy for the
developers to assemble their apps using those services. Note that SOA is different from
microservice architecture.

• SOA allows users to combine a large number of facilities from existing services to form
applications.
• SOA encompasses a set of design principles that structure system development and provide
means for integrating components into a coherent and decentralized system.
• SOA-based computing packages functionalities into a set of interoperable services, which
can be integrated into different software systems belonging to separate business domains.
The different characteristics of SOA are as follows :
o Provides interoperability between the services.
o Provides methods for service encapsulation, service discovery, service composition,
service reusability and service integration.
o Facilitates QoS (Quality of Services) through service contract based on Service Level
Agreement (SLA).
o Provides loosely couples services.
o Provides location transparency with better scalability and availability.
o Ease of maintenance with reduced cost of application development and
deployment.
There are two major roles within Service-oriented Architecture:

1. Service provider: The service provider is the maintainer of the service and the organization that
makes available one or more services for others to use. To advertise services, the provider can
publish them in a registry, together with a service contract that specifies the nature of the
service, how to use it, the requirements for the service, and the fees charged.
2. Service consumer: The service
consumer can locate the
service metadata in the registry
and develop the required client
components to bind and use the
service.
Services might aggregate information and data retrieved from other services or create
workflows of services to satisfy the request of a given service consumer. This practice is known
as service orchestration Another important interaction pattern is service choreography, which
is the coordinated interaction of services without a single point of control.

Components of SOA:

Guiding Principles of SOA

1. Standardized service contract: Specified through one or more service description documents.
2. Loose coupling: Services are designed as self-contained components, maintain relationships
that minimize dependencies on other services.

3. Abstraction: A service is completely defined by service contracts and description documents.

They hide their logic, which is encapsulated within their implementation.

4. Reusability: Designed as components, services can be reused more effectively, thus reducing
development time and the associated costs.

5. Autonomy: Services have control over the logic they encapsulate and, from a service consumer
point of view, there is no need to know about their implementation.

6. Discoverability: Services are defined by description documents that constitute supplemental

metadata through which they can be effectively discovered. Service discovery provides an
effective means for utilizing third-party resources.
7. Composability: Using services as building blocks, sophisticated and complex operations can be
implemented. Service orchestration and choreography provide a solid support for composing
services and achieving business goals.
Advantages of SOA:
• Service reusability: In SOA, applications are made from existing services. Thus, services can be
reused to make many applications.
• Easy maintenance: As services are independent of each other they can be updated and
modified easily without affecting other services.
• Platform independent: SOA allows making a complex application by combining services picked
from different sources, independent of the platform.
• Availability: SOA facilities are easily available to anyone on request.
• Reliability: SOA applications are more reliable because it is easy to debug small services rather
than huge codes
• Scalability: Services can run on different servers within an environment, this increases
scalability
Disadvantages of SOA:
• High overhead: A validation of input parameters of services is done whenever services interact
this decreases performance as it increases load and response time.
• High investment: A huge initial investment is required for SOA.
• Complex service management: When services interact they exchange messages to tasks. the
number of messages may go in millions. It becomes a cumbersome task to handle a large
number of messages.
Practical applications of SOA: SOA is used in many ways around us whether it is mentioned or
not.
1. SOA infrastructure is used by many armies and air forces to deploy situational awareness
systems.
2. SOA is used to improve healthcare delivery.
3. Nowadays many apps are games and they use inbuilt functions to run. For example, an app
might need GPS so it uses the inbuilt GPS functions of the device. This is SOA in mobile
solutions.
4. SOA helps maintain museums a virtualized storage pool for their information and content.

What is REST?
REpresentational State Transfer (REST) is a software architectural style that defines the
constraints to create web services. The web services that follows the REST architectural style
is called RESTful Web Services. It differentiates between the computer system and web
services. The REST architectural style describes the six barriers.
1. Uniform Interface

The Uniform Interface defines the interface between client and server. It simplifies and
decomposes the architecture which enables every part to be developed. The Uniform Interface
has four guiding principles:

•Resource-based: Individual resources are identified using the URI as a resource

identifier. The resources themselves are different from the representations returned to
the customer. For example, the server cannot send the database but represents some
database records expressed to HTML, XML or JSON depending on the server request
and the implementation details.
•Manipulation of resources by representation: When a client represents a resource
associated with metadata, there is information on the server to modify or delete it.
•Self-Descriptive Message: Each message contains enough information to describe
how the message is processed. For example, the parser can be specified by the Internet
media type (known as the MIME type).
•As the engine of Hypermedia Application State (HATEOAS): Customers provide
states by query-string parameters, body content, request headers, and requested URIs.
The services provide customers with the state by response codes, response
headers and body content. It is called hypermedia (hyperlink within hypertext).
•In addition to the above description, HATEOS also means that, where necessary, the
object or itself is contained in the linked body (or header) to supply the URI for retrieving
the related objects.
•The same interface that any REST services provide is fundamental to the design.

2. Client-server

A client-server interface separates the client from the server. For Example, the separation of
concerns not having an internal relationship with internal storage for each server to improve
the portability of customer's data codes. Servers are not connected with the user interface or
user status to make the server simpler and scalable. Servers and clients are independently
replaced and developed until the interface is changed.

3. Stateless

Stateless means the state of the service doesn't persist between subsequent requests and
response. It means that the request itself contains the state required to handle the request.
It can be a query-string parameter, entity, or header as a part of the URI. The URI identifies
the resource and state (or state change) of that resource in the unit. After the server
performs the appropriate state or status piece (s) that matters are sent back to the client
through the header, status, and response body.

•Most of us in the industry have been accustomed to programming with a container,

which gives us the concept of "session," which maintains the status among multiple
HTTP requests. In REST, the client may include all information to fulfil the server's
request and multiple requests in the state. Statelessness enables greater scalability
because the server does not maintain, update, or communicate any session state.
The resource state is the data that defines a resource representation.
Example, the data stored in a database. Consider the application state of having data that
may vary according to client and request. The resource state is constant for every customer
who requests it.

4. Layered system

It is directly connected to the end server or by any intermediary whether a client cannot tell.
Intermediate servers improve the system scalability by enabling load-
balancing and providing a shared cache. Layers can enforce security policies.

5. Cacheable

On the World Wide Web, customers can cache responses. Therefore, responses clearly define
themselves as unacceptable or prevent customers from reusing stale or inappropriate
data to further requests. Well-managed caching eliminates some client-server interactions to
improving scalability and performance.

6. Code on Demand (optional)

The server temporarily moves or optimizes the functionality of a client by logic that it executes.
Examples of compiled components are Java applets and client-side scripts.

Compliance with the constraints will enable any distributed hypermedia system with desirable
contingency properties such as performance, scalability, variability, visibility,
portability, and reliability.
Note: The optional lock of the REST architecture is Code on Demand. If a service violates a

constraint, it cannot be strictly referenced.

What is a system of systems?

A system of systems (SoS) is the collection of multiple, independent systems in context as part of a larger, more
complex system. A system is a group of interacting, interrelated and interdependent components that form a complex
and unified whole.
These independent and possibly distributed systems pool their resources together, creating a new and more complex
system. Individual systems in an SoS work together to provide functionalities and performance that none of the
independent systems, or constituent systems, could accomplish on their own.

The goal of an SoS architecture is to get maximum value out of a large system by understanding how each of the
smaller systems works, interfaces and is used. Such system designs require systems thinking -- a holistic approach to
analysis that focuses on the way constituent parts interoperate, work over time and function within the context of a
larger, evolving system.

The method for defining, abstracting, modeling and analyzing SoSes is called system of systems engineering. Many
systems engineers (SEs) work within the context of an SoS. The constituent systems in an SoS may be developed by
SEs independently and may continue to change over time independently.

In the data center, independent components of a large-scale system are connected through SoS-defined software
interfaces called middleware. Such programs ensure that constituents do not compete for subtasks within the larger
system and provide messaging services so that constituent systems can communicate. Constituent systems can also be
geographically distributed.

Introduced in 2016, the International Organization for Standardization (ISO) adopted standards for SoS engineering. In
2019, the following three standards were adopted:

• ISO/IEC/IEEE 21839 -- SoS considerations in lifecycle stages of a system

• ISO/IEC/IEEE 21840 -- Guidelines for the utilization of ISO/IEC/IEEE 15288 in the context of SoS

• ISO/IEC/IEEE 21841 -- Taxonomy of systems of systems

Why is a system of systems important?

A system of systems connects seemingly different systems with a larger, more complex configuration of systems. SoSes
enable the creation and operation of large and complex systems, such as manufacturing supply chains or the numerous
systems in an airplane, for example.
SoSes maintain operational independence of component systems, which also independently change over time as they
are managed. Systems engineers create frameworks to improve SoSes. They help make decisions in evolutionary
development, as well as determine infrastructure, policy or technology while also helping to solve any issues that
appear. SoS issues are typically complicated and may require intense resource or monetary investments.

What are the types of system of systems?

There are four types of system of systems: directed, acknowledged, collaborative and virtual. In most cases, an SoS is a
combination of these types and may change over time. The type of a SoS is based on the degree of independence of its
constituents as noted below:

• Directed. The SoS is created and managed to fulfill a specific purpose and the constituent systems operate
independently. However, independent operation is treated as less important.
 • Acknowledged. The SoS has a specific purpose, but the constituent systems maintain independent ownership,
objectives and development. Changes made in this SoS type are based on cooperative agreements between the
system and the SoS.

• Collaborative. Component systems freely interact with each other to fulfill a defined purpose. Management
authorities have little impact over the behavior of the component systems.

• Virtual. The SoS does not have central management authority or a centrally agreed-upon purpose. Typically, the
acquisition of a virtual SoS is unplanned and is made up of component systems that may not have been designed to
be integrated. Once its use is over, the components are normally disassembled and no longer operate in an SoS.

System of systems applications and examples

A system of systems can be applied in many settings. For example, they may appear in the following:

• Business. SoS can be found in banking and finance systems.

• Education. Universities such as Purdue, MIT and the Georgia Institute of Technology work on SoS problems.

• Government agencies. The Department of Defense (DoD), the U.S. Federal Aviation Administration and NASA
support research in systems of systems.

• Healthcare. Systems can be found in personal health management services and regional facilities management.

• Media. Systems can be found in film, radio and television.

• Transportation. SoS can be found in air traffic management systems, national and international rail network
systems, cargo transport, highway management, as well as space transportation systems.

What are web Services in Cloud Computing

Cloud Computing Web Services involve integrating web-based applications that are
implemented to allow open standards over an Internet Protocol backbone. It enables two
electronic devices to communicate over a network. The means are provided through different
standards that include HTTP, XML, SOAP, and WSDL, through which various applications can
communicate with each other. This makes the data and service provision across different
systems and platforms inter-operable and cooperative in the cloud.
Components of Web Services
•XML (Extensible Markup Language): XML can be described as a document tagging
language designed to use tag-based encoding when facilitating the creation of structured
documents. This is used in forming the format of structuring information being exchanged
between web services.
•SOAP (Simple Object Access Protocol): This is a communication platform that has no
relation to the transportation protocol. This is a specification for a framework, but SOAP is a
messaging protocol based on XML, and it is intended to be used together with other protocols
and technologies for building such a framework. The most beautiful thing about web services
and SOAP is that everything goes through HTTP, the standard web protocol.
•WSDL (Web Services Description Language): A Web service cannot be used if it can't be
found. The implementing client has to know where the web service is located. Also, to invoke
the correct web service, the client application has to understand what the web service does.
This is done with the help of Web services description language(WSDL). The WSDL file is
another XML-based file describing what the web service does to the client application. With this
WSDL document, the client application will know where the web service is located and how to
use it.
•UDDI (Universal Description, Discovery, and Integration): UDDI is a discovery of Web
services and publishing of Web services in the discovery type of repository. One of the possible
functionality deliverables includes the ability to let a service provider advertise their services
and enable the consumers to search for services that they need.
•REST (Representational State Transfer): REST is a working model or, more accurately, a
set of standards for building web applications. While the SOAP type of web service messages
use the XML language and are based on complex protocols, the REST type of SaaS messages
utilizes the HTTP standards and interchange data with the help of JSON.
•HTTP (Hypertext Transfer Protocol): HTTP is a protocol that is used when working with the
WWW, to exchange information between the clients’ and the servers’ sites. The protocol is
often used for Web services right now – both for services based on SOAP and REST.
•JSON (JavaScript Object Notation): JSON is an easy-to-read data format for the exchange
of data that is particularly used in REST-based web service systems. A specific type of markup
language, it is more compact than XML and is consistently chosen for its suitability and
convenience.
How Do Web Services Work?
1. Service Description (WSDL)
•Publishing the Service: According to the need of the service provider, the provider needs to
publish the web service with WSDL (Web Services Description Language). This information
contains the name of the service – or its location, the operations that enable it, the messages
passed between such services, and the data types employed.
•WSDL Document: This document represents a legal and digital agreement between the
specific service provider and the service consumer as to the ways of accessing the service.
2. Service Discovery (UDDI)
•Registering the Service: To use web service, the service provider first needs to list his
service in the UDDI (Universal Description, Discovery, and Integration). This is like a registry
that shows where Web services can be published and found.
•Finding the Service: UDDI is used to register and discover a web service where the service
consumer has to search for the most appropriate match. The Ws registry contains information
about the application services and their respective WSDL.
3. Service Invocation (SOAP/REST)
SOAP-based Web Services:
•Request Creation: The service consumer, to execute the service, will generate a SOAP
request message. Communicated below is the content of the message in XML and it is
compliant with the structure defined within the WSDL document.
•Sending the Request: The SOAP message is transferred over a network protocol over the
web to the service provider who has an endpoint.
•Processing the Request: The service provider then takes the SOAP request and completes
the required service on the message received from the requester.
•Response Creation: After it performs the requested operation, the service provider has to
construct a SOAP response message to return it to the consumer.
RESTful Web Services:
•HTTP Methods: Remisable services let developers leverage standard HTTP operations such
as GET, POST, PUT and DELETE. Each method represents the type of performed operation on
the data – read, write, modify, delete.
•Request Creation: The service consumer forges an HTTP request usually with a JSON
containing an expected language in the body of the request and specifying a URI for the target
resource.
•Sending the Request: This call is made in the form of an HTTP request sent to the endpoint
of the service provider.
•Processing the Request: The end-user’s HTTP request reaches the service provider where
the server processes the request and performs the necessary operation to respond.
•Response Creation: The service provider returns an HTTP response in most cases, which
includes a JSON body that contains the requested data or the status of the process.
4. Data Interchange (XML/JSON)
•XML: This specification is mainly applied to SOAP-based web services; however, XML formats
the request-response messages in such a way that they can be easily readable as well as
programmatically processible.
•JSON: As far as its application is concerned, JSON is mainly utilized in RESTful web services
but it is quite compact and is relatively easier to parse as compared to XML due to these
reasons the JSON format of data is quite suitable for web or mobile applications.
5. Service Security
•SSL/TLS: Protect messages transferred between the consumer of a service and the provider
by enciphering them.
•WS-Security: Safeguards the SOAP messages by enhancing the protocol through aspects
such as authentication, encryption, and digital signatures.
•OAuth: Mostly implemented in RESTful services to address a secure paradigm to control
access to the user information so that third-party applications can access them without
involving credentials.
•JWT: JSON Web Token is a compact, URL-safe means of representing claims to be transferred
among two parties in a portable data structure payload.
Features of Web Services
1. Interoperability
•Cross-Platform Communication: The interoperation of applications within the integrated
application incurs applications that are developed for two or more platforms and are coded in
different languages. Web services help in this process.
•Standardized Protocols: They employ standard protocols which include: HTTP, XML, SOAP,
and WSDL which enhance compatibility between the realms.
2. Extensibility
•Flexible Integration: One major advantage of web services is that they are highly
interoperable and portable hence they can easily be incorporated into new applications in an
organization without much alteration to existing systems and procedures.
•Modular Design: They are supposed to be developed in a rather logical manner, meaning
that new features may be incorporated into the system without influencing any of the services.
3. Scalability
•Distributed Computing: Web services are easily portable over different platforms and can
be implemented on different server instances so that issues with load balancing and failovers
can be easily addressed.
•Horizontal Scaling: They can be scaled horizontally towards bringing in more load by adding
another instance of the service.
4. Reusability
•Service Reusability: Web services enable medium and large companies to modularize their
business processes and use them in different applications with similar characteristics.
•Component Reuse: Services can be built on existing services making efficient reuse of
software in the form of available services possible.
5. Loose Coupling
•Minimal Dependency: Web services are inherently asynchronous so that the Web service
consumer and the Web service provider do not require entailment of implementations.
•Independent Deployment: This means that when a subtype is used instead of a more
general type of service implementation on the server side, it is not necessary to change the
client application as long as the interface is the same.
6. Discoverability
•UDDI Registries: To look for a particular service the services available can be registered and
searched through UDDI registries, for the available services.
•Dynamic Binding: The consumers are also able to procure the services dynamically and offer
their requests at runtime.
7. Security
•SSL/TLS: It is necessary to note that web services can employ SSL/TLS to provide secure
communication over the World Wide Web.
•WS-Security: With the current services, we can achieve WS-Security standards such as
message integrity, confidentiality and authentication with the use of SOAP-based services.
•OAuth: OAuth can be used for secure authorization in RESTful services.
•JWT: This is utilized in trustworthy messaging concerning RESTful services and identification.
8. Standardized Messaging
•SOAP: SOAP now has a messaging protocol that is articulated in XML which helps in
preserving the structure and manner in which messages are handled.
•REST: REST employs conventional HTTP methods and status assertion, which contributes
towards the reduction of the interaction model.
9. Support for Complex Operations
•Transaction Management: This means that web services can support multi-step and other
types of complex transactions, which will guarantee the right flow of processes.
•Asynchronous Processing: They can operate asynchronously, and this can be useful when
dealing with long-running operations since they can be worked on in the background.
10. Versatility
•Synchronous and Asynchronous: Web services may be either the request/response model,
where a Web service sends back confirmation of the receipt of a request and information
regarding how the request can be completed or the message-oriented model, where the two
elements exchange messages independent of each other.
•Various Payload Formats: It works with different payloads, though most applications use
either XML or JSON ones depending on the peculiarities of the given program.
11. Platform Independence
•Language Agnostic: The creation and use of Web services does not require the developer to
use any specific language but should support Web standards.
•Protocol Agnostic: Web services can run on different transports although, the most typical
transport is HTTP; other transports include SMTP and JMS.

What is Pub/Sub Architecture?

The Pub/Sub (Publisher/Subscriber) model is a messaging pattern used in software architecture
to facilitate asynchronous communication between different components or systems. In this
model, publishers produce messages that
are then consumed by subscribers. Key
points of the Pub/Sub model include:
•Publishers: Entities that generate and
send messages.
•Subscribers: Entities that receive and
consume messages.
•Topics: Channels or categories to which
messages are published.
•Message Brokers: Intermediaries that manage the routing of messages between publishers
and subscribers.

Components of Pub/Sub Architecture?

In the Pub/Sub (Publish/Subscribe) model, there are several key components that work together
to enable communication between publishers and subscribers. These components include:

•Publisher
The Publisher is responsible for creating and sending messages to the Pub/Sub system.
Publishers categorize messages into topics or channels based on their content. They
do not need to know the identity of the subscribers.
•Subscriber
 The Subscriber is a recipient of messages in the Pub/Sub system. Subscribers express
interest in receiving messages from specific topics.
They do not need to know the identity of the publishers. Subscribers receive messages
from topics to which they are subscribed.
•Topic
A Topic is a named channel or category to which messages are published. Publishers
send messages to specific topics, and subscribers can subscribe to one or more topics
to receive messages of interest.
Topics help categorize messages and enable targeted message delivery to interested
subscribers.
•Message Broker
The Message Broker is an intermediary component that manages the routing of
messages between publishers and subscribers.
It receives messages from publishers and forwards them to subscribers based on their
subscriptions.
The Message Broker ensures that messages are delivered to the correct subscribers
and can provide additional features such as message persistence, scalability, and
reliability.
•Message
A Message is the unit of data exchanged between publishers and subscribers in the
Pub/Sub system.
Messages can contain any type of data, such as text, JSON, or binary data. Publishers
create messages and send them to the Pub/Sub system, and subscribers receive and
process these messages.
•Subscription
A Subscription represents a connection between a subscriber and a topic.
Subscriptions define which messages a subscriber will receive based on the topics to
which it is subscribed.
Subscriptions can have different configurations, such as message delivery guarantees
(e.g., at-most-once, at-least-once) and acknowledgment mechanisms.
How does Pub/Sub Architecture work?
Below is how the Pub/Sub architecture works:
•Step 1: Publishers create messages and send them to the Pub/Sub system. They sort these
messages into topics or channels based on what the messages are about.
•Step 2: Subscribers let the system know which topics they’re interested in. They’ll get
messages only from those topics.
•Step 3: Topics are basically channels for messages. Publishers send their messages to specific
topics, and subscribers can choose one or more topics to follow.
•Step 4: Message brokers act as go-betweens, managing how messages get from publishers to
subscribers. They take messages from publishers and send them to the subscribers who are
interested.
•Step 5: When a publisher sends a message to a topic, the message broker grabs it and sends
it out to all the subscribers who have signed up for that topic.
•Step 6: The Pub/Sub system allows for asynchronous communication. This means publishers
can send messages without waiting for subscribers to be ready, and subscribers can pick up
messages whenever it suits them, without needing the publisher to be around
Real-World Example of Pub/Sub Architecture
A real-life example of Pub/Sub architecture can be seen in the operation of a social media
platform like Twitter.
•Publishers: Users post tweets, which are published to the Twitter platform.
•Subscribers: Followers of a user subscribe to their tweets to receive updates.
•Topics: Each user’s tweets can be considered a topic, and subscribers receive updates for the
topics they are interested in.
•Message Broker: Twitter’s backend infrastructure acts as the message broker, routing tweets
from publishers to subscribers.
•Message: Each tweet is a message that is published by the user and received by their
followers.
In this example, the Pub/Sub architecture allows for scalable and efficient distribution of tweets
to followers. Publishers (users) can publish tweets without knowing who will receive them, and
subscribers (followers) receive updates in real-time without the need for direct communication
with the publishers.
Use-cases of Pub/Sub Architecture
The Pub/Sub (Publisher/Subscriber) architecture is widely used in various scenarios where
asynchronous and scalable communication between components is required. Some common
use cases of Pub/Sub architecture include:
•Real-time Data Streaming: It’s often used in applications that deal with real-time data, like
IoT devices and sensor networks, allowing multiple subscribers to get data streams right away.
•Event-driven Architectures: This setup works well for systems that react to events instead
of constantly checking for updates, making applications more responsive.
•Message Queues: Pub/Sub can act as a message queue, temporarily holding messages until
subscribers can process them, which helps manage how messages are delivered.
•Notifications and Alerts: It’s used for sending notifications and alerts, letting publishers
send important updates that subscribers can receive instantly.
•Scalable Web Applications: In web apps, Pub/Sub supports features like real-time updates
and chat, so many users can receive information at the same time without overloading the
server.
•Microservices Communication: Finally, it helps microservices talk to each other, allowing
them to communicate without being tightly connected, which improves scalability and
reliability.
Types of Pub/Sub Services
Below are the two types of Pub/Sub Services:
1. Pub/Sub Service
This is the main messaging service that most users and applications choose. It provides:
•High Reliability: Ensures that messages are delivered consistently.
•Integrations: Supports a wide range of integrations with other services.
•Automatic Capacity Management: Handles scaling automatically based on demand.
•Data Replication: Synchronously replicates all data to at least two zones and offers best-
effort replication to a third zone for added reliability.
2. Pub/Sub Lite Service
This is a separate messaging service designed to be more cost-effective but comes with some
trade-offs:
•Lower Reliability: Offers less reliability compared to the standard Pub/Sub service.
•Zonal or Regional Storage: Zonal Lite topics are stored in one zone, while Regional Lite
topics replicate data to a second zone asynchronously.
•Pre-provisioning Required: You need to manage and provision your own storage and
throughput capacity.
•Cost-Effective: Consider this option if keeping costs low is essential, and you can accept the
lower reliability and some extra management tasks.
Comparing Pub/Sub to other Messaging Technologies
Below is the comparison of Pub/Sub to other messaging Technologies:
•Pub/Sub vs. Message Queues
Message Queues (e.g., RabbitMQ): Focus on delivering messages to one
consumer at a time, ensuring delivery and often preserving order. Best for point-to-
point communication.
Pub/Sub: Broadcasts messages to multiple subscribers simultaneously, making it
ideal for event-driven systems where many services need to react to the same event.
•Pub/Sub vs. Streaming Platforms
Streaming Platforms (e.g., Kafka): Designed for processing continuous streams of
data and can retain messages for longer periods. More complex but great for analytics.
Pub/Sub: Simpler and geared towards real-time delivery of messages without
extensive setup.
•Pub/Sub vs. WebSockets
WebSockets: Enable real-time, bidirectional communication between clients and
servers. Best for applications like chat.
Pub/Sub: Decouples publishers and subscribers, allowing for multiple subscribers to
receive the same message without needing direct connections.
•Pub/Sub vs. HTTP APIs
HTTP APIs: Follow a synchronous request-response model where clients wait for
replies. Good for direct interactions.
Pub/Sub: Allows asynchronous communication, letting publishers send messages
without waiting for subscribers to respond
When and When not to Use Pub/Sub Architecture
1. When to Use the Pub/Sub Architecture
Use Pub/Sub Architecture when:
•Subscribers don’t need to know about each other, making the system more flexible and easier
to scale.
•Pub/Sub helps you build systems that can grow easily. You can add more publishers or
subscribers without disrupting the existing setup.
•If you want parts of your system to communicate without waiting for each other, Pub/Sub is a
great option. Publishers can send messages without needing to wait for subscribers.
•This approach is perfect for event-driven systems. Publishers can send out events, and
subscribers can respond to those events without being tightly linked together.
•With Pub/Sub, subscribers can change their interests at runtime. They can subscribe to
different topics or types of messages, adding more flexibility to the system.
2. When Not to Use the Pub/Sub Architecture
Donot use Pub/Sub Architecture when:
•If you need very quick communication between parts of your system, Pub/Sub might not be
the best option. The process of routing messages and managing subscriptions can slow things
down.
•Using Pub/Sub can make your system more complicated, especially when it comes to routing
messages and handling subscriptions.
•Pub/Sub doesn’t guarantee that messages will be delivered in the order they were sent. If your
application needs messages to arrive in a specific sequence, Pub/Sub may not work for you.
•For smaller applications where a few components communicate directly, using Pub/Sub could
add unnecessary complexity that isn’t needed.
How Scalable and Secure is Pub/Sub Architecture?
The scalability and security of the Pub/Sub model depend on the implementation and the
specific requirements of the system. However, in general, the Pub/Sub model can be both
scalable and secure if implemented correctly.
•Scalability:
Horizontal Scalability: Pub/Sub systems can be designed to scale horizontally by
adding more publishers, subscribers, or message brokers. This allows the system to
handle a large number of messages and subscribers without a significant decrease in
performance.
Load Balancing: Pub/Sub systems can use load balancing techniques to distribute
messages evenly across multiple message brokers or nodes, ensuring efficient use of
resources and handling of high message volumes.
•Security:
Access Control: Pub/Sub systems can implement access control mechanisms to
ensure that only authorized publishers and subscribers can send or receive messages.
This helps protect against unauthorized access and data breaches.
 Encryption: Messages exchanged in a Pub/Sub system can be encrypted to protect
them from eavesdropping and ensure data confidentiality. Transport Layer Security
(TLS) can be used to secure message transmission over the network.
Authentication and Authorization: Pub/Sub systems can use authentication and
authorization mechanisms to verify the identity of publishers and subscribers and
ensure that they have the necessary permissions to send or receive messages.
•Challenges:
Message Ordering: Ensuring strict message ordering can be challenging in a
distributed Pub/Sub system, especially when messages are processed by multiple
subscribers concurrently.
Message Delivery Guarantees: Pub/Sub systems typically provide at-least-once or
at-most-once message delivery guarantees, which may not be suitable for all
applications requiring exactly-once semantics.
Benefits and Challenges of Pub/Sub Architecture
1. Benefits of Pub/Sub Architecture
Below are the benefits of Pub/Sub Architecture:
•Scalability: Pub/Sub systems can easily expand to accommodate numerous publishers,
subscribers, and messages. This flexibility comes from the separation of publishers and
subscribers, along with the use of message brokers to manage how messages are sent out.
•Decoupling: In a Pub/Sub setup, publishers and subscribers are kept separate, allowing them
to function independently. This separation simplifies the design and maintenance of the
system, making it easy to add or remove components as needed.
•Asynchronous Communication: Pub/Sub enables asynchronous communication between
components, enhancing the responsiveness and efficiency of the system. Publishers can send
messages without having to wait for subscribers, who can take their time processing the
messages.
•Reliability: Pub/Sub systems are designed to be dependable, incorporating features that
ensure messages are successfully delivered. This includes things like message
acknowledgments, retries, and strategies to deal with faults.
2. Challenges of Pub/Sub Architecture
Below are the challenges of Pub/Sub Architecture:
•Message Ordering: Pub/Sub systems usually don’t guarantee the order in which messages
reach subscribers. This can create problems for applications that need messages to arrive in a
specific sequence, as subscribers might get them out of order.
•Exactly-once Message Delivery: Achieving exactly-once message delivery can be tricky in
Pub/Sub systems, especially when there are failures or network problems. Creating methods to
ensure messages are delivered once without duplicates can be complicated.
•Latency: Pub/Sub systems can introduce delays because of how messages are routed and
delivered. Balancing low latency with scalability and reliability is especially challenging for real-
time applications where speed is crucial.
•Complexity: Setting up a Pub/Sub architecture can add complexity, particularly in large-scale
environments. Managing subscriptions, routing messages, and keeping everything consistent
across distributed components requires careful planning and oversight.
Pub/Sub Vs. Point to Point Messaging
Below are the differences between Pub/Sub and Point to Point Messaging:

Aspect Pub/Sub Messaging Point-to-Point Messaging

Messages are broadcast to multiple

Message Delivery Messages are delivered to a single receiver
subscribers

Subscriber Publishers do not need to know about

Sender needs to know the receiver
Knowledge subscribers

Scalable, as new subscribers can be added Less scalable, as each message is sent
Scalability
without affecting publishers directly to a specific receiver

Loosely coupled, as publishers and Tightly coupled, as sender and receiver are
Coupling
subscribers are decoupled directly connected

Suitable for broadcasting messages to

Use Case Suitable for one-to-one communication
multiple recipients

Google Cloud Pub/Sub, Amazon

Example Technology JMS, AMQP, RabbitMQ
SNS/SQS, Kafka

Virtualization
Virtualization is used to create a virtual version of an underlying service With the help of
Virtualization, multiple operating systems and applications can run on the same machine and
its same hardware at the same time, increasing the utilization and flexibility of hardware. It was
initially developed during the mainframe era.

It is one of the main cost-

effective, hardware-reducing,
and energy-saving techniques
used by cloud providers.
Virtualization allows sharing of
a single physical instance of a
resource or an application
among multiple customers and
organizations at one time. It
does this by assigning a logical
name to physical storage and
providing a pointer to that
physical resource on demand.
The term virtualization is often
synonymous with hardware
virtualization, which plays a fundamental role in efficiently delivering Infrastructure-as-a-
Service (IaaS) solutions for cloud computing. Moreover, virtualization technologies provide a
virtual environment for not only executing applications but also for storage, memory, and
networking.
•Host Machine: The machine on which the virtual machine is going to be built is known as Host
Machine.
•Guest Machine: The virtual machine is referred to as a Guest Machine.
Work of Virtualization in Cloud Computing
Virtualization has a prominent impact on Cloud Computing. In the case of cloud computing,
users store data in the cloud, but with the help of Virtualization, users have the extra benefit of
sharing the infrastructure. Cloud Vendors take care of the required physical resources, but
these cloud providers charge a huge amount for these services which impacts every user or
organization. Virtualization helps Users or Organisations in maintaining those services which
are required by a company through external (third-party) people, which helps in reducing costs
to the company. This is the way through which Virtualization works in Cloud Computing.
Benefits of Virtualization
•More flexible and efficient allocation of resources.
•Enhance development productivity.
•It lowers the cost of IT infrastructure.
•Remote access and rapid scalability.
•High availability and disaster recovery.
•Pay peruse of the IT infrastructure on demand.
•Enables running multiple operating systems.
Drawback of Virtualization
•High Initial Investment: Clouds have a very high initial investment, but it is also true that it
will help in reducing the cost of companies.
•Learning New Infrastructure: As the companies shifted from Servers to Cloud, it requires
highly skilled staff who have skills to work with the cloud easily, and for this, you have to hire
new staff or provide training to current staff.
•Risk of Data: Hosting data on third-party resources can lead to putting the data at risk, it has
the chance of getting attacked by any hacker or cracker very easily.
For more benefits and drawbacks, you can refer to the Pros and Cons of Virtualization.
Characteristics of Virtualization
•Increased Security: The ability to control the execution of a guest program in a completely
transparent manner opens new possibilities for delivering a secure, controlled execution
environment. All the operations of the guest programs are generally performed against the
virtual machine, which then translates and applies them to the host programs.
•Managed Execution: In particular, sharing, aggregation, emulation, and isolation are the
most relevant features.
•Sharing: Virtualization allows the creation of a separate computing environment within the
same host.
•Aggregation: It is possible to share physical resources among several guests, but
virtualization also allows aggregation, which is the opposite process.
For more characteristics, you can refer to Characteristics of Virtualization.
Types of Virtualization
1.Application Virtualization

2.Network Virtualization
3.Desktop Virtualization
4.Storage Virtualization

5.Server Virtualization
6.Data virtualization

1. Application Virtualization: Application virtualization helps a user to have remote access

to an application from a server. The server stores all personal information and other
characteristics of the application but can still run on a local workstation through the internet.
An example of this would be a user who needs to run two different versions of the same
software. Technologies that use application virtualization are hosted applications and packaged
applications.

2. Network Virtualization: The ability to run multiple

virtual networks with each having a separate control and
data plan. It co-exists together on top of one physical
network. It can be managed by individual parties that are
potentially confidential to each other. Network virtualization
provides a facility to create and provision virtual networks,
logical switches, routers, firewalls, load balancers, Virtual
Private Networks (VPN), and workload security within
days or even weeks.

3. Desktop Virtualization: Desktop virtualization allows the users’ OS to be remotely stored

on a server in the data center. It allows the user to access their desktop virtually, from any
location by a different machine. Users who want specific operating systems other than
Windows Server will need to have a virtual desktop. The main benefits of desktop virtualization
are user mobility, portability, and easy management of software installation, updates, and
patches.

4. Storage Virtualization: Storage virtualization is an array of servers that are managed by a

virtual storage system. The servers aren’t aware of exactly where their data is stored and
instead function more like worker bees in a hive. It makes managing storage from multiple
sources be managed and utilized as a single repository. storage virtualization software
maintains smooth operations, consistent performance, and a continuous suite of advanced
functions despite changes, breaks down, and differences in the underlying equipment.

5. Server Virtualization: This is a kind of

virtualization in which the masking of server
resources takes place. Here, the central server
(physical server) is divided into multiple different
virtual servers by changing the identity number,
and processors. So, each system can operate its
operating systems in an isolated manner. Where
each sub-server knows the identity of the central
server. It causes an increase in performance and
reduces the operating cost by the deployment of main server resources into a sub-server
resource. It’s beneficial in virtual migration, reducing energy consumption, reducing
infrastructural costs, etc.

6. Data Virtualization: This is the kind of virtualization in which the data is collected from
various sources and managed at a single place without knowing more about the technical
information like how data is collected, stored & formatted then arranged that data logically so
that its virtual view can be accessed by its interested people and stakeholders, and users
through the various cloud services remotely. Many big giant companies are providing their
services like Oracle, IBM, At scale, Cdata, etc.
Uses of Virtualization
•Data-integration
•Business-integration
•Service-oriented architecture data-services
•Searching organizational data

Implementation Levels of Virtualization In Cloud Computing

It is not simple to set up virtualization. Your computer runs on an operating system that gets configured on some
particular hardware. It is not feasible or easy to run a different operating system using the same hardware.
To do this, you will need a hypervisor. Now, what is the role of the hypervisor? It is a bridge between the hardware and
the virtual operating system, which allows smooth functioning.
Talking of the Implementation levels of virtualization in Cloud Computing., there are a total of five levels that are
commonly used. Let us now look closely at each of these levels of virtualization implementation in Cloud Computing.
1) Instruction Set Architecture Level (ISA)
ISA virtualization can work through ISA emulation. This is used to run many legacy codes written for a different
hardware configuration. These codes run on any virtual machine using the ISA. With this, a binary code that originally
needed some additional layers to run is now capable of running on the x86 machines. It can also be tweaked to run on
the x64 machine. With ISA, it is possible to make the virtual machine hardware agnostic.
For the basic emulation, an interpreter is needed, which interprets the source code and then converts it into a hardware
format that can be read. This then allows processing. This is one of the five implementation levels of virtualization in
Cloud Computing..
2) Hardware Abstraction Level (HAL)
True to its name HAL lets the virtualization perform at the level of the hardware. This makes use of a hypervisor which
is used for functioning. The virtual machine is formed at this level, which manages the hardware using the virtualization
process. It allows the virtualization of each of the hardware components, which could be the input-output device, the
memory, the processor, etc.
Multiple users will not be able to use the same hardware and also use multiple virtualization instances at the very same
time. This is mostly used in the cloud-based infrastructure.
3) Operating System Level
At the level of the operating system, the virtualization model is capable of creating a layer that is abstract between the
operating system and the application. This is an isolated container on the operating system and the physical server,
which uses the software and hardware. Each of these then functions in the form of a server.
When there are several users and no one wants to share the hardware, then this is where the virtualization level is used.
Every user will get his virtual environment using a dedicated virtual hardware resource. In this way, there is no question
of any conflict.
4) Library Level
The operating system is cumbersome, and this is when the applications use the API from the libraries at a user level.
These APIs are documented well, and this is why the library virtualization level is preferred in these scenarios. API
hooks make it possible as it controls the link of communication from the application to the system.
5) Application Level
The application-level virtualization is used when there is a desire to virtualize only one application and is the last of the
implementation levels of virtualization in Cloud Computing. One does not need to virtualize the entire environment of
the platform.
This is generally used when you run virtual machines that use high-level languages. The application will sit above the
virtualization layer, which in turn sits on the application program.
It lets the high-level language programs compiled to be used at the application level of the virtual machine run
seamlessly.

What is Virtualization Architecture?

Virtualization Architecture is defined as a
model that describes the concept of
virtualization. The use of virtualization is
important in cloud computing. In cloud
computing, end users share data through
an application called the cloud. However,
end users can share their entire IT
infrastructure with virtualization itself.

In the diagram above, virtualization includes virtual applications and virtual infrastructure
services.
•The virtual application services help in application management, and the virtual infrastructure
services can help in infrastructure management.
•Both services are integrated into the virtual data center or operating system. Virtual services
can be used on any platform and programming environment. These services can be accessed
from the local cloud or external cloud. In return, cloud users must pay a monthly or annual fee
to the third party.
•This fee is paid to third parties for providing cloud services to end users, who in turn provide
applications in different forms according to the needs of cloud end users.
•A hypervisor separates the operating system from the underlying hardware. It allows the host
computer to run multiple virtual machines simultaneously and share the same computer
resources.

Cloud computing and Virtualization Architecture

Virtualization in cloud computing helps create virtual versions of hardware such as desktop
computers with a virtual ecosystem of operating systems, storage, memory, and networking
services. Virtualization architecture uses the same hardware to run multiple operating systems
on the same machine and optimize their performance.
Virtualization and virtualization architecture are important concepts in cloud computing. In fact,
since the definition of cloud computing also includes virtual ecosystems, these terms are often
used interchangeably. Whether the ecosystem is private (i.e., cloud) or public (public cloud),
virtualization reduces the need for organizations to maintain physical (on-premises)
infrastructure for their computing needs. With cloud computing and virtualization architecture,
applications can be shared with many active users. With a public cloud like Amazon Web
Services (AWS) or Microsoft Azure, these can be shared with multiple businesses.
Types of Virtualization Architectures
There are two main types of virtualization architectures: hosted and bare metal.
Hosted Architecture
In this type of configuration, first, the host
operating system is installed on the
hardware, then the software is installed. The
software is a hypervisor or virtual machine
(VM) that requires many guest operating
systems or VMs to be installed on the
hardware to set up the virtualization
architecture. Once the hypervisor is in place,
applications can be installed and run on the
virtual machine as if they were installed on
the physical machine.

Bare Metal Architecture

In this architecture, the hypervisor is installed
directly on the hardware, not on top of the
operating system. Hypervisors and virtual
machines are configured the same way as
infrastructure. Bare metal virtualization
architecture is designed for applications that
provide real-time access or perform some form
of data processing.

More about Hypervisors in Virtualization Architecture

Virtualization is hypervisor based. A hypervisor separates the operating system and

applications from the underlying computer hardware so that the host computer can run
multiple virtual machines as guests and share physical resources such as network processor,
memory space, and network bandwidth. A hypervisor allocates memory or storage services
and distributes some of these services to each virtual machine according to the needs of the
virtualization architecture.
•Type 1 hypervisors: Sometimes called bare metal hypervisors, run directly on top of the host
system hardware. The bare metal hypervisor provides a high level of control and management.
Their direct access to system hardware provides better performance, scalability and stability.
Examples include: Microsoft Hyper-V, Citrix XenServer, VMware ESXi.
•Type 2 hypervisors: (also called management hypervisors) are installed on the host
operating system rather than directly on the hardware like Type 1 hypervisors. All guest
operating systems or virtual machines run on top of the hypervisor. The simplicity of the host
operation is known to facilitate the installation and management of the project. But adding a
layer of layering can limit functionality and introduce a security vulnerability. Example: VMware
Workstation Pro, VMware Fusion, Oracle VirtualBox, Oracle Solaris Zones, Oracle VM Server
for x86.
Benefits of a Virtualization Architecture
•Virtualization architecture provides cloud-native organizations with a flexible, cost-effective,
and versatile way to run multiple virtual machines or systems using a single service or host. As
a result, it provides efficient and economical deployment and reduces IT infrastructure costs.
•Since the host computer user can limit the number of users of virtual resources, the
environment provides better control while also reducing energy and resource consumption.
•Virtualization also enables remote, anytime, anywhere access, which is essential for remote or
isolated locations.
•It also reduces the need for physical equipment such as servers, resulting in more uptime and
fewer interruptions, better error management, and more efficient and capable product balance.
Resources can be replaced or removed as needed, providing business teams with greater
scalability and agility.
•All major cloud service providers offer virtualization solutions based on a one-time payment
model. This ensures that organizations only pay for resources that are actually used, not for
resources that are not used or have no capacity or potential. As a result, they have more
control over the weather budget and spending.
•Virtualization is also beneficial from a security perspective. You can configure tasks, devices,
applications, etc. on each virtual machine. There may be more than one guest user who can
run it. This extraction helps protect sensitive or business-critical data.
Virtualization and Containerization
The alternative to hypervisor-based virtualization in cloud computing is containerization. For
example, operating system virtualization is an important volume-based virtualization approach.
In this architecture, the operating system is set up to act as multiple separate systems,
allowing distributed applications to be deployed and run without having to turn on the entire
virtual machine for each system. Instead, multiple isolated machines (called volumes) run on a
single host, each accessing a single key.
Like virtual machines, container is a way to create virtual packages. Essentially, a container is a
lightweight virtual machine that is part of the same operating system instance or hypervisor.
But virtualization is a way to run multiple operating systems on a single physical server.
Containerization, on the other hand, is a way of running multiple applications on the same
machine in a virtual machine. VMs are best suited for applications that require full-scale
performance. Containers are a better choice for short-term projects that need to reduce the
number of servers used by multiple applications.

VIRTUALIZATION STRUCTURES/TOOLS AND MECHANISMS

CLOUD COMPUTING admin — July 20, 2021 · 10 Comments

In general, there are three typical classes of VM architecture. Figure 3.1 showed
the architectures of a machine before and after virtualization. Before
virtualization, the operating system manages the hardware. After virtualization,
a virtualization layer is inserted between the hardware and the operat-ing
system. In such a case, the virtualization layer is responsible for converting
portions of the real hardware into virtual hardware. Therefore, different
operating systems such as Linux and Windows can run on the same physical
machine, simultaneously. Depending on the position of the virtualiza-tion layer,
there are several classes of VM architectures, namely
the hypervisor architecture, para-

virtualization, and host-based virtualization. The hypervisor is also known as

the VMM (Virtual Machine Monitor). They both perform the same virtualization
operations.

1. Hypervisor and Xen Architecture

The hypervisor supports hardware-level virtualization (see Figure 3.1(b)) on
bare metal devices like CPU, memory, disk and network interfaces. The
hypervisor software sits directly between the physi-cal hardware and its OS.
This virtualization layer is referred to as either the VMM or the hypervisor. The
hypervisor provides hypercalls for the guest OSes and applications. Depending
on the functional-ity, a hypervisor can assume a micro-kernel architecture like
the Microsoft Hyper-V. Or it can assume a monolithic hypervisor
architecture like the VMware ESX for server virtualization.

A micro-kernel hypervisor includes only the basic and unchanging functions

(such as physical memory management and processor scheduling). The device
drivers and other changeable components are outside the hypervisor. A
monolithic hypervisor implements all the aforementioned functions, including
those of the device drivers. Therefore, the size of the hypervisor code of a
micro-kernel hyper-visor is smaller than that of a monolithic hypervisor.
Essentially, a hypervisor must be able to convert physical devices into virtual
resources dedicated for the deployed VM to use.

1.1 The Xen Architecture

Xen is an open source hypervisor program developed by Cambridge University.
Xen is a micro-kernel hypervisor, which separates the policy from the
mechanism. The Xen hypervisor implements all the mechanisms, leaving the
policy to be handled by Domain 0, as shown in Figure 3.5. Xen does not include
any device drivers natively [7]. It just provides a mechanism by which a guest
OS can have direct access to the physical devices. As a result, the size of the
Xen hypervisor is kept rather small. Xen provides a virtual environment located
between the hardware and the OS. A number of vendors are in the process of
developing commercial Xen hypervisors, among them are Citrix XenServer [62]
and Oracle VM [42].

The core components of a Xen system are the hypervisor, kernel, and
applications. The organi-zation of the three components is important. Like other
virtualization systems, many guest OSes can run on top of the hypervisor.
However, not all guest OSes are created equal, and one in
particular controls the others. The guest OS, which has control ability, is called
Domain 0, and the others are called Domain U. Domain 0 is a privileged guest
OS of Xen. It is first loaded when Xen boots without any file system drivers
being available. Domain 0 is designed to access hardware directly and manage
devices. Therefore, one of the responsibilities of Domain 0 is to allocate and
map hardware resources for the guest domains (the Domain U domains).

For example, Xen is based on Linux and its security level is C2. Its management
VM is named Domain 0, which has the privilege to manage other VMs
implemented on the same host. If Domain 0 is compromised, the hacker can
control the entire system. So, in the VM system, security policies are needed to
improve the security of Domain 0. Domain 0, behaving as a VMM, allows users
to create, copy, save, read, modify, share, migrate, and roll back VMs as easily
as manipulating a file, which flexibly provides tremendous benefits for users.
Unfortunately, it also brings a series of security problems during the software
life cycle and data lifetime.

Traditionally, a machine’s lifetime can be envisioned as a straight line where

the current state of the machine is a point that progresses monotonically as the
software executes. During this time, con-figuration changes are made, software
is installed, and patches are applied. In such an environment, the VM state is
akin to a tree: At any point, execution can go into N different branches where
multiple instances of a VM can exist at any point in this tree at any given time.
VMs are allowed to roll back to previous states in their execution (e.g., to fix
configuration errors) or rerun from the same point many times (e.g., as a
means of distributing dynamic content or circulating a “live” system image).

2. Binary Translation with Full Virtualization

Depending on implementation technologies, hardware virtualization can be
classified into two cate-gories: full virtualization and host-based virtualization.
Full virtualization does not need to modify the host OS. It relies on binary
translation to trap and to virtualize the execution of certain sensitive,
nonvirtualizable instructions. The guest OSes and their applications consist of
noncritical and critical instructions. In a host-based system, both a host OS and
a guest OS are used. A virtuali-zation software layer is built between the host
OS and guest OS. These two classes of VM architec-ture are introduced next.

2.1 Full Virtualization

With full virtualization, noncritical instructions run on the hardware directly
while critical instructions are discovered and replaced with traps into the VMM
to be emulated by software. Both the hypervisor and VMM approaches are
considered full virtualization. Why are only critical instructions trapped into the
VMM? This is because binary translation can incur a large performance
overhead. Noncritical instructions do not control hardware or threaten the
security of the system, but critical instructions do. Therefore, running
noncritical instructions on hardware not only can promote efficiency, but also
can ensure system security.

2.2 Binary Translation of Guest OS Requests Using a VMM

This approach was implemented by VMware and many other software
companies. As shown in Figure 3.6, VMware puts the VMM at Ring 0 and the
guest OS at Ring 1. The VMM scans the instruction stream and identifies the
privileged, control- and behavior-sensitive instructions. When these instructions
are identified, they are trapped into the VMM, which emulates the behavior of
these instructions. The method used in this emulation is called binary
translation. Therefore, full vir-tualization combines binary translation and direct
execution. The guest OS is completely decoupled from the underlying
hardware. Consequently, the guest OS is unaware that it is being virtualized.

The performance of full virtualization may not be ideal, because it involves

binary translation which is rather time-consuming. In particular, the full
virtualization of I/O-intensive applications is a really a big challenge. Binary
translation employs a code cache to store translated hot instructions to
improve performance, but it increases the cost of memory usage. At the time of
this writing, the performance of full virtualization on the x86 architecture is
typically 80 percent to 97 percent that of the host machine.

2.3 Host-Based Virtualization

An alternative VM architecture is to install a virtualization layer on top of the
host OS. This host OS is still responsible for managing the hardware. The guest
OSes are installed and run on top of the virtualization layer. Dedicated
applications may run on the VMs. Certainly, some other applications
can also run with the host OS directly. This host-based architecture has some
distinct advantages, as enumerated next. First, the user can install this VM
architecture without modifying the host OS. The virtualizing software can rely
on the host OS to provide device drivers and other low-level services. This will
simplify the VM design and ease its deployment.

Second, the host-based approach appeals to many host machine

configurations. Compared to the hypervisor/VMM architecture, the performance
of the host-based architecture may also be low. When an application requests
hardware access, it involves four layers of mapping which downgrades
performance significantly. When the ISA of a guest OS is different from the ISA
of the underlying hardware, binary translation must be adopted. Although the
host-based architecture has flexibility, the performance is too low to be useful
in practice.

3. Para-Virtualization with Compiler Support

Para-virtualization needs to modify the guest operating systems. A para-
virtualized VM provides special APIs requiring substantial OS modifications in
user applications. Performance degradation is a critical issue of a virtualized
system. No one wants to use a VM if it is much slower than using a physical
machine. The virtualization layer can be inserted at different positions in a
machine soft-ware stack. However, para-virtualization attempts to reduce the
virtualization overhead, and thus improve performance by modifying only the
guest OS kernel.

Figure 3.7 illustrates the concept of a paravirtualized VM architecture. The

guest operating systems are para-virtualized. They are assisted by an
intelligent compiler to replace the nonvirtualizable OS instructions by
hypercalls as illustrated in Figure 3.8. The traditional x86 processor offers four
instruction execution rings: Rings 0, 1, 2, and 3. The lower the ring number, the
higher the privilege of instruction being executed. The OS is responsible for
managing the hardware and the privileged instructions to execute at Ring 0,
while user-level applications run at Ring 3. The best example of para-
virtualization is the KVM to be described below.

3.1 Para-Virtualization Architecture

When the x86 processor is virtualized, a virtualization layer is inserted
between the hardware and the OS. According to the x86 ring definition, the
virtualization layer should also be installed at Ring 0. Different instructions at
Ring 0 may cause some problems. In Figure 3.8, we show that para-
virtualization replaces nonvirtualizable instructions with hypercalls that
communicate directly with the hypervisor or VMM. However, when the guest
OS kernel is modified for virtualization, it can no longer run on the hardware
directly.

Although para-virtualization reduces the overhead, it has incurred other

problems. First, its compatibility and portability may be in doubt, because it
must support the unmodified OS as well. Second, the cost of maintaining para-
virtualized OSes is high, because they may require deep OS kernel
modifications. Finally, the performance advantage of para-virtualization varies
greatly due to workload variations. Compared with full virtualization, para-
virtualization is relatively easy and more practical. The main problem in full
virtualization is its low performance in binary translation. To speed up binary
translation is difficult. Therefore, many virtualization products employ the para-
virtualization architecture. The popular Xen, KVM, and VMware ESX are good
examples.
3.2 KVM (Kernel-Based VM)
This is a Linux para-virtualization system—a part of the Linux version 2.6.20
kernel. Memory management and scheduling activities are carried out by the
existing Linux kernel. The KVM does the rest, which makes it simpler than the
hypervisor that controls the entire machine. KVM is a hardware-assisted para-
virtualization tool, which improves performance and supports unmodified guest
OSes such as Windows, Linux, Solaris, and other UNIX variants.

3.3 Para-Virtualization with Compiler Support

Unlike the full virtualization architecture which intercepts and emulates
privileged and sensitive instructions at runtime, para-virtualization handles
these instructions at compile time. The guest OS kernel is modified to replace
the privileged and sensitive instructions with hypercalls to the hypervi-sor or
VMM. Xen assumes such a para-virtualization architecture.

The guest OS running in a guest domain may run at Ring 1 instead of at Ring 0.
This implies that the guest OS may not be able to execute some privileged and
sensitive instructions. The privileged instructions are implemented by
hypercalls to the hypervisor. After replacing the instructions with hypercalls,
the modified guest OS emulates the behavior of the original guest OS. On an
UNIX system, a system call involves an interrupt or service routine. The
hypercalls apply a dedicated service routine in Xen.

Example 3.3 VMware ESX Server for Para-Virtualization

VMware pioneered the software market for virtualization. The company has
developed virtualization tools for desktop systems and servers as well as
virtual infrastructure for large data centers. ESX is a VMM or a hypervisor for
bare-metal x86 symmetric multiprocessing (SMP) servers. It accesses hardware
resources such as I/O directly and has complete resource management control.
An ESX-enabled server consists of four components: a virtualization layer, a
resource manager, hardware interface components, and a service console, as
shown in Figure 3.9. To improve performance, the ESX server employs a para-
virtualization architecture in which the VM kernel interacts directly with the
hardware without involving the host OS.

The VMM layer virtualizes the physical hardware resources such as CPU,
memory, network and disk controllers, and human interface devices. Every VM
has its own set of virtual hardware resources. The resource manager allocates
CPU, memory disk, and network bandwidth and maps them to the virtual
hardware resource set of each VM created. Hardware interface components are
the device drivers and the
VIRTUALIZATION OF CPU, MEMORY, AND I/O
DEVICES

To support virtualization, processors such as the x86 employ a special running mode
and instructions, known as hardware-assisted virtualization. In this way, the VMM
and guest OS run in different modes and all sensitive instructions of the guest OS and
its applications are trapped in the VMM. To save processor states, mode switching is
completed by hardware. For the x86 architecture, Intel and AMD have proprietary
technologies for hardware-assisted virtualization.

1. Hardware Support for Virtualization

Modern operating systems and processors permit multiple processes to run

simultaneously. If there is no protection mechanism in a processor, all instructions
from different processes will access the hardware directly and cause a system crash.
Therefore, all processors have at least two modes, user mode and supervisor mode, to
ensure controlled access of critical hardware. Instructions running in supervisor mode
are called privileged instructions. Other instructions are unprivileged instructions. In
a virtualized environment, it is more difficult to make OSes and applications run
correctly because there are more layers in the machine stack. Example 3.4 discusses
Intel’s hardware support approach.

At the time of this writing, many hardware virtualization products were available.
The VMware Workstation is a VM software suite for x86 and x86-64 computers. This
software suite allows users to set up multiple x86 and x86-64 virtual computers and
to use one or more of these VMs simultaneously with the host operating system. The
VMware Workstation assumes the host-based virtualization. Xen is a hypervisor for
use in IA-32, x86-64, Itanium, and PowerPC 970 hosts. Actually, Xen modifies Linux
as the lowest and most privileged layer, or a hypervisor.

One or more guest OS can run on top of the hypervisor. KVM (Kernel-based
Virtual Machine) is a Linux kernel virtualization infrastructure. KVM can support
hardware-assisted virtualization and paravirtualization by using the Intel VT-x or
AMD-v and VirtIO framework, respectively. The VirtIO framework includes a
paravirtual Ethernet card, a disk I/O controller, a balloon device for adjusting guest
memory usage, and a VGA graphics interface using VMware drivers.

Example 3.4 Hardware Support for Virtualization in the Intel x86 Processor

Since software-based virtualization techniques are complicated and incur performance overhead, Intel
provides a hardware-assist technique to make virtualization easy and improve performance. Figure
3.10 provides an overview of Intel’s full virtualization techniques. For processor virtualization, Intel offers
the VT-x or VT-i technique. VT-x adds a privileged mode (VMX Root Mode) and some instructions to
processors. This enhancement traps all sensitive instructions in the VMM automatically. For memory
virtualization, Intel offers the EPT, which translates the virtual address to the machine’s physical
addresses to improve performance. For I/O virtualization, Intel implements VT-d and VT-c to support
this.
2. CPU Virtualization

A VM is a duplicate of an existing computer system in which a majority of the VM

instructions are executed on the host processor in native mode. Thus, unprivileged
instructions of VMs run directly on the host machine for higher efficiency. Other
critical instructions should be handled carefully for correctness and stability. The
critical instructions are divided into three categories: privileged
instructions, control-sensitive instructions, and behavior-sensitive instructions.
Privileged instructions execute in a privileged mode and will be trapped if executed
outside this mode. Control-sensitive instructions attempt to change the configuration
of resources used. Behavior-sensitive instructions have different behaviors depending
on the configuration of resources, including the load and store operations over the
virtual memory.

A CPU architecture is virtualizable if it supports the ability to run the VM ’s

privileged and unprivileged instructions in the CPU’s user mode while the VMM runs
in supervisor mode. When the privileged instructions including control- and
behavior-sensitive instructions of a VM are exe-cuted, they are trapped in the VMM.
In this case, the VMM acts as a unified mediator for hardware access from different
VMs to guarantee the correctness and stability of the whole system. However, not all
CPU architectures are virtualizable. RISC CPU architectures can be naturally
virtualized because all control- and behavior-sensitive instructions are privileged
instructions. On the contrary, x86 CPU architectures are not primarily designed to
support virtualization. This is because about 10 sensitive instructions, such
as SGDT and SMSW, are not privileged instructions. When these instruc-tions
execute in virtualization, they cannot be trapped in the VMM.

On a native UNIX-like system, a system call triggers the 80h interrupt and passes
control to the OS kernel. The interrupt handler in the kernel is then invoked to
process the system call. On a para-virtualization system such as Xen, a system call in
the guest OS first triggers the 80h interrupt nor-mally. Almost at the same time,
the 82h interrupt in the hypervisor is triggered. Incidentally, control is passed on to
the hypervisor as well. When the hypervisor completes its task for the guest OS
system call, it passes control back to the guest OS kernel. Certainly, the guest OS
kernel may also invoke the hypercall while it’s running. Although paravirtualization
of a CPU lets unmodified applications run in the VM, it causes a small performance
penalty.

2.1 Hardware-Assisted CPU Virtualization

This technique attempts to simplify virtualization because full or paravirtualization is

complicated. Intel and AMD add an additional mode called privilege mode level
(some people call it Ring-1) to x86 processors. Therefore, operating systems can still
run at Ring 0 and the hypervisor can run at Ring -1. All the privileged and sensitive
instructions are trapped in the hypervisor automatically. This technique removes the
difficulty of implementing binary translation of full virtualization. It also lets the
operating system run in VMs without modification.

Example 3.5 Intel Hardware-Assisted CPU Virtualization

Although x86 processors are not virtualizable primarily, great effort is taken to virtualize them. They are used
widely in comparing RISC processors that the bulk of x86-based legacy systems cannot discard easily.
Virtuali-zation of x86 processors is detailed in the following sections. Intel’s VT-x technology is an example of
hardware-assisted virtualization, as shown in Figure 3.11. Intel calls the privilege level of x86 processors the
VMX Root Mode. In order to control the start and stop of a VM and allocate a memory page to maintain the

CPU state for VMs, a set of additional instructions is added. At the time of this writing, Xen, VMware,
and the Microsoft Virtual PC all implement their hypervisors by using the VT-x technology.

Generally, hardware-assisted virtualization should have high efficiency. However, since the transition from
the hypervisor to the guest OS incurs high overhead switches between processor modes, it sometimes
cannot outperform binary translation. Hence, virtualization systems such as VMware now use a hybrid
approach, in which a few tasks are offloaded to the hardware but the rest is still done in software. In addition,
para-virtualization and hardware-assisted virtualization can be combined to improve the performance further.

3. Memory Virtualization

Virtual memory virtualization is similar to the virtual memory support provided by

modern operat-ing systems. In a traditional execution environment, the operating
system maintains mappings of virtual memory to machine memory using page
tables, which is a one-stage mapping from virtual memory to machine memory. All
modern x86 CPUs include a memory management unit (MMU) and a translation
lookaside buffer (TLB) to optimize virtual memory performance. However, in a
virtual execution environment, virtual memory virtualization involves sharing the
physical system memory in RAM and dynamically allocating it to the physical
memory of the VMs.

That means a two-stage mapping process should be maintained by the guest OS and
the VMM, respectively: virtual memory to physical memory and physical memory to
machine memory. Furthermore, MMU virtualization should be supported, which is
transparent to the guest OS. The guest OS continues to control the mapping of virtual
addresses to the physical memory addresses of VMs. But the guest OS cannot
directly access the actual machine memory. The VMM is responsible for mapping the
guest physical memory to the actual machine memory. Figure 3.12 shows the two-
level memory mapping procedure.

Since each page table of the guest OSes has a separate page table in the VMM
corresponding to it, the VMM page table is called the shadow page table. Nested
page tables add another layer of indirection to virtual memory. The MMU already
handles virtual-to-physical translations as defined by the OS. Then the physical
memory addresses are translated to machine addresses using another set of page
tables defined by the hypervisor. Since modern operating systems maintain a set of
page tables for every process, the shadow page tables will get flooded. Consequently,
the perfor-mance overhead and cost of memory will be very high.

VMware uses shadow page tables to perform virtual-memory-to-machine-memory

address translation. Processors use TLB hardware to map the virtual memory directly
to the machine memory to avoid the two levels of translation on every access. When
the guest OS changes the virtual memory to a physical memory mapping, the VMM
updates the shadow page tables to enable a direct lookup. The AMD Barcelona
processor has featured hardware-assisted memory virtualization since 2007. It
provides hardware assistance to the two-stage address translation in a virtual
execution environment by using a technology called nested paging.

Example 3.6 Extended Page Table by Intel for Memory Virtualization

Since the efficiency of the software shadow page table technique was too low, Intel developed a hardware-
based EPT technique to improve it, as illustrated in Figure 3.13. In addition, Intel offers a Virtual Processor ID
(VPID) to improve use of the TLB. Therefore, the performance of memory virtualization is greatly improved.
In Figure 3.13, the page tables of the guest OS and EPT are all four-level.

When a virtual address needs to be translated, the CPU will first look for the L4 page table pointed to by
Guest CR3. Since the address in Guest CR3 is a physical address in the guest OS, the CPU needs to
convert the Guest CR3 GPA to the host physical address (HPA) using EPT. In this procedure, the CPU will
check the EPT TLB to see if the translation is there. If there is no required translation in the EPT TLB, the
CPU will look for it in the EPT. If the CPU cannot find the translation in the EPT, an EPT violation exception
will be raised.

When the GPA of the L4 page table is obtained, the CPU will calculate the GPA of the L3 page table by using
the GVA and the content of the L4 page table. If the entry corresponding to the GVA in the L4
page table is a page fault, the CPU will generate a page fault interrupt and will let the guest OS kernel handle
the interrupt. When the PGA of the L3 page table is obtained, the CPU will look for the EPT to get the HPA of
the L3 page table, as described earlier. To get the HPA corresponding to a GVA, the CPU needs to look for
the EPT five times, and each time, the memory needs to be accessed four times. There-fore, there are 20
memory accesses in the worst case, which is still very slow. To overcome this short-coming, Intel increased
the size of the EPT TLB to decrease the number of memory accesses.

4. I/O Virtualization

I/O virtualization involves managing the routing of I/O requests between virtual
devices and the shared physical hardware. At the time of this writing, there are three
ways to implement I/O virtualization: full device emulation, para-virtualization, and
direct I/O. Full device emulation is the first approach for I/O virtualization.
Generally, this approach emulates well-known, real-world devices.
All the functions of a device or bus infrastructure, such as device enumeration,
identification, interrupts, and DMA, are replicated in software. This software is
located in the VMM and acts as a virtual device. The I/O access requests of the guest
OS are trapped in the VMM which interacts with the I/O devices. The full device
emulation approach is shown in Figure 3.14.

A single hardware device can be shared by multiple VMs that run concurrently.
However, software emulation runs much slower than the hardware it emulates
[10,15]. The para-virtualization method of I/O virtualization is typically used in Xen.
It is also known as the split driver model consisting of a frontend driver and a
backend driver. The frontend driver is running in Domain U and the backend dri-ver
is running in Domain 0. They interact with each other via a block of shared memory.
The frontend driver manages the I/O requests of the guest OSes and the backend
driver is responsible for managing the real I/O devices and multiplexing the I/O data
of different VMs. Although para-I/O-virtualization achieves better device
performance than full device emulation, it comes with a higher CPU overhead.

Direct I/O virtualization lets the VM access devices directly. It can achieve close-to-
native performance without high CPU costs. However, current direct I/O
virtualization implementations focus on networking for mainframes. There are a lot
of challenges for commodity hardware devices. For example, when a physical device
is reclaimed (required by workload migration) for later reassign-ment, it may have
been set to an arbitrary state (e.g., DMA to some arbitrary memory locations) that can
function incorrectly or even crash the whole system. Since software-based I/O
virtualization requires a very high overhead of device emulation, hardware-assisted
I/O virtualization is critical. Intel VT-d supports the remapping of I/O DMA transfers
and device-generated interrupts. The architecture of VT-d provides the flexibility to
support multiple usage models that may run unmodified, special-purpose,
or “virtualization-aware” guest OSes.

Another way to help I/O virtualization is via self-virtualized I/O (SV-IO) [47]. The
key idea of SV-IO is to harness the rich resources of a multicore processor. All tasks
associated with virtualizing an I/O device are encapsulated in SV-IO. It provides
virtual devices and an associated access API to VMs and a management API to the
VMM. SV-IO defines one virtual interface (VIF) for every kind of virtua-lized I/O
device, such as virtual network interfaces, virtual block devices (disk), virtual camera
devices, and others. The guest OS interacts with the VIFs via VIF device drivers.
Each VIF consists of two mes-sage queues. One is for outgoing messages to the
devices and the other is for incoming messages from the devices. In addition, each
VIF has a unique ID for identifying it in SV-IO.

Example 3.7 VMware Workstation for I/O Virtualization

The VMware Workstation runs as an application. It leverages the I/O device support in guest OSes,
host OSes, and VMM to implement I/O virtualization. The application portion (VMApp) uses a driver
loaded into the host operating system (VMDriver) to establish the privileged VMM, which runs directly
on the hardware. A given physical processor is executed in either the host world or the VMM world, with
the VMDriver facilitating the transfer of control between the two worlds. The VMware Workstation
employs full device emulation to implement I/O virtualization. Figure 3.15 shows the functional blocks
used in sending and receiving packets via the emulated virtual NIC.
The virtual NIC models an AMD Lance Am79C970A controller. The device driver for a Lance controller
in the guest OS initiates packet transmissions by reading and writing a sequence of virtual I/O ports;
each read or write switches back to the VMApp to emulate the Lance port accesses. When the last
OUT instruc-tion of the sequence is encountered, the Lance emulator calls a normal write() to the
VMNet driver. The VMNet driver then passes the packet onto the network via a host NIC and then the
VMApp switches back to the VMM. The switch raises a virtual interrupt to notify the guest device driver
that the packet was sent. Packet receives occur in reverse.

5. Virtualization in Multi-Core Processors

Virtualizing a multi-core processor is relatively more complicated than virtualizing a

uni-core processor. Though multicore processors are claimed to have higher
performance by integrating multiple processor cores in a single chip, muti-core
virtualiuzation has raised some new challenges to computer architects, compiler
constructors, system designers, and application programmers. There are mainly two
difficulties: Application programs must be parallelized to use all cores fully, and
software must explicitly assign tasks to the cores, which is a very complex problem.

Concerning the first challenge, new programming models, languages, and libraries
are needed to make parallel programming easier. The second challenge has spawned
research involving scheduling algorithms and resource management policies. Yet
these efforts cannot balance well among performance, complexity, and other issues.
What is worse, as technology scales, a new challenge called dynamic
heterogeneity is emerging to mix the fat CPU core and thin GPU cores on the same
chip, which further complicates the multi-core or many-core resource management.
The dynamic heterogeneity of hardware infrastructure mainly comes from less
reliable transistors and increased complexity in using the transistors [33,66].

5.1 Physical versus Virtual Processor Cores

Wells, et al. [74] proposed a multicore virtualization method to allow hardware

designers to get an abstraction of the low-level details of the processor cores. This
technique alleviates the burden and inefficiency of managing hardware resources by
software. It is located under the ISA and remains unmodified by the operating system
or VMM (hypervisor). Figure 3.16 illustrates the technique of a software-visible
VCPU moving from one core to another and temporarily suspending execution of a
VCPU when there are no appropriate cores on which it can run.

5.2 Virtual Hierarchy

The emerging many-core chip multiprocessors (CMPs) provides a new computing

landscape. Instead of supporting time-sharing jobs on one or a few cores, we can use
the abundant cores in a space-sharing, where single-threaded or multithreaded jobs
are simultaneously assigned to separate groups of cores for long time intervals. This
idea was originally suggested by Marty and Hill [39]. To optimize for space-shared
workloads, they propose using virtual hierarchies to overlay a coherence and
caching hierarchy onto a physical processor. Unlike a fixed physical hierarchy, a
virtual hierarchy can adapt to fit how the work is space shared for improved
performance and performance isolation.

Today’s many-core CMPs use a physical hierarchy of two or more cache levels that
statically determine the cache allocation and mapping. A virtual hierarchy is a cache
hierarchy that can adapt to fit the workload or mix of workloads [39]. The hierarchy’s
first level locates data blocks close to the cores needing them for faster access,
establishes a shared-cache domain, and establishes a point of coherence for faster
communication. When a miss leaves a tile, it first attempts to locate the block (or
sharers) within the first level. The first level can also pro-vide isolation between
independent workloads. A miss at the L1 cache can invoke the L2 access.

The idea is illustrated in Figure 3.17(a). Space sharing is applied to assign three
workloads to three clusters of virtual cores: namely VM0 and VM3 for database
workload, VM1 and VM2 for web server workload, and VM4–VM7 for middleware
workload. The basic assumption is that each workload runs in its own VM. However,
space sharing applies equally within a single operating system. Statically distributing
the directory among tiles can do much better, provided operating sys-tems or
hypervisors carefully map virtual pages to physical frames. Marty and Hill suggested
a two-level virtual coherence and caching hierarchy that harmonizes with the
assignment of tiles to the virtual clusters of VMs.

Figure 3.17(b) illustrates a logical view of such a virtual cluster hierarchy in two
levels. Each VM operates in a isolated fashion at the first level. This will minimize
both miss access time and performance interference with other workloads or VMs.
Moreover, the shared resources of cache capacity, inter-connect links, and miss
handling are mostly isolated between VMs. The second level maintains a globally
shared memory. This facilitates dynamically repartitioning resources without costly
cache flushes. Furthermore, maintaining globally shared memory minimizes changes
to existing system software and allows virtualization features such as content-based
page sharing. A virtual hierarchy adapts to space-shared workloads like
multiprogramming and server consolidation. Figure 3.17 shows a case study focused
on consolidated server workloads in a tiled architecture. This many-core mapping
scheme can also optimize for space-shared multiprogrammed workloads in a single-
OS environment.