POLYTECHNIC UNIVERSITY OF THE PHILIPPINES

College of Business Administration

Department of Marketing Management

INSTRUCTIONAL
MATERIAL
IN
FUNDAMENTALS OF
Prescriptive analytics
BUMA 30083

Mecmack A. Nartea

Course Objectives:
✓ Understand data management concepts and criticality of data availability in order to make reliable
business decisions.

✓ Demonstrate understanding of business intelligence including the importance of data gathering, data
storing, data analyzing and accessing data.

✓ Apply different analytics modeling concepts on enterprise data.

✓ Understand the functions and data access constraints of various departments with an organization and
provide compliance reports.

✓ Work on various analytics tools available in the market for various business functions.

✓ Participate actively in business discussions with various departments and create common reports or
specific/unique reports with regard to predictive and prescriptive analytics.

✓ Understand the business process as they relate to data analysis and optimization.

✓ Convery results of data analysis to organizational stakeholders at various levels.

1
 TABLE OF CONTENTS

Part 1: Understanding Simulation and Analytics

Chapter 1 Analytics and Simulation Basics 3

Chapter 2 Simulation and Business Processes 27

Chapter 3 Build the Conceptual Model 36

Chapter 4 Build the Simulation 52

Chapter 5 Use Simulation for Descriptive, Predictive and Prescriptive

Analytics 64

Part 2: Simulation Case Studies

2
Chapter 1
Analytics and Simulation Basics

Organizations need to provide goods and services that meet customer needs such as
low price, fast delivery, wide range, and high quality. To do this, these organizations operate
as complex systems with many internal parts interacting with an external environment that
is ever changing in response to forces such as technological advances. Because of this
increasing complexity, organizations re- quire tools that can help them both understand
their current business processes and plan for future changes in response to their internal
and external environment. This book is about the how simulation modeling of business
processes for descriptive, predictive, and descriptive analytics can attempt to explain
behavior and thus help make decisions in the face of an uncertain future. Figure 1.1 shows
these three areas in context and shows how they work together.

Figure 1.1: Simulating business processes for descriptive, predictive, and prescriptive analytics.
Simulation can take many forms, but the type of simulation that is the focus of this
text is based on a mathematical model that can implemented on a computer.

3
 These models can be used to represent a simplified version of a real system in order
to aid its understanding and to provide a prediction of its future behavior. The use of models
allows us to overcome the drawbacks of predicting future behavior with the real system
which can be costly, time-consuming, unfeasible while testing many design options and in
some cases dangerous for safety critical systems. In order to determine the “best” option
for future actions, we may use optimization techniques. Both mathematical modeling and
optimization techniques fall under the areas of operations research as the focus on business
applications has expanded management science. These terms are often used
interchangeably or under the umbrella terms operations research/management science
(OR/MS). Optimization may also come under the information systems area when the
optimization is enabled by program code in the form of machine- learning algorithms.
The type of simulation can vary and so can its application range into physics,
chemistry, biology and many other areas. This text is focused on the application of
simulation to analyze processes within business organizations. The process perspective is
associated with the area of operations management, which consid- ers the transformation
of materials, information and customers into goods and services. This transformation is
undertaken by processes that require the allocation of resources such as people and
equipment. Business process management (BPM) is a discipline that is focused on the use
of business processes as a way of improving organizational performance. Deriving and
using process performance measures is a key aspect of both operations management and
BPM. In operations management performance is often measured using the metrics of cost,
quality, speed, flexibility and dependability. These measures not only provide an indication
of performance but the identification and pursuit of a subset of these measures provide a
way of connecting the strategic direction of the company with its operational decisions.
Analytics or business analytics can be seen as incorporating the use of modeling and
statistics from OR/MS and information systems capabilities such as the storage of big data in
order to transform data into actions through analysis and insights in the context of
organizational decision making. A key part of analytics is the use of performance measures
to assess business performance.
Analytics is usually associated with the use of large data sets termed big data and
computer programs running algorithms to process that data in what is known as data-
driven analysis. This text is focused on the use of a model-driven approach using simulation
to analyze business processes to produce analytic outcomes. So before describing
simulation in more detail, the data- and model- driven perspectives for the analysis of
business processes will be covered. This analysis will include the possibility of combining
the data- and model-driven approaches.

Data- and Model-Driven Analysis

In the context of analyzing organizational business processes, analytics can be classified into
the following:
4
 Descriptive Analytics
– This is the use of reports and visual displays to explain or understand past and current
business performance.
– Descriptive data-driven reports often contain statistical summaries of metrics such as sales
and revenue and are intended to provide an outline of trends in current and past
performance. Model-driven techniques are often used for descriptive analysis in the context
of the design of new products and processes when little current data exists.

Predictive Analytics
– This is the ability to predict future performance to help plan for the future.
– Data-driven models often do this by detecting patterns or relationships in historical data
and then projecting these relationships into the future. Model-driven approaches use
domain knowledge to construct a simplified representation of the structure of the system
that is used to predict the future.

Prescriptive Analytics
– This is the ability to recommend a choice of action from predictions of future performance.
– Data-driven models often do this by recommending an optimum decision based on the need
to maximize (or minimize) some aspect of performance. Model-driven approaches may use
optimization software to try many different scenarios until one is found that best meets the
optimization criteria.

A data-driven modeling approach aims to derive a description of behavior from

observations of a system so that it can describe how that system behaves (its output) under
different conditions or scenarios (its input). Because they can only describe the relationship
between input and output, they are called descriptive models. One approach is to use
pattern recognition as a way to build a model that allows us to make predictions. The idea
of pattern recognition is based on learning relationships through examples. Pattern
recognition is achieved through techniques such as associations, sequences, classification
and clustering of the data. These techniques are implemented in models that use equations,
logical statements and algorithms to find the patterns. In essence, this approach produces
a model that imitates real behavior based on past observations of that behavior termed a
descriptive model. This imitation can be achieved by defining a relationship that relates
model input to model output. Generally, the more data (observations) that can be used to
form the description, the more accurate the description will be and thus the interest in big
data analytics that uses large data sets. Machine learning uses a selection of learning
algorithms that use large data sets and a desired outcome to derive an algorithm that can
be used for descriptive, predictive, and prescriptive analytics.

A model-driven modeling approach aims to explain a system’s behavior not just derived
from its inputs but through a representation of the internal system’s structure. The model-
driven approach is a well-recognized way of understanding the world based on a systems
approach in which a real system is simplified into its essential elements (its processes) and
relationships between these elements (its structure). Thus in addition to input data,
5
 information is required on the system's processes, the function of these processes and the
essential parts of the relationships between these processes. These models are called
explanatory models as they represent the real system and attempt to explain the behaviour
that occurs. This means that the effect of a change on design of the process can be assessed
by changing the structure of the model. These models generally have far smaller data needs
than data-driven models because of the key role of the representation of structure. For
example, we can represent a supermarket by the customers that flow through the
supermarket and the pro- cesses they undertake—collecting groceries and paying at the till.
A model would then not only enable us to show current customer waiting time at the
supermarket tills (descriptive analytics) but also allow us to change the design of the system
such as changing the number of tills and predict the effect on customer waiting time
(predictive analytics). We can also specify the target customer waiting time based on the
number of tills required (prescriptive analytics). However most real systems are very
complex—a supermarket has many different staff undertaking many processes using
different resources—for example, the collection and unpacking of goods, keeping shelves
stocked, heating and ventilation systems, etc. It is usually not feasible to include all the
elements of the real system, so a key part of modeling is making choices about which parts
of the system should be included in the model in order to obtain useful results. This
simplification process may use statistics in the form of mathematical equations to represent
real-life processes (such as the customer arrival rate) and a computer program (algorithm)
in the form of process logic to represent the sequence of activities that occur within a
process.

Simulation for Descriptive, Predictive, and Prescriptive Analytics

Simulation is not simply a predictive or even a prescriptive tool but can also be used in a descriptive mode to
develop understanding. Here the emphasis is not necessarily on developing accurate predictive models but on using
the simulation model to help develop theories regarding how an organizational system works. In this role simulation
is used as an experimental methodology where we can explore the effect of different parameters by running the
simulation under many different conditions. What we do is start with a deductive method in which we have a set of
assumptions and test these assumptions and their consequences. We then use an experimental method to generate
data which can be analyzed in an inductive manner to develop theories by generalization of observations. In fact the
simulation analyst can alternate between a deductive and inductive approach as the model is developed.

Data-Driven Analysis Techniques

In general terms, there are many analysis techniques that can be considered as data-
driven techniques including regression analysis, econometric modeling, time series
experiments and yield management. However, data-driven techniques considered here are
most often associated with big data analytics. These techniques re- late to those that are
used for the analysis on large-scale data sets termed big data. A brief description follows of
each of the main categories of big data-driven analytics techniques.

6
 Data Mining

In a general sense, data mining can be defined as identifying patterns in complex and ill-
defined data sets. Particular data mining techniques include the following:
– Identifying associations involves establishing relationships about items that occur at a
particular point in time (e.g., what items are bought together in a supermarket).
– Identifying sequences involves showing the order in which actions occur (e.g., click-stream
analysis of a website).
– Classification involves analyzing historical data into patterns to predict future behavior
(e.g., identifying groups of website users who display similar visitor patterns).
– Clustering involves finding groups of facts that were previously unknown (e.g. identifying
new market segments of customers or detecting e-commerce fraud).

There are various categories of mining depending on the nature of the data that is being
analyzed. For example, there is text mining for document analysis and web mining of
websites.

Machine Learning

Machine learning uses an iterative approach for the analysis of prepared training and test
sample data in order to produce an analytical model. Through the use of iteration, learning
algorithms build a model that may be used to make predictions. This model may be in the
form of a mathematical equation, a rule set or an algorithm. Thus, machine learning does
not refer to actual learning by a machine (computer) but the use of algorithms that through
iteration provide an ability to predict outcomes from a data set. The main steps involved in
machine learning are preprocessing of the data set, creation of a training set (usually 80% of
the data) and a test set (usually 20% of the data) and selection of a learning algorithm to
process the data.

Supervised machine learning relates to learning algorithms that build models that can
be used to make predictions using classification and regression techniques while
unsupervised machine learning relates to identifying similar items using clustering
techniques. In supervised machine learning, our training data sets have values for both our
input (predictor) and output (outcome) variables that are known to us so that we can use
classification techniques such as support vector machines (SVMs) and regression
techniques such as linear regression, decision trees (DTs) and neural networks for
prediction. In unsupervised learning our training data sets have values for our input
(predictor) variables but not for our output (outcome) variables so this approach involves
7
examining attributes of a data set in order to determine which items are most similar to one
another.

This clustering function can be achieved using techniques such as K-Means algorithms
and neural networks. In addition to the categories of supervised and unsupervised machine
learning, Reinforcement Learning is a subfield of machine learning that uses learning
algorithms that explore options and when they achieve their aim, deduce how to get to that
successful endpoint in the future. A reinforcement approach can be implemented by the

Simulation vs Machine Learning for Prediction

The model-driven approach of simulation requires the model builder to understand causations and codify
them in the model. The model then permits prediction by running the model into the future—simulation.
Machine Learning’s great promise is by using a data-driven approach it can generate algorithms that may
provide predictions. However there are a number of challenges for the Machine Learning approach when
used for prediction
– Although the prediction algorithm is generated, the learning algorithm and training method must be
devised to enable this. This task can be challenging.
– We often do not understand how the prediction algorithm has arrived at its prediction. Thus
algorithms based on approaches such as neural networks are “black box” and are thus difficult to
validate.
– The data used to train and test the algorithm is based on a fixed period of time (i.e. a sample) and
thus may not cover all required learning examples—this is termed incompleteness.
– There is a need to distinguish natural variation in the data from changes in the data due to rare or
infrequent behavior not representative of typical behavior—this is termed noise
– As the context of the prediction widens the number of potential variables impacting on the prediction
increases vastly. Thus there is a need for increasingly massive data sets to cover the “state space” of
the effects of these variables.

use of a reward and penalty system to guide a choice from a number of random options.
Simulation is particularly relevant for this type of machine learning as it can provide a virtual
environment in which the reinforcement training can take place safely and far quicker than
in a real system.

Process Mining

The use of process mining involves obtaining and extracting event data to pro- duce an
event log and transforming the event log into a process model termed process discovery.
The process model can then be used to check conformance of the system with the process
design and to measure the performance of the process. In terms of event log construction,
the data required to make an event log can come from a variety of sources including
collected data in spreadsheets, databases and data warehouses or directly from data
streams. The minimum data required to construct an event log consists of a list of process
instances (i.e., events), which are related to a case identification number and for each event

8
a link to an activity label such as “check ticket.” Activities may reoccur in the event log, but
each event is unique and events within a case need to be presented in order of execution in
the event log so that casual dependencies can be derived in the process model. It is also
usual for there to be a timestamp associated with each event in the event log. Additional
attributes associated with each event may also be included such as the association of a
resource required to undertake the event and the estimated cost of the event.

Once we are satisfied that the process model does provide a suitable representation of behavior,
then we can use the model in a normative mode and judge discrepancies in terms of deviations
from the ideal behavior shown by the model. Undesirable behavior is when deviations occur
due to unwanted actions (for example, not obtaining authorization for a purchase) and
desirable deviations occur when actions occur that are outside normal parameters but show
flexibility in meeting the process objectives (for example, providing additional customer service).
Conformance checking of processes against a normative model is a major use of process
mining. In addition to conformance checking, process mining can be used to assess
performance across a number of dimensions by providing additional information in the event
log, which is subsequently incorporated into the process model. For example, performance can
be reviewed by associating resources to the people undertaking the activities. The interactions
between people can be mapped in a social network to provide an organizational perspective. In
addition, a cost perspective can be achieved by associating costs with activities.

Visual Analytics

The basic idea of visual analytics is to present large-scale data in some visual form, allowing
people to interact with the data to understand processes better. In order to facilitate better
understanding of data, software that provides a visual representation of data is available in the
form of applications such as spreadsheets, dashboards and scorecards. In conjunction with
their statistical and forecasting capabilities, spread- sheets are particularly useful at providing
graphical displays of trends such as sales for analysis by an organization. To meet the needs of
managers who do not use computers frequently, a graphical interface, called a dashboard (or a
digital dashboard), permits decision makers to understand statistics collated by an
organization. A dashboard display is a graphical display on the computer presented to the
decision maker, which includes graphical images such as meters, bar graphs, trace plots and text
fields to con- vey real-time information. Dashboards incorporate drill-down features to enable
data to be interrogated in greater detail if necessary. Dashboards should be designed so that the
data displayed can be understood in context. For example, sales figures can be dis- played
against sales figures for the previous time period or the same time period in the previous year.
Figures can also be compared against targets and competitors. For ex- ample, quality
performance can be benchmarked against best-in-class competitors in the same industry. The
visual display of data can also be used to show the amount of difference between performance
and targets both currently and the trend over time. Visual indicators, such as traffic lights, can
be used to show when performance has fallen below acceptable levels (red light) is a cause for
9
concern (amber light) and is acceptable (green light).

While dashboards are generally considered to measure operational performance,

scorecards provide a summary of performance over a period of time. Scorecards may be
associated with the concept of the balanced scorecard strategy tool and examine data from
the balanced scorecard perspectives of financial, customer, business process and learning
and growth.

Data Farming

Data farming is the purposeful generation of data from computer-based models, including
simulation models. Large-scale simulation experiments can be initiated by varying many
input variables, examining many different scenarios or both. Data farming offers the
possibility of using simulation to generate big data, with the ad- vantage that the data
generated is under the control of the modeler. However, the implementation of data
farming may require the use of simulation software with a relatively fast execution speed.

People Analytics

Some of the pitfalls around data driven analytics are shown by the use of people analytics in
organizations. People analytics deals with perceptual data and data based on intangible variables rather
than the factual data used in finance for example. Historically data on people within a business has been
used for applications such as workforce modeling in order to match the supply of people and skills to
planned workload. Performance measurement of people has also taken place in the context of the business
itself. However the use of big data to drive analytics has seen the development of people analytic models
that provide measurement based on data gathered on a massive scale. The idea is that the sheer scale of
data will improve the accuracy of the analytical process and allow “fact-based” decisions to be made on
people at the individual level. However as Cathy O’Neil (2016) found, the complexity of people has led to
a number of pitfalls with the use of people analytic methods, including:
Proxy measures are used to attempt to measure complex human behaviors that may not be an accurate
representation.
The algorithms have inbuilt feedback loops that reinforce the assumptions of the model leading to
self-fulfilling results.
There is inbuilt bias by model builders reflecting their viewpoint on people’s behaviors.
There is a lack of transparency of the workings of the models leading to a lack of knowledge around the
limitations of the results of the models and a lack of accountability regarding the model’s validity.

Model-Driven Analysis Techniques

Model-driven analysis techniques use a model that can be defined as a simplified

representation of a real system that is used to improve our understanding of that real
system. The representation is simplified because the complexity of most systems means
that it is infeasible to provide all details of the real system in the model.

10
Indeed, the simplification process actually benefits understanding, where it allows a focus
on the elements of the system that are relevant to the decision. For this reason, a model
should be as simple as possible, while being valid, in order to meet its objectives. The
modeling process thus involves deciding what is relevant and should be included in the
model to meet the aims of the current investigation.

The model then provides information for decision making that can be used to make
predictions of real-world system behavior (Figure 1.2).

Figure 1.2: The modeling process.

There are many different approaches to modeling, but mathematical models represent a
system as a number of mathematical variables (termed state variables) with mathematical
equations used to describe how these state variables change over time. An important
distinction between mathematical models is the classification between static (fixed in time)
or dynamic (change over time), with dynamics systems being modeled using a continuous
or discrete approach (Figure 1.3).

Linear programming
Spreadsheets
Monte Carlo simulation

System dynamics

Discrete-event simulation
agent-based simulation

Figure 1.3: Categories of mathematical models.

11
Static Mathematical Models

Static models include the use of a computer spreadsheet, which is an example of a

numerical static model in which relationships can be constructed and studied for different
scenarios. Another example of a static numerical model is the Monte Carlo simulation
method. This consists of experimental sampling with random numbers and deriving results
based on these. Although random numbers are being used, the problems that are being
solved are essentially determinate. The Monte Carlo method is widely used in risk analysis
for assessing the risks and benefits of decisions. Linear programming is a modeling
technique that seeks defined goals when a set of variables and constraints are given. A
linear programming technique is data envelopment analysis (DEA), which is a method for
calculating efficiency. DEA can be used as a benchmarking tool to generate a score that
indicates the relative distance of an entity to the best practices so as to measure its overall
performance compared with its peers. This overall performance measured by DEA can be
manifested in the form of a composite measure that aggregates individual indicators.

Dynamic Mathematical Models

A dynamic mathematical model allows changes in system attributes to be derived as a

function of time. A classification is made between continuous and discrete model types. A
discrete system changes only at separate points in time. For example, the number of
customers in a service system is dependent on individual arrivals and departures of
customers at discrete points in time.

Continuous systems vary over time; for example, the amount of petrol in a tanker
being emptied is varying continuously over time and is thus classified as a continuous
system. In practice most continuous systems can be modeled as discrete and vice versa at
different levels of detail. Also, systems will usually have a mixture of both discrete and
continuous elements. In general, continuous models are used at a high level of abstraction,
for example, investigating cause-and-effect linkages in organizational systems, while
discrete models are used to model business processes. The system dynamics (SD)
approach is described as an example of a continuous mathematical model, while discrete-
event simulation (DES) is described as a discrete mathematical modeling approach.

Simulation

Simulation is a particular kind of dynamic modeling in which the model (usually

represented on a computer) is “run” forward through (simulated) time. This book is
focused on the use of simulation in an organizational context to measure business process
performance. In order to use simulation, we must represent a theory of how the
organization works (conceptual model) and transform that into a procedure that can be
represented as a computer program (simulation model). Simulation has an experimental
12
methodology in that we can explore the effect of different parameters by running the
simulation under many different conditions. From these observations, we can refine our
theory about how the organization works and can make predictions about how it might
work in the future. Thus simulation can be used to:
➢ Understand past and current behavior of business processes (descriptive
analytics).
➢ Predict the future behavior of business processes (predictive analytics).
➢ Recommend action based on the future behavior of business processes
(prescriptive analytics).

Data- and Model-Driven Analysis with Simulation and Analytics

So far we have defined data- and model-driven approaches to the analysis of business
processes. Analytics is categorized as a data-driven approach and simulation is categorized
as a model-driven approach. There are instances, however, of the use of analytics techniques
that are driven by data generated from a model that will be termed model-driven analytics
and simulations that are data driven, termed data- driven simulation.
Simulation and analytics and thus each of these combinations attempt to codify the real
world into a computer model that can be used for understanding and prediction of the real
system. This reality will usually be based on knowledge of only a part of all the data that
exists (or ever existed) about the real system. The relationship between data-driven, model-
driven, analytics and simulation is presented in this context. Figure 1.3 shows how the four
combinations of simulation and analytic analysis can be represented by four types of reality
that reflect their different emphasis in terms of the use of a subset of all the data that exists
that is related to a system.

Data-driven Model-driven

Selected reality Farmed reality

Data (raw) Data (simulated)

Digital reality Simplified reality

Data (analyzed) Data (sampled)

Figure 1.3: Data- and model-driven analysis with simulation and analytics.

13
 The categories in Figure 1.3 cover the following:
– Data-driven analytics techniques that use raw data to learn from the past to represent a
selected reality based on the variables and observations included. This is the data-driven
approach described earlier and is represented by analytics techniques such as data mining,
machine learning and process mining. Data-driven analytics represent a selected reality in
that no matter how large the data sets used for analysis they will only present a selected
view of all the data generated by a process over time.

– Model-driven simulation techniques that use sampled data from the past to rep- resent a
simplified reality. This is the model-driven approach described earlier and is represented
by the technique of simulation. This is termed a simplified reality as the modeling process
employs a simplification of reality by removing elements that are not considered relevant
to the study objectives.
– Data-driven simulations that use analyzed data to drive simulation to provide a digital
reality. These applications allow data, which may be processed through analytic techniques
such as process mining, data mining and machine learning, to advance the capability of
simulation model development and experimentation. The use of a data-driven approach to
provide model-building capabilities and thus enable recoding of the model to reflect the
actual state of a system is a particularly important advance represented by the use of
applications such as digital twins. This is termed digital reality as the approach is used to
construct a real-time digital replica of a physical object.
– Model-driven analytics that use simulated data to drive analytics techniques to provide a
farmed reality. This enables simulation to be used for training and testing machine-
learning algorithms and facilitating the use of analytic techniques for future system
behaviors and for systems that do not currently exist. This is termed a farmed reality in
reference to the term data farming, which refers to the use of a simulation model to generate
synthetic data.

The categories in Figure 1.3 cover data-driven analytics techniques that use raw data to learn
from the past to represent a selected reality based on the variables and observations included;
and model-driven simulation techniques that use sampled data from the past to represent
a simplified reality. The predictive capabilities of both of these approaches are limited by the
transient nature of organizational processes. No matter how large the dataset used in a data-
driven approach it may not describe a future behavior owing to changes in the system
causing that behavior. This will occur at least until the new behavior has been incorporated
into the data provided to the learning algorithms. For model-driven approaches no matter
how large the model we may not incorporate a future behavior owing to the simplified
representation of the model, at least until we have recoded the model to incorporate the
cause of that behavior. Two further categories are shown in Figure 1.5, data-driven simulation
that use data from analytics to drive simulation to provide a digital reality; and model-
driven analytics that use data from simulation to drive analytics techniques to pro- vide a
farmed reality. In terms of data-driven simulation, practitioners need to take into account

14
the limitations of the data-driven approach in terms of the use of historical data to represent
the future of a transient system. In terms of model-driven analytics simulation, here the
limitation is based around the use of a sampled dataset that is a simplification of the raw
data generated by the real system.
A barrier to the combined use of simulation and analytics is the different back- grounds and
skillsets of simulation and analytics practitioners. Simulation practitioners typically
combine the technical knowledge required to undertake simulation such as model building
and statistical methods with an understanding of an application domain such as
manufacturing or healthcare.

In a business setting analytics may be undertaken by teams consisting of data scientists with
data, statistical and IT skills, business analysts with deep domain knowledge and IT
professionals to develop data products. Many simulation practitioners began their
simulation careers coding models in simulation languages such as SIMAN and using
languages such as FORTRAN for file processing. However in the light of the development
of drag and drop interfaces in such tools as Arena, recent users may find it a particular
challenge to adapt to the need for coding when developing a machine learning algorithm in
Matlab, R or Python. One way of addressing this issue may be to emphasize the need for
training of simulation practitioners in data science techniques and the adoption of a multi-
disciplinary approach to research and training.

Types of Simulation

There follows a brief overview of the three main simulation approaches, namely,
SD, agent-based simulation (ABS) and DES. The three methods have their own
philosophies, communities of users and main areas of application.

System Dynamics (SD)

SD is a modeling technique that was originally developed by Professor Jay Forrester when it
was known as industrial dynamics. In SD models, stocks of variables are connected together
via flows. SD has been used extensively in a wide range of application areas, for example,
economics, supply chain, ecology and population dynamics to name a few. SD has a well-
developed methodology in that the main stages and phases of the construction of a model are
defined. SD attempts to describe systems in terms of feedback and delays. Negative feedback
loops provide a control mechanism that compares the output of a system against a target and
adjusts the input to eliminate the difference. Instead of reducing this variance between actual
output and target out- put, positive feedback adds the variance to the output value and thus

15
increases the overall variance. Most systems consist of a number of positive and negative
feedback cycles, which make them difficult to understand. Adding to this complexity is the time
delay that will occur between the identification of the variation and action taken to eliminate
it and the taking of that action and its effect on output. What often occurs is a cycle of
overshooting and undershooting the target value until the variance is eliminated. The SD
concept can be implemented using computer software such as Stella II. A system is represented
by a number of stocks (also termed levels) and flows (also termed rates). A stock is an
accumulation of a resource such as materials and a flow is the movement of this resource that
leads to the stock rising, falling or remaining constant. A characteristic of stocks is that they
will remain in the system even if flow rates drop to zero and they act to decouple flow rates.
An example is a safety stock of finished goods which provides a buffer between a production
system which manufactures them at a constant rate and fluctuating external customer demand
for the goods. An SD flow diagram maps out the relationships between stocks and flows. In
Stella II, resource flows are represented by a double arrow and information flows by a single
arrow. Stocks are represented by rectangles. Converters, which are used for a variety of tasks
such as combining flows, are represented by a circle.

Agent-Based Simulation (ABS)

The use of agents in the design of simulation models has its origins in complexity of science
and game theory. Agents are components in a system (for example, a person or an
organization) that have a set of rules or behavior that controls how they take in information,
process that information and effect change on their environment. ABS refers to the study of
the behavior of agents from the bottom up. This means that agent behaviors are defined,
and then the agents are released into the environment of study. The behavior of the agents
then emerges as a consequence of their interaction. In this sense, the system behavior is an
emergent property of the agent interactions and the main source of structural change in the
system itself is in the form of the relationship between the agents. ABS has been applied
across a wide area, for example, economics, human behavior, supply chains, emergency
evacuation, transport and healthcare. A particular class of agent-based systems termed
multiagent simulations are concerned with modeling both individual agents (with

autonomy and interactivity) and also the emergent system behavior that is a consequence
of the agent’s collective actions and interactions.

Discrete-Event Simulation (DES)

DES takes a process view of the world and individual entities can be represented as they
move between different resources and are processed or wait in queues. It is hard to estimate
the number of global users of DES, but there is little doubt that of the three types of
simulation outlined here, DES has the largest user base. Evidence for this is provided by the
biannual simulation survey carried out by OR/MS Today, which demonstrates the wide
16
 range of applications for which DES has been used. The main areas of application are
manufacturing, supply chain and logistics, military and more recently healthcare. DES is
concerned with the modeling of systems that can be represented by a series of events. The
simulation describes each individual event, moving from one to the next as time progresses.
When constructing a DES, the system being simulated is seen as consisting of a number
of entities (e.g., products, people) that have a number of attributes (e.g., product type, age).
An entity may consume work in the form of people or a machine, termed a resource. The
amount and timing of resource availability may be specified by the model user. Entities may
wait in a queue if a resource is not available when required. The main components of a DES
are as follows:
– Event—an instantaneous occurrence that may change the state of the system.
– Entity—an object (e.g., material, information, people) that moves through the simulation
activating events.
– Attribute—a characteristic of an entity. An entity may have several attributes associated
with it (e.g., component type).
– Resource—an object (e.g., equipment, person) that provides a service to an entity (e.g.,
lathe machine, shop assistant).

For a DES, a system consists of a number of objects (entity) that flow from point to point in
a system while competing with each other for the use of scarce resources (resource). The
approach allows many objects to be manipulated at one time by dealing with multiple
events at a single point in time. The attributes of an entity may be used to determine future
actions taken by the entities. In DES time is moved forward in discrete chunks from event
to event, ignoring any time between those events. Thus, the simulation needs to keep a
record of when future events will occur and activate them in time order. These event timings
are kept on what is termed as the simulation calendar that is a list of all future events in time
order. The simulation calendar is also known as the future event list. The simulation
executes by advancing through these events sequentially. When an event has been
completed, the simulation time—stored as a data value called the simulation clock—is
advanced in a discrete step to the time of the next event. This loop behavior of executing all
events at a particular time and then advancing the simulation clock is controlled by the
control program or executive of the simulation. There are three main methods of executive

control.

In an event-based simulation, future events are scheduled on an event list. In the first phase
of the approach, the executive program simply advances the simulation clock to the time of
the next event. At the second phase, all events at that particular clock time are then
executed. Any new events that are derived from these events are added to the simulation
calendar. When all events have been executed at the current time, the executive program
advances the simulation clock to the time of the next event and the loop repeats. The

17
 simulation continues until no events remain on the simulation calendar or a termination
event is executed.
The activity-based approach works by scanning activities at a fixed time interval and
activities that satisfy the necessary conditions are immediately scheduled. Unlike the event-
based approach, the activity scanning method does not require event lists to be maintained.
However, the method is relatively inefficient and there- fore slow because of the number of
unnecessary scans that are needed when no events may be occurring. Also an event may be
scheduled between two consecutive scans and thus will not be activated at the correct time.
Most commercial software uses the process-based approach, which allows the user to
enter a program in a more intuitive flowchart format. The simulation pro- gram is built as
a series of process flowcharts that detail the events through which a class of entity will pass.
The use of entity attributes allows decision points to be incorporated into the flowchart,
providing alternative process routes for entity classes.
A popular method of control is the three-phase approach that combines the event- and
activity-based methods. The three phases are shown in Figure 1.4 and described as follows:

– The “A” phase involves advances the simulation clock to the next event time. The
simulation calendar is inspected and the clock jumps directly to the event with the time
closest to the current simulation clock time. The clock is held constant during the three
phases until the next “A” phase.

– The “B” phase involves execution of all activities whose future time is known (i.e., bound
events). The simulation takes all bound events that are due to occur at the current
simulation time from the calendar and executes them. The execution of bound events may
cause further events to occur. These are placed on the simulation calendar to be activated
at the appropriate time.

– The “C” phase involves execution of all activities whose future time depends on other events
(i.e., conditional events termed C-events). For each “C” phase, all conditional events are
checked to see if the conditions determining whether they can be executed are met. If the
conditions are met, the conditional event is executed. The execution of a C-event may cause
other C-event conditions to be met. For this reason the C-events are repeatedly scanned
until all C-event conditions are not met at this time point.

In general, bound events are events such as the end of a process when time can be predicted
by simply adding the current simulation time to the process duration. Conditional events
are occurrences that are dependent on resource availability whose future timing cannot be
predicted (e.g., a customer awaiting service at a bank). The three-phase approach simply
scans all conditional events after the bound events have been executed to check if the
simulation state allows the conditional event to take place. The operation of the three-phase
discrete-event method can be shown by studying the actions of the next event mechanism
on the simulation clock.
18
 Figure 1.5 illustrates the next-event time advance approach. Arrival times (A1, A2, .. .)
and service times (S1, S2, .. .) will normally be random variables taken from a suitable
distribution. The discrete-event system operates as follows. The simulation clock advances
to the first event at time 8. This is an arrival event (A1) where an entity arrives at the
resource. At this time the resource is available (“idle”) and so is immediately serviced for
16 time units (S1). During this period, the server status is set to “busy.” The simulation
calculates the service completion time (C1) of 24 units and inserts an event on the calendar
at that time. At time 20, a second entity arrives (A2). Because the server is currently in the
“busy” state, the entity waits at the server queue until the server becomes available. At each
future event, the status of the server is checked using a conditional (C) event. At time 24 the
first entity completes service (C1) and thus changes the server status from “busy” to “idle.”
Entity 2 will now leave the queue and commence service, changing the server status back
from “idle” to “busy.” The completion time is calculated as the current time + ser- vice
time (24+12 = 36) and a completion event is entered on the calendar at this time. At time
30, entity 3 arrives (A3). Again, the server is busy so the entity waits at the server queue. At
time 36, the second entity completes service (C2) and entity 3 can now leave the queue and
commence service. The simulation continues until a termination state is reached. The
time in the system for each entity can be calculated by the addition of the queuing time and
service time (Table 1.1).

19
Figure 1.4: The three-phase executive.

20
 6 6
4
20 24 30 36
8 12
0 8

S = Service time S1 S2

A = Arrival
C = Completion A1 A2 C1 A3 C2

Figure 1.5: Operation of the three-phase approach.

Table 1.1: Queue and Service Times for Entities.

Entity Queue Time Service Time System Time

This demonstrates how the next-event time mechanism increments the simulation clock to
the next (in time order) event on the calendar. At this point the system status is updated
and future event times are calculated. The time between each advance will vary depending
on the pattern of future events.

Using Simulation to Model Human Behavior

The modeling of people is becoming increasingly important in the design of business

processes. Thus to provide a realistic basis for decision support, people’s behavior will need
to be included in simulation models if they are to be effective tools. Many of the systems
that we would like to understand or improve involve human actors, either as customers of
the system or as people performing various roles within the system. Modeling passive,
predictable objects in a factory or a warehouse, however, is very different from trying to
model people. Modeling people can be challenging be- cause people exhibit many traits to
do with being conscious sentient beings with free will. Human beings can be awkward and
unpredictable in their behavior and they may not conform to our ideas about how they
should behave in a given situation. This presents a practical challenge to model builders,
i.e. when and how to represent human behavior in our simulation models. In some
situations, the role of human behavior in the model may be small and may be simplified or
even left out of the model. In other cases, human behavior may be central to the
understanding of the system under study and then it becomes important that the modeler
represents this in an appropriate way.

21
Simplify (Eliminate Human Behavior by Simplification)

This involves the simplification of the simulation model in order to eliminate any
requirement to codify human behavior. This strategy is relevant because a simulation
model is not a copy of reality and should only include those elements necessary to meet the
study objectives. This may make the incorporation of human behavior un- necessary. It may
also be the case that the simulation user can utilize their knowledge of human behavior in
conjunction with the model results to provide a suitable analysis. Actual mechanisms for
the simplification of reality in a simulation model can be classified into omission,
aggregation and substitution and will be considered under the topic of conceptual
modeling.

Externalize (Incorporate Human Behavior Outside of the Model)

This approach involves incorporating aspects of human behavior in the simulation study,
but externalizing them from the simulation model itself. For example, the “externalize”
approach to represent human decision making is to elicit the decision rules from the people
involved in the relevant decisions and so avoid the simplification inherent when codifying
complex behavior. Analytic techniques such as ma- chine learning and neural networks can
be interfaced with the simulation and be used to provide a suitable repository for human
behavior.

Flow (Model Humans as Flows)

At the highest level of abstraction inside the model, humans can be modeled as a group
which behaves like a flow in a pipe. In the case of the flow method of modeling human
behavior, the world view is termed continuous and the model abstraction is termed macro.
The type of simulation used for implementation of the flow method is usually the SD
technique. The flow approach models humans at the highest level of abstraction using
differential equations. The level of abstraction, however, means that this approach does not
possess the ability to carry information about each entity (person) through the system being
modeled and is not able to show queuing behavior of people derived from demand and
supply. Thus, the simulation of human behavior in customer-processing applications, for
example, may not be feasible using this approach.

Entity (Model Human as a Machine or Material)

This relates to a mesoscopic (meso) simulation in which elements are modeled as a number
of discrete particles whose behavior is governed by predefined rules. One way of modeling
human behavior in this way would mean that a human would be either a resource, such as
a unit of equipment that is either “busy” or “idle.” Alternatively modeling a human as an
entity would mean that they would under- take a number of predetermined steps, such
as the movement of material in a manufacturing plant. This approach can be related to
the process world view, which models the movement of entities through a series of process

22
steps. The entity approach models human behavior using the process world view to either
represent people by simulated machines (resources) and/or simulated materials (entities).
This allows the availability of staff to be monitored in the case of resources and the flow
characteristics of people, such as customers, to be monitored in the case of entities.

Task (Model Human Performance)

This method models the action of humans in response to a predefined sequence of tasks
and is often associated with the term human performance modeling. Human performance
modeling relates to the simulation of purposeful actions of a human as generated by well-
understood psychological phenomenon, rather than modeling in detail all aspects of human
behavior not driven by purpose. The task approach can be related to the process world view
and mesoscopic (meso) modeling abstraction level that models the movement of entities,
in this case people, through a series of process steps. The task approach is implemented
using rules governing the behavior of the simulation attributes of human behavior. These
attributes may relate to factors such as skill level, task attributes such as length of task and
organizational factors such as perceived value of the task to the organization. Two
assumptions of simulation models are seen as particular barriers to modeling knowledge
workers. The first is that all resources are assumed to belong to pools where any worker
within the pool has the ability to carry out the task. Secondly there is an assumption that once
a task is initiated it will be completed. In DES people can be represented as entities, rather
than resource pools, which enable work on a task to be segmented into sessions. At the end
of each session, work priorities are reassessed and work continues either on the same tasks
if priorities have not changed or on an alternative task. Thus, the task approach attempts to
model how humans act without the complexity of modeling the cognitive and other variables
that lead to that behavior.

Individual (Model Human Behavior)

This method involves modeling how humans actually behave based on individual attributes
such as perception and attention. The approach is associated with an ob- ject world view
where objects are not only self-contained units combining data and functions, but are also
able to interact with one another. The modeling approach can be termed microscopic
(micro) and utilizes either the discrete-event or ABS types. The approach can use cognitive
models for modeling human behavior at an individual level. This approach is implemented
by assigning numerical attributes, representing various psychological characteristics, to the
model entities (people). These characteristics could include patient anxiety, perceived
susceptibility, knowl- edge of disease, belief about disease prevention, health motivation
and educational level for a medical application for example. The individual approach
attempts to model the internal cognitive processes that lead to human behavior. A number

23
of architectures that model human cognition have been developed. However, the difficulty
of implementation of the results of studies on human behavior by behavioral and cognitive
researchers into a simulation remains a significant barrier. There is a debate about the
suitability of DES to model human behavior but a solution could be the use of DES software
to implement agent-based models.

Enabling a Simulation Capability in the Organization

The use of simulation is both a technical issue involving model development and analysis
and a process of the implementation of organizational change. This section discusses
technical issues such as the selection of simulation software and organizational issues
such as the selection of personnel and the acquisition of resources required to provide the
capability to undertake a simulation project. It is important that the full costs of
introducing simulation are considered, in- cluding user time and any necessary training
activities. The potential benefits of simulation must also be estimated. One of the reasons
simulation is not used more widely is the benefits from change, undertaken as a result of a
simulation study can be difficult to quantify.
However, simulation may not always be the appropriate tool. Also for providing a
positive cost/benefit analysis, it should be compared to alternative approaches for solving
the problem. Solutions such as spreadsheet analysis and the use of analyti- cal methods
may be faster and cheaper. It may be that the organization lacks the infrastructure to
provide the necessary data required by the simulation model. Finally some aspects of the
organization, such as human behavior and social inter- actions, may be too complex to
represent as a model.
The steps in introducing simulation in the organization are outlined as follows:
1. Select a simulation sponsor
2. Estimate the benefits of simulation
3. Estimate the costs of simulation
4. Select the simulation software

Simulation Application Areas

Simulation modeling is a flexible tool and is capable of analyzing most aspects of an

organization. The two main areas of operations are in the design and management of
processes. In terms of design, simulation has an obvious role in the testing of system designs
for systems that do not yet exist. Simulation can also be used for the design of existing
systems where it can be used to assess a number of design options without disruption to the
real system. One of the advantages of thoroughly testing processes at the design stage is that
errors found at this stage will generally be much more costly to rectify at later stages of
installation and operation. Simulation is often used to assess large capital investments such
24
as for equipment and plant, where it can re- duce the risk of implementation at a relatively
small cost. Simulation can also be used in the management of business processes. For
example a service operation may wish to ensure continued good customer service whilst
meeting increased demand. Making decisions around aspects such as staffing levels and job

priorities to meet in- creased demand requires the ability to predict changes in these areas
on performance. Simulation can be used to provide a predictive capability to help make
better decisions and ensure future service levels are maintained.

To ensure the maximum value is gained from using the technique it is necessary to
define the areas of the organization that are key to overall performance and select feasible
options for the technique in these areas. Some examples of simulation use are given below
with reference to decision areas that are applicable to simulation analysis and to the case
studies included later in this module.

Manufacturing Systems

In order to remain competitive manufacturing organizations must ensure their systems can
meet changing market needs in terms of product mix and capacity levels whilst achieving
efficient use of resources. Because of the complex nature of these systems with many
interdependent parts, simulation is used extensively to optimize performance. Design
decision areas in manufacturing include estimating required re- source capacity, layout
design, bottleneck analysis, machine setup time reduction, implementation of automation
and production lead time estimation. Management decision areas in manufacturing include
estimating batch size, determining parts sequencing, workforce scheduling and assessing
preventative maintenance policies.

Service Systems

The productivity of service sector systems has not increased at the rate of manufacturing
systems and as the service sector has increased the potential increase in productivity from
improving services has been recognized. The use of methodologies to streamline service
processes has many parallels in techniques used in manufacturing for many years.
Simulation is now being used to help analyze many service processes to improve customer
service and reduce cost. For example the emphasis on performance measures in
government services such as health care has led to the in- creased use of simulation to
analyze systems and provide measures of performance under different configurations.
Design decision areas include customer queuing time estimation, layout planning, service
capacity planning. Management decision areas include staff scheduling, customer service
priorities, emergency planning.

Supply Chain Management Systems

Supply chains systems are a network of activities that entities flow through from raw
25
materials supply to production, distribution and then retail to the customers. The supply
chain should aim to minimize costs whilst maintaining service levels. Supply chains will
incorporate transportation systems such as rail and airline serv- ices as well as internal
systems such as automated guided vehicles (AGVs) which can be analyzed using simulation.
Many simulation software packages have special facilities to model track-based and
conveyor type systems and simulation is ideally suited to analyze the complex interactions
and knock-on effects that can occur in these systems.

Design decision areas include supply chain structure, process rede- sign, supplier selection,
facilities and capacity planning, supply chain integration, the Bullwhip Effect, reverse
logistics, replenishment control policies, supply chain optimization, cost reduction, system
performance, inventory planning and management and customer service levels.

Information Systems

Simulation is used to predict the performance of the computerization of processes. This

analysis can include both the process performance and the technical performance of the
computer network itself, often using specialist network simulation software. Design
decision areas include the effects of automation by IS systems on customer service levels,
estimating IS capacity requirements to meet customer transaction volumes and designing
client-server systems.

26
Chapter 2
Simulation and Business Processes
This chapter is focused on the use of simulation to analyze processes within business
organizations. The process perspective is associated with the area of operations
management that considers the transformation of materials, information and customers
into goods and services (Figure 2.1). This transformation is undertaken by processes that
require the allocation of resources such as staff and facilities.
Business process management is a discipline that focuses on the use of business
processes as a way of improving organizational performance. Deriving and using process
performance measures is a key aspect of both operations management and business process
management. This section will outline a range of performance measures that can be used
to measure process performance, cover the use of tools for documenting the process design
and describe methods for process improvement.
The task of designing processes should be undertaken in a structured manner and the
steps involved can be described as follows:
1. Identify opportunities for process improvement.
2. Document the process design.
3. Redesign the process.

1 Identify Opportunities for Process Improvement

In order to identify where performance improvement should take place, it is necessary to

compare the performance against a performance standard. This standard can be internal
to the organization such as comparing against previous performance or against targets for
future performance. Internal targets are often based on a comparison between past
financial and sales performance and targets for future performance. The advantage of these
measures is that they are widely used and comparable across organizations and use data
that are readily available. However, they may be of limited value in identifying why
performance is above or below the target value. External targets include comparison to
competitor performance, best practice or “best-in-class” performance or market
requirements. External performance targets have the advantage of providing a comparison
of performance against competitors operating in similar competitive markets. This
approach is often termed benchmarking. There now follows a discussion of traditional
economic performance measures, the balanced scorecard, the five strategic performance
measures of operations management, process performance measures and activity-based
costing.

27
 Transforming resources Goods and services
Facilities

Staff

Transformed resources
Materials
Information
Customers

Figure 2.1: The process perspective of operations management.

Traditional Economic Performance Measures

Traditional performance measures have focused on indicators such as productivity.

Productivity divides the value of the output by the value of the input resources consumed:

Productivity = output/input

Productivity is used at both the organizational and national level as a comparative measure
of performance. From the equation it can be seen that productivity can be increased by
either increasing the output without a proportionate increase in the input or by decreasing
the input without a proportionate decrease in the output. Although productivity provides
an indication of the level of utilization of resources, it can be difficult to find appropriate
input and output parameters for the calculation, and the measure also fails to consider
performance from a wider viewpoint encompassing customer and other stakeholder needs.
Another performance measure is efficiency:

Efficiency = actual output/effective capacity

Efficiency is defined as a measure of the use of capacity remaining after the loss of output
due to planned factors such as maintenance and training. In other words, efficiency relates
to the use of a resource in terms of availability. However, a high level of efficiency does not
necessarily imply that resources are being used effectively in improving the overall
performance. Effectiveness can be defined as the ex- tent to which the output of a process
meets the requirements of its customers. This is more difficult to measure quantitatively
than productivity or efficiency.

28
In service operations, effectiveness is often measured by surveys of customer satisfaction.
However, customer satisfaction will be dependent on the perceptions and expectations of
individual customers. Other indicators of effectiveness could be sales volume and sales
margin figures for products and services.

The Balanced Scorecard

Another approach to performance measurement has been the use of the balanced
scorecard. The balanced scorecard approach is an attempt to incorporate the interests of a
broader range of stakeholders through performance measures across four perspectives of
financial, customer, business process, and learning/growth. The idea of the scorecard is to
provide managers with multiple perspectives of the goals that need to be met for
organizational success. Although designed for performance measurement at a strategic
level, its relevance to operations is that it provides a direction for the organization that
will impact on and be impacted by operations. The balanced scorecard also provides a way
of translating strategy into action. It does this by translating performance measures at a
strategic level into operational performance measures.

The case study at the end of this chapter shows how strategic targets on individual
performance at a UK Police Service were translated into measurements at an operational
level.

The Five Strategic Performance Measures of Operations Management

From the area of operations management comes the five strategic performance measures
of quality, speed, dependability, flexibility and cost. Two models that use these measures to
identify where performance should be improved are as follows.
The Hill methodology (Hill and Hill, 2012) is based on market requirements. The
concepts of “order winning” and “qualifying factors” are used to distinguish between those
factors that directly contribute to winning business and those that are necessary to qualify
for the customer’s consideration between a range of products/services. Its importance is that
while it may be necessary to raise performance on some qualifying factors to a certain level
in order to be considered by the customer, a further rise in the level of performance may
not achieve an increase in competitiveness. Instead competitiveness will depend on raising
the level of performance of different “order-winning” factors. Because these factors are
critical to winning customers, they are translated into the five performance measures
mentioned earlier.
The second model of Slack et al. (2019) uses a combination of market and competitive
factors and two dimensions—importance and performance—to help prioritize performance
measures. The relative importance of a competitive factor is assessed in terms of its
importance to internal or external customers using a nine- point scale. The degrees of order
winning, order qualifying and less important customer viewed competitive factors are
measured on this scale. The next step ranks the relative performance of a competitive factor
29
against competitor achievement. A nine-point performance scale (rating from consistently
better than the nearest competitor to consistently worse than most competitors) is used for
each performance measure. This indicates what customers find important in achieved
performance when compared with that of competitor performance.

Once key performance areas have been identified using the above-mentioned models,
the performance measures can provide both an indication of performance that can be
derived from customer requirements and can be used to align the physical resources and
management systems of the company’s operations. Thus, the pursuit of a subset of these
measures provide a way of connecting the strategic direction of the company with its
operational decisions. For example, if key metrics are identified as speed and cost, then
investments in resources such as customer- processing technology should improve these
aspects of performance.
An additional aspect regarding the five performance measures is that there is both an
external benefit in improving performance within a performance area and an internal
benefit in terms of reducing cost in that area. For example, improving quality performance
not only means higher customer satisfaction but also reduces internal cost through a
reduction in the need to rectify mistakes. This shows the potential of replacing strategies
that rely on immediate cost cutting, which might achieve short-term savings by lowering
the costs of inputs such as staff and equipment into the process, but risks lowering the
capability of the process to meet customer needs. Although simulation is usually associated
with operational measurement—it can actually be used to measure performance across all
five of the strategic performance measures of speed, cost, quality, dependability and
flexibility. Table 2.1 shows links to case studies that focus on the different performance
measures.

Process Performance Measures

Simulation is mostly associated with the measurement of processes at the

operational level and a number of these measures are found in the reports provided
by simulation software. These include:
Flow Time. The time taken to undertake the steps in a process. This can also be
referred to as throughput time. This can relate to the speed performance metric for
delivering goods and services to customers.
Queue Time. The time taken between being ready to be processed and the re- source
undertaking the processing to become available and processing to commence. This can be
related to speed for a manufacturing process in which the aim is to minimize the proportion
of flow time that is queue time. In a service operation we need to minimize queue time to
ensure customer service quality.

30
 Table 2.1: Business Process Performance Measures.

Resource Utilization. The time that a resource is in operation as a proportion of total

time. This can be related to the cost performance metric for staff and equipment.
Cost. Most simulation software allows cost to be estimated by defining usage cost rates
and idle cost rates in the model. Some resources may also incur a fixed cost with each entity
handled which may be defined as a cost per use. Simulation is an ideal tool to implement
the concept of Activity-Based Costing (ABC) which is covered in the next section.
Chapter 4 will provide further information on the performance measures re- ported by
simulation software.

Activity-Based Costing

Cost is traditionally calculated by estimating in terms of staff, facilities and materials which
are the resources that are required for the input and transformation processes in an
operation. However, the way these costs are allocated to products and services is often
arbitrary. For example, the actual costs of producing a product in a factory where hundreds
of other products are also being made are dependent on an accurate allocation of direct
costs (staff, equipment and material costs directly connected to the product) and indirect
costs (for example, overhead such as factory space, energy, administration and central
staffing costs). The aim of performance measurement is to identify where cost is being
incurred within an operation, so improvement efforts can be focused in the correct areas.
As an alternative to the usual overhead-based costing methods, activity-based costing
provides a way of allocating costs to manufacturing and service activities in order that a
company can determine how much it costs to make a certain product or deliver a service.
In activity-based costing (ABC) there are three main drivers of cost: the cost driver, the
resource driver and the activity driver.
– The cost driver relates to the amount of resources needed to perform an activity and can
be reduced by, for example, redesigning the process.

31
– The resource driver relates to the type of resources needed for an activity and can be
reduced, for example, by using different personnel, information technology or
equipment.
– The activity driver relates to the number of times the process occurs and can be reduced,
for example, by training to improve the reliability of a process.

The cost assignment view of ABC allocates costs to activities by identifying resource drivers,
which determine the cost of resources, and activity drivers, which deter- mine the use of
these resources. The process view of ABC provides information about the effort needed for
the activity (called the cost driver) and provides performance measures of the outcome of
the activities. The cost assignment view can be used to reduce the activity cost by either
reconfiguring the resources needed for an activity (resource driver) or reducing the amount
of resource required (activity driver). The process view provides a link between the inputs
needed to perform an activity (cost driver) and the outputs required by the internal or
external customer of that activity (performance measure). Thus, an investigation of a
combination of resource drivers, activity drivers and cost drivers for an activity can improve
performance of the process by identifying why cost has been incurred.

2 Document the Process Design

The identification of activities in a current process design is a data-collection exercise using

methods such as examination of current documentation, interviews and observation. In
order to provide a framework for the design and improvement of service processes, the
documentation techniques of process mapping and service blueprinting can be used.

Process Mapping

A document of the process can be created by the use of a process map, also called a flow
chart, which is a graphical representation of the elements that make up a process. This
is a useful way of understanding any business process and showing the interrelationships
between activities in a process. This can help in identifying and fixing problems with the
process, assisting the development of new processes and comparing the design of similar
processes. For larger projects, it may be necessary to represent a given process at several
levels of detail. Thus, a single activity may be shown as a series of sub-activities on a
separate diagram. Process maps are useful in a number of ways. For example, the procedure
of building a process map helps people define roles and see who does what. The process
map can also serve as the first step in using simulation as it identifies the processes and
decision points required to build the model. More details on the use of process maps are
given in Chapter 3.

32
Service Blueprinting

Let’s consider a case where a service consists of a number of sub-processes that are not
linked and the service “output” is a number of customer–employee interactions. In this
case, the process design may first focus on the design of the customer–employee
interactions and then identify external performance measures such as customer
satisfaction. To assist in the analysis of customer–employee inter- actions, the process maps
can be extended to show how a business interacts with customers. One system is a flow
chart (termed a service blueprint) that structures the process activities on either side of a
customer’s “line of visibility” (Figure 2.2). The activities above the line are visible to the
customer and those below the line are operations that the customer does not see. Activities
above the line of visibility are subdivided into two fields separated by the “line of
interaction”; this divides activities undertaken by the customer and the service provider. In
the case of below the line of visibility, a “line of internal interaction” separates the activities
of front-line personnel who carry out setting-up actions prior to service provision (not in
view of the customer) and support personnel who contribute materials or services required
for the provision of the service. Finally, the “line of implementation” separates sup- port
activities from management activities such as planning, controlling and decision making.
Figure 2.2 shows an example of a service blueprint for a restaurant.
The objective of the service blueprint is that it not only charts the service process flow
(from left to right) as does a process map but also shows the structure of the service
organization on the vertical axis, showing relationships between, for ex- ample, internal
customers, support staff and front-line providers. In particular, the diagram aims to
highlight the interactions between the customer and process where customer services can
be affected. The diagrams can also be used as a design tool to determine staffing levels, job
descriptions and selection of equipment and as a control tool to identify gaps in service
provision through the analysis of fail points. Fail points are potential service system
shortfalls between what the service delivers and what the targeted customers have been led
to expect.

33
3 Redesign the Process

There are many ways in which a process can be redesigned to meet particular objectives,
and so it is necessary to generate a range of innovative solutions for evaluation. Three
approaches that can be used to generate new ideas are as follows:

Generating new designs through brainstorming

This approach offers the greatest scope for radical improvements to the process design but
represents a risk in the implementation of a totally new approach. A deep understanding of
the process is required so that the design will be feasible.

Modifying existing designs

This approach is less risky than a blue-skies approach, but may mean that the opportunity
for a radical improvement in process design is missed.

Using an established “benchmark” design

This approach applies the idea of identifying the best-in-class performer for the particular
process in question and adopting that design. Disadvantages with this approach may be
that the process design of the best-in-class performer may not be available or the context of
the best-in-class performer may not match the context for the new design.
The process map or service blueprint provides an overall view of the current or expected
process design, and this should be used so that an overall view is taken when process design
options are generated. This helps to ensure that design solutions proposed in a specific area
do not have a detrimental effect in other areas of the process and thus affect the overall
process performance. The design of service processes in particular is a key factor in meeting
the needs of the customer. The process incorporates employees, customers and facilitating
goods in a dynamic manner that may be undertaken differently each time, according to the
demands of the individual customer.
Peppard and Rowland (1995) provide a number of areas for the potential rede- sign of
processes under the headings of eliminate, simplify, integrate and automate (ESIA) (Table
2.2).
It will be necessary to reduce the number of design alternatives generated and this can
be achieved by a rating scheme that scores each design solution against key performance
dimensions such as response time and cost of operation. The out- come of this analysis will
be a reduced number of design solutions, which can then be subjected to a more detailed
analysis using simulation.
The following case study shows the use of performance measures at a strategic level,
using the balanced scorecard to link them to operational process measures.

34
35
Chapter 3
Build the Conceptual Model

The main steps in undertaking a simulation study, considered in this section, are
shown in Figure 3.1.

Step 1: Build the Conceptual Model

The conceptual model provides a specification for the computer-based simulation model
that will be developed in step 2. It does this by providing an outline of the simplification of
the real-world system by a domain expert (a person who under- stands how the system
works) into a form that can be represented as a computer model. The conceptual model will
be based on an understanding of the problem objectives and the definition of the study
objectives. This will allow a definition of the model scope and level of detail and the
simplifications and assumptions that are made when forming a description of the real-
world system. A process map should provide a medium for obtaining information from a
variety of viewpoints regarding the system being organized. In order to prepare for the
simulation build (step 2), it is necessary to specify data collection activities in terms of the
source of the information and its form, such as documentation, observation and interview.
In addition, a specification of the type of statistical analysis used for modeling input data
should be made.

Step 2: Build the Simulation Model

This step involves implementing the conceptual model specification on a computer using a
selected simulation software package. Before the model can be used for analytics it must be
verified and validated. Verification involves ensuring that the computer model is a correct
representation of the conceptual model developed in step 1. Validation concerns ensuring that
the simplifications and assumptions made about the real-world system are acceptable in the
context of the simulation study objectives.

Step 3: Use Simulation for Descriptive, Predictive and Prescriptive Analytics

This step involves the use of the model to provide information for decision making by
studying the effect that changes in the model (termed scenarios) have on performance
measures defined in the model. For each scenario, the statistical analysis re- quired should
be defined and the use of a terminating or steady-state analysis should be stated. The results
of the simulation study should be presented in report form and include full model
documentation, study results and recommendations for further studies. An
implementation plan may also be specified.

36
 When combining simulation with analytic techniques to undertake data-driven
simulation and model-driven analytics, the methodology shown in Figure 3.1 is ex- tended
as follows (Figure 3.2).

Figure 3.2 shows the use of the big data analytics techniques of process mining, ma- chine
learning and data mining and visual analytics to facilitate stages of the simulation
methodology. At the “Build the Conceptual Model” stage process mining can be used to
generate a process map. Machine Learning and Data Mining can be used at this stage to
cluster data, and machine learning can be used to determine decision logic. At the “Build
the Simulation” stage, machine learning algorithms can be used to facilitate digital twins.

37
At the “Use Simulation for Descriptive, Predictive and Prescriptive Analytics” stage,
machine learning and data mining can be used to facilitate simulation experimentation, for
example, in optimization, and visual analytics is relevant in providing analysis of large-scale
simulation experiments through techniques such as clustering. The large data sets required
of these techniques are derived at the data collection phase of the “Build the Conceptual
Model” stage through sources such as sensors. Alternatively, the simulation can generate
large data sets, known as data farming.

Although the three main steps in Figure 3.1 and Figure 3.2 are shown as sequential it is
important to recognize that the simulation study will be an iterative process. Thus the model
building stage may identify areas in the conceptual modeling stage where additional data
collection and input modeling are required. Indeed it is normal practice to begin the model
building stage before the conceptual modeling stage is complete to minimize development time
whilst data collection activities continue. This iterative approach also applies to the final stage
when it may transpire that generating the metrics required involves additional work in the
conceptual modeling and model building stages. One approach to undertaking a simulation
study is to build a simple “prototype” version of the model in order to help define a detailed
specification. This prototype can then either be “thrown away” and the simulation study
process started afresh or the prototype can be developed and refined based on the new
specification.
The following activities for step 1 are now defined. These provide a specification for the
computer-based simulation model that will be developed in step 2 (Chapter 4):
– Understand the problem situation
– Define the study objectives
– Define the scope and level of detail
– Process mapping
– Data collection
– Modeling input data

The first two activities of understanding of the problem situation and defining the study
objectives provide the context for the development of the conceptual model. This contains
a definition of the scope and level of detail of the model and a specification for the
process map. In order to build the model, further work is re- quired in terms of a
specification for data collection and modeling input data.

Understand the Problem Situation

This step involves the modeler in understanding the context of the problem so that they are
able to identify what elements of the real system will be relevant to the model and also the
relationship between these elements. Modeling incorporates both information from data
collection on aspects such as process durations but also con- textual understanding of the
problem situation which is needed to conceptualize the system into a simplified
representation of a model. This conceptualization enables the model to project into the
future and be used for predictive and prescriptive analytics.

38
 Usually the problem situation will differ for each simulation project and will require
knowledge of the process being modeled, whether it is a just-in-time (JIT) system in a
manufacturing context or a customer call center in the service industry.

This will most likely require data collection through discussions with personnel in-
volved in the process and the use of existing documentation. It may be necessary to seek this
information from a number of people in order to provide a full picture of the problem. One
way of representing this knowledge is to produce a process map of the real system showing
the steps involved and the flow of information, materials and people through the system.
This can act as a communication device with the client so that an agreement on the outline
of the process can be agreed. This process map can form the basis of the conceptual model,
which will take into consideration those aspects of the system that require modeling in
terms of the study objectives and need for model assumptions and simplifications.
Finally, there may be aspects of the problem in which we are not able to gain an
understanding. If these aspects fall within the scope of our investigation we should codify
this lack of knowledge in the simulation specification as an assumption of the model. For
example, we might assume that workers involved in our process do not increase their
productivity through learning or training during the period of the simulation. At this point,
we need to obtain agreement with the client on these assumptions and our overall
understanding of the problem situation, which is called establishing the model credibility.
In general, simulation is suited to well-structured problems and tends to leave the
human aspect of systems aside. For example, the different values, beliefs and interests of
people involved in the problem lead to what are called different world views of the problem
situation. Also different model builders may well have a different definition of what the
real system is (i.e., exhibit bias). Thus, a modeler will produce a model that will differ from
another person modeling the same system based on their different knowledge and
interpretation of reality. For certain types of problems, the need to take into consideration
the human aspects of systems and provide a holistic view of issues has led to the use of
approaches such as Soft Systems Methodology and Systems Thinking.

Define the Study Objectives

Within a problem area, many different models could be built encompassing different
aspects of the problem at different levels of details.

Table 3.1: Performance Improvement Approaches.

Performance Improvement Approach Examples

Change process design Changes to the process map; routing, decision points
and layout by elimination, simplification and
automation.
Increase resource availability Increase in the amount of staff or equipment.
Changes to shift patterns. Staff training to increase
productivity.

39
Reduce variability Moving to a customer appointment system, using
preventative maintenance to reduce equipment
breakdown, adding products to the product mix to
smooth demand, training workers on processes,
automating processes.

Many projects will study a combination of the above, but it is important to study each area
in turn to establish potential subjects for investigation at the project proposal stage. As well
as determining a performance improvement approach the study requires a decision on the
performance measures to be used in the study.
The next step is to define more specifically the objectives of the study. For ex- ample, a
call center may wish to study staffing costs. The objective may be written as follows:

The simulation model will estimate the staff cost of processing a telephone call to the emergency
call center by call type.

From this the simulation analyst can derive the following requirements from the simulation
model specification. Important information on model detail (the process for responding to
an emergency call), input requirements (demand for each call type) and performance
measures required (staffing cost) can be implied from the objective stated. The project
proposal should contain a number of objectives at such a level of detail, which allows the
simulation analyst to derive an outline model specification upon which an estimate in terms
of time and cost can be prepared. For instance, the previous statement requires clarification
on the exact nature of the call process. What elements of this process are required for
modeling, the initial call handling or should the coordination of the response to the call also
be modeled? These factors may re- quire simply additional simulation runs, or major
changes to the model design—so an iterative process of redefining model objectives between
the analyst and the user is required at this stage.

Define the Scope and Level of Detail

This stage can be considered the most difficult in the design of the simulation model and
can be summarized as decisions regarding what should be left out (or included in the
model). These decisions are made by defining both the scope and level of detail of the
model. The model scope is the definition of the boundary be- tween what is to be included
in the model and what is considered external to the specification. The scope will need to
encompass aspects of the system that connect the defined input factors (e.g., customer
demand) with our output factors (e.g., customer service time).
One way of considering the boundaries or scope of what should be incorporated into the
simulation model is to consider the context level of the process being simulated. The
decision on what elements at each context level to include should be made by considering
the objectives of the study. Four main context levels can be defined moving from a narrow
to a wide context.

40
Entity (instance)

At the entity level the consideration is for a single instance of an entity moving through the
process and here the attributes of the entity are considered.

For example customers (entities) of different customer types (attribute) may take different
process flow paths through the process model.

Process (flow)

At the process level we consider that it is unlikely that a single entity is processed in
isolation but many entities may be moving through the process at the same time. This means
that how the entity is processed will be determined by its interaction with other entities
such as the need to share resources. If insufficient resources are available then entity
queuing may occur or the entity process flow may be altered.

Organizational (people)

Here we extend the boundary of the model to include aspects that are not normally
covered in simulation studies. At this context level we take account of attributes of
people’s behavior in the organization and how they will impact the process. This
includes people’s work priorities, skill levels, teamwork and motivation. Thus at the
organizational level consideration is made that people work on a variety of tasks in
different ways and how they organize their work and their productivity will affect
how they are processed.

External (environment)

Here we consider external factors, for an organizational process this may include
customers, suppliers, finance, government actions, weather conditions or any other factors.
At the external level there are a number of factors such as supplier actions that may affect
the process performance.

Once the model scope has been defined, the next stage is to specify the level of detail of the
modeling of components within the scope of our model. In determining the level of detail,
the aim is to minimize model complexity while providing a model capable of achieving the
project objectives. Model complexity can be considered in terms of the number of
components within the model and the amount of connections between them. One reason we
wish to minimize complexity is that verification and validation becomes more difficult as the
complexity of the model increases. However, if we leave too much out of the model, there is a
41
greater conceptual leap between the conclusions drawn from the model and interpretation in
relation to the system. Thus, a model that meets the objectives of the study with the least
amount of complexity required would be the ideal solution.
Strategies for reducing model complexity by reducing the level of detail of how we
model system components are termed simplifications. Simplifications are distinguished
from model assumptions in that simplifications are aspects we choose to simplify to reduce
model complexity, whereas assumptions are made to eliminate the need to model aspects
of the system we do not fully understand. A key reason for making model simplifications is
that the results of the model are easier to interpret since the structure of the model is easier
to understand.
Actual mechanisms for the simplification of reality in a simulation model can be
classified into omission, aggregation and replacement:
– Omission: omitting aspects of the system from the model. For example, the
model may omit staff breaks for meals and presume staff are continuously
available. Often machine-based processes are modeled without reference to the
human operator they employ.
– Aggregation: merging individual processes into a single process. For example,
processes or the work of whole departments may be aggregated if their internal
working is not the focus of the simulation study. The model may only consider
a customer service process as a whole rather than modeling individual transac-
tions within that process.

– Replacement: replacing complex processes. For example, human processes are

often substituted by a “resource” element in a simulation model. This sim-
plifies a human process removing many of the complicating factors of human
behavior.

From a modeling perspective, these strategies are useful in reducing model complexity and may
be necessary to produce a model within the time and cost constraints of the development.
However, there is also a consideration with regard to the perspective of the client of the model.
Clients tend to expect every aspect of a process included in the model, and simplification can
result in a model that may be too far removed from reality for the client to have confidence in
model results. This issue of model credibility requires an interaction between the client and
model builder to explain the purpose of the model in terms of assisting decision making rather
than an attempt to replicate reality. An important aspect of this is for assumptions and
simplifications to be stated explicitly in the simulation report, so that the client is aware of
them.
In general, this stage of the simulation study requires the skill and experience of the
modeler to decide in the level of model detail required. Practical considerations such as
data availability will also need to be taken into account, but the main decisions should be
based around the objectives of the simulation study. As a guide it is usually the case that
greater detail is required for models that are for predictive and prescriptive analysis than
for descriptive purposes.

42
 Finally, it should be remembered that the model will normally be used in conjunction with
the model user (client) in order to achieve understanding and make predictions. If this is the case
then the level of detail in the model should take into consideration that some aspects of the
system understanding can be derived from the user’s knowledge and the model can be seen as
a tool to support the user’s decision making.

Process Mapping

A useful method for representing the conceptual model that can be used as a basis for the
model build stage and provide a communication tool between model developer and client
is the use of a process map or a process flow diagram.

A process map should be formulated in line with the scope and level of detail defined within
the project specification mentioned in this chapter. An essential component of this activity
is to construct a diagrammatic representation of the process in order to pro- vide a basis for
understanding between the simulation developer and process owner. Two diagramming
methods used in discrete-event simulation are activity cycle diagrams and process maps.
Activity cycle diagrams can be used to represent any form of simulation system. Process
maps are most suited to represent a process interaction view that follows the life cycle of an
entity (e.g., customer, product) through a system comprising a number of activities with
queuing at each process (e.g., waiting for service, equipment). Most simulation applications
are of this type, and the clear form of the process map makes it the most suitable method in
these instances.

Activity Cycle Diagrams

Activity cycle diagrams can be used to construct a conceptual model of a simulation, which
uses the event, activity or process orientation. The diagram aims to show the life cycles of
the components in the system. Each component is shown in ei- ther of two states: the dead
state is represented by a circle and the active state is represented by a rectangle (Figure
3.4). Each component can be shown moving through a number of dead and active states in
a sequence that must form a loop. The dead state relates to a conditional (“C”) event where
the component is waiting for something to happen such as the commencement of a service,
for example. The active state relates to a bound (“B”) event or a service process, for
example. The duration of the active state is thus known in advance while the duration
of the dead state can not be known, because it is dependent on the behavior of the whole
system.

43
Figure 3.5: Arrest activity cycle diagram.

Process Maps

The construction of a process flow diagram is a useful way of understanding and

documenting any business process and showing the interrelationships between activities in
a process. Figure 3.6 shows the representations that are used in a simple process flow
diagram

44
Process

Decision

Start/end

Flow (direction)

Figure 3.6: Symbols used for a process flow diagram.

For larger projects, it may be necessary to represent a given process at several levels of
details. Thus, a single activity may be shown as a series of sub activities on a separate
diagram. Figure 3.7 shows a process map for a simplified arrest process.

Figure 3.7: Process map of arrest process.

No

45
Process Mining

The analytics software of process mining can be used to generate a process map design from
raw data that can then be used to build a simulation model. For example, calls to an
emergency call service can be logged on arrival and through the process they flow through.
Process mining offers the promise of fast construction of representations of complex
processes incorporating activities that may not be captured by traditional manual
development of the simulation process map. However, process mining does not generally
generate a usable process map directly from the event logs but uses a variety of analytic
techniques such as inductive mining for abstraction, dealing with issues such as noisy and
incomplete data. These approaches abstract events in an automated way that may not
capture the required detail to meet the simulation project objectives. Thus, process maps
derived using process mining should be cross-checked and validated prior to their use for
developing simulation models. See Chapter 17 for the use of process mining software in a
simulation study. Figure 3.8 shows a process map generated by the process mining software.

Figure 3.8: Process map generated by process mining software (www.minit.io).

Data Collection

The data collection phase may be a daunting and time-consuming task depending on the
complexity of the model and the availability of the data required. A well-defined model
specification is vital in providing a realistic assessment of data needs. One of the issues with data
collection is the reliance of the simulation analyst on company personnel to collect or provide
access to the required data in a timely manner. This will re- quire a clear specification of the
46
data requirements and effective communication with the data providers and if required
company management to ensure prioritization is given to this task.
A number of factors will impact on how the data collection process is under- taken
including the time and cost constraints within which the project must be con- ducted.
Compromises will have to be made on the scale of the data collection activity and so it is
important to focus effort on areas where accuracy is important for simulation results and
to make clear assumptions and simplifications made when reporting those results. If it has
not been possible to collect detailed data in certain areas of the process, it is not sensible to
then model in detail that area. Thus, there is a close relationship between simulation
objectives, model detail and data collection needs. If the impact of the level of data
collection on results is not clear, then it is possible to use sensitivity analysis to ascertain
how much model results are affected by the data accuracy. It may be then necessary to either
under- take further data collection or quote results over a wide range.
Two main problems associated with data are that little useful data is available (when
modeling a system that does not yet exist, for example) or that the data is not in the correct
format.
If no data exist you are reliant on estimates from vendors or other parties, rather than
samples of actual performance, so this needs to be emphasized during the presentation of any
results. An example of data in the wrong format is a customer service time calculated from
entering the service queue to completion of service. This data could not be used to approximate
the customer service time in the simulation model as you require the service time only.

The queuing time will be generated by the model as a consequence of the arrival rate and
service time parameters. In this case, the client may assume that your data requirements have
been met and will specify the time and cost of the simulation project around that. Thus it is
important to establish as soon as possible the actual format of the data and its suitability for
your needs to avoid misunderstandings later. Be sure to distinguish between input data which
is what should be collected and output data which is dependent on the input data values. For
example, customer arrival times would usually be input data while customer queue time is
output data, dependent on input values such as customer arrival rate. However, although we
would not enter the data collected on queue times into our model, we could compare these times
to the model results to validate the model.
If the required data is not available in a suitable format the analyst must either collect
the data or find a way of working around the problem.
As with other stages of a simulation project, data collection is an iterative process with
further data collected as the project progresses. For instance, statistical tests during the
modeling of input data or experimentation phases of development may suggest a need to
collect further data in order to improve the accuracy of results. Also the validation process
may expose inaccuracies in the model which re- quire further data collection activities.
Thus, it should be expected that data collection activities will be ongoing throughout the

47
project as the model is refined.

In order to amass the data required, it is necessary to use a variety of data sources shown in
Table 3.2.

Table 3.2: Sources of Data.

Data Source Example

Historical records Diagrams, schematics, schedules

Observations Time studies, walkthroughs
Interviews Discussion of process steps
Process owner and vendor estimates Process time estimates for equipment
Sensors Timings of events from RFID sensors

Modeling Input Data

Modeling input data concerns representing the uncertainty we find in dynamic systems that
show variability and interdependence. Modeling input data will be re- quired for the
following modeling areas and use the following modeling methods.

Modeling Areas

There are two main modeling areas that require input modeling. First, process logic data is
required to represent the processes and enable construction of the process map. This
provides us with an outline of what is being performed and defines the design of the process.

However, to construct the simulation model we need additional information on the system
regarding the time taken to undertake the processes (process durations), the resources and
the availability of those resources required to undertake the processes and the rate of arrival
of demand on the processes.
Data Required for the Process Definition

Process Routing
This is all possible routes through the process or the process flow. The process flow can be
of people, materials or information. Data collection may be from observation or discussion
with people familiar with the process design. Process mining can be used to derive the
process routing.

Decision Points
The logic defining each decision point within the process routing. This is needed to
represent branching and merging of the process flow. Decisions can be modeled by
48
conditional (if.. . then x, else y) rules. Conditional decisions occur when certain de- fined
conditions are satisfied and thus are unpredictable and cannot be known in advance. It may
be difficult to codify conditional decisions and so they may be rep- resented by a probability.
This represents substituting a cause-and-effect relation- ship by a distribution based on
historical behavior. For a probabilistic decision a sample value is drawn from a
continuous, uniform distribution that generates a value between 0 and 1. This means any
value is equally likely to occur. For a decision of (0.1,x; 0.5,y; else z) a value generated of
0.76 would lead to a flow following the route defined by the z option.
Data collection for probability values may be by sampling methods while conditional
decision rules may be determined by data collection from interviews with personnel
involved in decision making. Decision points may also be modeled as other mechanisms
such as triggers that activate on an event (e.g., reorder of inventory when levels drop below
a defined point).
Additional Data Required for the System Simulation Model Process Duration
These are durations for all relevant processes, for example, the customer service time at a
bank. The process duration does not include the queuing time for the process. Data
collection may be by observation or from event logs derived from sensor readings. Processes
that do not require a resource are modeled as a delay.

Resource
For processes that do require a resource, the type of resource and the number of units of
resource required to operate the process are defined.

Resource Availability
Resource availability schedules for all relevant resources such as people and equipment.
Depending on the level of detail this may need to include shift patterns and breakdown
events.

Demand Pattern
A schedule of demand “drives” the model. The demand can be in the form of materials,
information flows or customers. Demand can be determined through observation for some
service systems and analysis of production schedules in manufacturing. The demand pattern
or arrival rate is normally represented in the simulation as the “time between arrivals” or
interarrival rate. The interarrival time can be calculated by simply recording arrival times and
then for each arrival time subtracting the time of the previous arrival.

Process Layout
This covers process routing of material, information and customers and the layout of the
location in which this movement takes place. This information can be gath- ered from
observation, documentation or the use of process maps associated with Enterprise
Resource Planning (ERP) and workflow systems. The simulation back- ground display and
49
animation can be developed with the use of simulation soft- ware draw facilities and the
import of graphics files.

Other Elements
Note other fixed or random variables that may need modeling including quantities
or levels such as batch sizes, worker absences, the group size of customer arrivals
and characteristics of components in the model such as staff skill levels, component
sizes and customer types.

Modeling Methods
For the modeling areas previously defined we will normally wish to model the random
variation in these areas. Random variables are not undefinable or even unpredictable but
vary statistically and are thus probabilistically predictable. This behavior is usually defined
by the use of theoretical or empirical distributions. However for each modeling area and
depending on the level of detail we require, there are a number of alternative modeling
method strategies available. We may also be con- strained in our choice of modeling method
by the availability and time required for data collection and thus a number of options are
now presented.

These options include analytical methods and the use of raw data that is sampled
(bootstrapping) or used directly (trace).
1. Estimation
2. Theoretical distribution
3. Empirical distribution
4. Mathematical equation
5. Cognitive architectures
6. Analytics
7. Bootstrapping
8. Trace/Event Logs

50
Table 3.3: Modeling Input Data Methods.

Disadvantages Advantages Comments

Estimation Lack of accuracy. May be the only option. Often used in the
development phase
before further data
collection is
undertaken.

Theoretical No available theoretical Can “smooth” data to the Best choice if

distribution distributions may fit data. underlying distribution. a reasonable data fit
Generates values outside Generates values outside of can be made.
of the data range which data sampled. For example,
may not be appropriate. rare events at the tail of the
distribution which may be
missing from a small data
sample collected.
Compact method of
representing data values.
Distribution is easy to scale
for sensitivity analysis.

Disadvantages Advantages Comments

Empirical Can not usually generate Provides distribution when no An option if no

distribution values outside the range theoretical distribution theoretical
of data (therefore may provides an adequate fit to distribution can be fit.
miss “extreme” values). data.
Difficult to scale for
sensitivity analysis.

Equation May require extensive Shows relationships that are Can be useful for
data collection. a consequence of simulated expressing
time such as learning effects. relationships between
variables.

Cognitive Difficult to validate Attempts to model factors Limited use due to

cognitive models. leading to human behavior. demanding data
collection and
validation needs.

Analytics Using some analytic Can employ complex logic Need for modeler to
techniques, the logic processes. have analytic skillset.
implemented may not be
inspected.

Bootstrapping Distributions are derived Avoids issue of poorly fitted Likely to require large
only from values held in distributions. data set to ensure
the data set. coverage of full range
of data values.

Trace/Event Only uses values held in Replicates reality to check for Not suitable for
Logs the data set conformance prediction.

51
Chapter 4
Build the Simulation

The model-building process involves using computer software to translate the conceptual
model into a computer simulation model that can be “run” to generate model results.
In order to undertake the exercises in this chapter, we require the installation of the
Arena simulation software system. This is available as a free download with full
functionality but with limited model size at:
https://www.arenasimulation.com/simulation-software-download

The Simio Personal Edition that permits full modeling capability but supports saving and
experimentation on only small models that can be downloaded from:
www.simio.com/evaluate.php
The use of the Arena and Simio software systems will be demonstrated using a simple case
study of a bank clerk system. Note that the models built in this chapter incorporate
probability distributions and so the results you get may differ slightly to those in the text
due to random variation. This is to be expected and you can see that the results from the
same model run on different simulation software will also vary. This section provides only
an introduction to the model-building features of Arena and Simio.

The Bank Clerk Simulation

A bank clerk services two types of customer. The time between customer arrivals is
exponentially distributed with a mean of 15 minutes for customer type 1 and 10 minutes for
customer type 2. The processing time is uniformly distributed between 2 and 6 minutes for
type 1 customers and between 4 and 10 minutes for type 2 customers. Performance statistics
are required on the average time a customer is in the system, which includes both queue
time and service time.

Two versions of the bank clerk systems will be simulated. A single queue system is the
one where both types of customers form a queue at a service location staffed by two
members of staff. An alternative dual queue system is then modeled. This incorporates a
decision rule that directs customers to one of the two service location depending on which
location’s queue holds the minimum number of customers. Statistical techniques are used
to determine if there is a statistically significant difference between the performance of the
two systems.

The process map for the single queue bank clerk simulation is shown in Figure 4.1. The two
types of customers arrive from outside the system and are processed by one of the two bank
clerks. They then leave the system once the process has been completed

52
Figure 4.1: Process map of single queue bank clerk simulation.

Building the Bank Clerk Model in Arena

When the Arena system is run, the screen shown in Figure 4.2 should be displayed.
The project bar (to the left of the screen display) contains the modules from which you
construct the simulation by dragging them onto the model window. Data can be entered into
each module by either double clicking each module to obtain a dialog box or by using the
spreadsheet data module window.

Figure 4.2: Arena BASIC model window.

53
 The four main elements in an Arena simulation are entities, processes, resources and
decisions.

Entities

➢ Entities flow through a network of modules that describe their logical

behavior.
➢ Entities can represent people, materials or information flows in the
system being modeled.
➢ Entities must be created to get them into the model and are disposed
of when they leave.
➢ Entities are distinguished by their attributes.
➢ Attributes could refer to the characteristics of the people, parts or
any other in- formation regarding the entity that can be expressed in
numerical form.
➢ Attributes are useful as their value stays with the entity as it moves
through the model and their value can be used at any time by the
model.
➢ Values that are not associated with an individual entity are defined
as variables.
➢ Attributes and variables can be held in an array.

Modeling Entities

Not all entities or entity types in the real system will usually need to be modeled in the
simulation. We will often model product types as different entity types if they have different
process routes for example, but some product types may be aggregated if they are processed
in a similar fashion. Customers may also be grouped into different types if they require
directing to resources based on conditional decisions. Customers arriving in groups that are
processed as a group may be treated as a single entity within the model. If the customers
are processed individually they may be treated as individuals with a small interarrival time
to simplify modeling. Alternatively the model software can generate group arrivals by
specifying both the interarrival time and batch size of the arrival.

Processes

➢ Processes represent an activity that takes place over a period of

time.
➢ When an entity arrives at a process, it will try to obtain (seize) the
resource necessary to undertake that process.
➢ If no resource is required for the activity, then the process is modeled
as a delay.

54
 ➢ If a resource is required for the activity, then the process is
modeled by a seize–delay–release resource procedure.
➢ If a resource is not available, the entity waits in a queue.
➢ When the resource is available the length of time the entity uses the
resource is the delay.
➢ The entity releases the resource when processing is complete

Modeling Processes

Real processes are often merged or grouped into a single modeled process if that provides
the detail necessary. Processes that do not consume a resource (such as storage time when
the storage resource is not modeled) are represented by a delay. Processes that do consume
a resource must request the resources required from a resource pool. Multiple resources
may be needed to undertake a process (such as two members of the staff for an interview).
The process can not commence until suitable resources are available. Different processes
can be assigned priorities over other processes for the use of resources.

Resources
➢ Resources can be named.
➢ Have a capacity (number of identical units of this resource).
➢ Can have a schedule (how many available when).
➢ Resources can be animated as part of the simulation display.

Modeling Resources

Resources that enable processes can be modeled with either a capacity that varies with time (a
schedule) or a fixed capacity. A fixed capacity resource can simply represent a fixed number
of staff members who are available to serve any process in the model that requires them. A
schedule will alter staff availability in time buckets (for example for 15 minutes) to represent a
staff rotation. Resource availability can also be affected by planned maintenance and equipment
failures both of which can be modeled by changing the capacity of a resource at the appropriate
time. Consumable resources such as energy use may be modeled using variables. Some types of
resources are modeled in Arena using transporters for material handling equipment and
conveyors.

Decisions

➢ Decisions control the flow of individual entities.

➢ Decisions can use either chance or conditional rules to direct flow.
➢ Chance rules are represented as a percentage.
➢ Conditional rules are represented as an if–then–else rule.

55
Modeling Decisions

If conditional rules are understood, then decisions can be modeled in this way. Conditional
rules may take any form and be quite complex; for example the rules around scheduling
parts in a factory may incorporate many variables. Some of the most used conditional rules
include checking the smallest queue for customer ser- vice operations, checking resources
in turn, and choosing the first resource that is available in a manufacturing option or simply
choosing a flow path at random. If conditional rules are not used, then a probabilistic
approach is taken based on historical data. Decisions are also required when entities are
held in queues. In this case when allocating entities in a queue to a process, various decision
rules can be employed, such as the order in which the entity joined the queue; for example
the FIFO (first in, first out) rule. In addition an entity attribute value can be set and the
decision can be made based on that value. For example, customers assigned as priority
customers can jump the queue.

Building the Bank Clerk Model in Simio

When the Simio version 11 system is run, the screen display shown in Figure 4.3 should be
displayed.
The ribbon across the top of the screen has a number of tabs that collect together
commands into logical groups. Tabs are included for run commands and view options.
Below the ribbon there is a further series of tabs called the Project Model tabs that are used
to select options regarding the active model. If the Facility Tab is selected as in Figure 4.3,
then to the left of the screen the object library is displayed. By default the screen will display
the Standard Library and at the bottom of the screen the Project Library. The Properties
Window is on the right- hand side of the screen and shows the characteristics of any object
or item currently selected. The main area of screen (with grid) is named the Navigation
Window. Objects are placed from an Object Library on to the Navigation Window and are
de- fined/edited from the Navigation Window in the Properties Window. To move around the
Navigation Window, hold down the left mouse button and move the mouse. To zoom in and
out of the Navigation Window, hold down the right mouse button and move the mouse.

Before we begin building the simulation to set the time units used in the soft- ware,
click on the Run tab and select the Units Settings option. Select Minutes for Time Units.
Make sure the Facility Tab is selected in the Project Model tabs. To begin the single queue
bank clerk simulation drag two source objects, a server object and a sink object on to the
Navigation Window. Also draw a model entity object on to the Navigation Screen

56
Figure 4.3: Simio simulation screen display.

The next step is to connect the objects on the Navigation Screen. To do this, click on the
Connector Object in the Standard Library. Then click on the Output Node of Source1
(shown as a blue diamond on the screen) and move the mouse to the input node (grey
diamond) for the Server1. A left mouse click will define the connection. Repeat the
operation to connect Source2 to Server1 and Server1 to the Sink.
Click on the Source1 object and enter Random.Exponential(15) for the Interarrival time in
the Source1 Object Properties Window Make sure the Units entry immediately below the
Interarrival Time entry is set to minutes.
We can now set the process time attribute that will be used by the server to set the
process time. In Simio in order to define a numeric or string value that we can vary during
the simulation, we define a State variable. When a State is added to the main model object,
it can be accessed by all the objects within the main model simi- lar to a global variable in
Arena. When a State is added to another object, such as the Model Entity object it is part of
that object’s definition similar to an attribute variable in Arena. Thus in order to set the
process time variable at the Source1 and Source2 objects, we need to do the following.

In the properties Window in the Navigation section, select the ModelEntity object. Click
on the Definitions Tab and Select the States option. Click on the Real option in the Ribbon
menu at the top of the screen. In the Properties Window enter Process Time for name.
Now having a State variable associated with the ModelEntity object (in effect an entity
attribute), we can assign it a value. Click on the Model object in the Properties Window and
then select the Facilitys tab and Select the Source1 object in the Navigation screen. Select
the State Assignments option in the property Window and Select the “Before Exiting”
option and click on the dotted button. A menu will ap- pear. Click on Add and enter
ModelEntity.ProcessTime for the State Variable Name and Random.Uniform(2,6) for the
New Value. Click on close. Now click on Source2 object and repeat, assigning a value of

57
Random.Uniform(4,10).Move to the Server1 in the Navigation Window. Enter
ModelEntity.ProcessTime for the Processing Time entry in the Properties Window. Make
sure the Units entry is set to minutes. Set the Initial Capacity entry in the Properties Window
to 2.
We can now run the model. Click on the Run Tab in the Facility Ribbon and enter 480
minutes for the simulation end time. Set the speed factor to 100. Click in the Run button to
the left of the tab. The simulation should now run through 480 minutes of simulated time.
When the simulation run has finished, click on the Reports tab in the Project Model ribbon
and the results will be displayed. The results show that the average time in the system for
the customer is 6.55 mi- nutes with a minimum time of 2.28 minutes and a maximum time
of 11.04 minutes. The results should be similar (but not the same) as the results for the Arena
version of the model.
For the dual queue bank clerk model rather than using the single queue bank clerk
model, it is more convenient to create a new model for the dual queue version. Select File
and New to create a new model. Drag two Source objects, two Server ob- jects, a sink object
and a ModelEntity object to the Navigation Window. Now con- nect the objects, but unlike
the single queue model that used connectors, use the path objects to do this. This allows us
to direct the entities leaving the Source ob- jects along different paths to the Server objects
(i.e., allows implementation of the smallest queue rule). Source1 should be connected to
both Server1 and Server2 as should Source2. Both Servers should be connected to the Sink.
Enter the interarrival rates as in the single queue model (Exponential (10) for Server1
and Exponential (15) for Server 2). Define processtime as a State Variable (in ModelEntity)
and set the before Exiting State Assignment for processtime for Source1 and Source2. The
value is Random.Uniform(2,6) for Source1 and Random. Uniform(4,10) for Source2.
We can now direct customers to the server with the smallest queue size for our dual
queue model. We need to set up a list of destinations from which we can choose our
destination. To do this select the Definitions tab and select the List option. Click on the Node
option in the ribbon at the top of the screen. Give the node the name Servers in the
Properties Window. Click on the Node grid and select from the pull- down menu the
Input@Server1 option. Click on the Node grid again and select the Input@Server2 option.
If further destinations are possible, they can be easily added here.
Click on the Facility Tab to see the model. Move to the Server1 in the Navigation Window.
Enter ModelEntity.ProcessTime for the Processing Time entry in the Properties Window.
Make sure the Units entry is set to minutes. The Initial Capacity entry in the Properties
Window should be 1. Click on the output node for Server1 (named Output@Source1). In the
properties Window, select Entity Destination and select the Select from List option. Select
Node List Name and select the Servers option that should appear on the pulldown menu if
it has been defined previ- ously. For Selection Goal, select Smallest Value. For Selection
expression enter Candidate.Node.AssociatedStationLoad. Note the default value is
Candidate. Node.AssociatedStationOverload so that will need changing.

58
 Repeat the above steps for Server2 to set the ProcessTime value and for the out- put node
for Server 2 (Output@Source2). On the Run tab, set the ending time to 480 minutes and
make sure the time units in the unit settings tab are set to minutes. Set the Speed Factor to
100. Run the model and the customers should be directed to the server with the smallest
queue. At the end of the run select the Results tab for the results. The results should be
similar to those shown for the ARENA model in Figure 4.3.

Figure 4.3: Dual queue bank clerk simulation results.

Figure 4.4: 3D view of bank clerk simulation.

59
Simio provides facilities that allow 3D representations of a model to be developed quickly.
For the bank clerk simulation with the Facilitys tab selected and the model visible, select the
View Tab on the top ribbon and select the 3D option. The model will be redrawn in a 3D view
(Figure 4.4). Hold down the right mouse button and move up and down to zoom and move
left and right to rotate the image in the screen. Hold down the left button and move to move
around the model landscape.
We can make changes to the image so that it more closely represents the system we are
modeling. To replace the entity image from green triangles to people, select the Symbols
Tab on the top ribbon and select the Apply Symbols option. A pull- down menu provides a
selection of images to choose from. Select the people option in the Type selection box and
the people images will be displayed. Scroll down and pick one of the images from the
Library/People/Animated/Female category. Now click on Server1, and select an image
from the same category for Server1 and from the Library/People/Animated/Male category
for Server2. You can change the image display when the Servers are activated (move from
idle to busy) by selecting the server image and selecting the Active Symbol option. Select
the Processing option from the list and then apply a different symbol using the Apply
Symbol option. When the simulation is run, the symbol image will change when the server
is activated (Figure 4.5).

Figure 4.5: 3D animation of bank clerk simulation.

Verification and Validation

Before experimental analysis of the simulation model can begin, it is necessary to ensure that
the model constructed provides a valid representation of the system we are studying. This
process consists of verification and validation of the simulation model. Verification means
ensuring that the computer model built using the simulation software is a correct
representation of the conceptual model of the system under investigation.

60
Validation concerns ensuring that the assumptions and simplifications made in the conceptual
model about the real-world system are acceptable in the context of the simulation study
objectives. Both topics will now be discussed in more detail.

Verification

Verification is analogous to the practice of “debugging” a computer program in order to check

that the program does what has been planned. Thus, many of the following techniques will
be familiar to programmers of general-purpose computer languages.

Model Design
The task of verification is likely to become greater with an increase in model size. This is
because a large complex program is both more likely to contain errors and these errors are
less likely to be found. Due to this behavior most practitioners ad- vise on an approach of
building a small simple model, ensuring that this works correctly, and then gradually adding
embellishments over time. This approach is intended to help limit the area of search for
errors at any one time. It is also important to ensure that unnecessary complexity is not
incorporated in the model design. The design should incorporate only enough detail to
ensure the study objectives and not attempt to be an exact replica of the real-life system.

Structured Walkthrough
This enables the modeler to incorporate the perspective of someone outside the im- mediate
task of model construction. The walkthrough procedure involves talking through the program
code with another individual or team. The process may bring fresh insight from others, but the
act of explaining the coding can also help the per- son who has developed the code discover
their own errors. In discrete event simulation code is executed non sequentially and different
coding blocks are executed simultaneously. This means that the walkthrough may best be
conducted by following the “life-history” of an entity through the simulation coding, rather
than a sequential examination of coding blocks.

Test Runs
Test runs of a simulation model can be made during program development to check model
behavior. This is a useful way of checking model behavior as a defective model will usually
report results (e.g., machine utilization, customer wait times) that do not conform to
expectations, either based on the real system performance or common-sense deductions. It
may be necessary to add performance measures to the model (e.g., costs) for verification
purposed, even though they may not be re- quired for reporting purposes. An approach is
to use historical (fixed) data, so model behavior can be isolated from behavior caused by the
use of random variates in the model. It is also important to test model behavior under a
number of scenarios, particularly boundary conditions that are likely to uncover erratic
behavior. Boundary conditions could include minimum and maximum arrival rates, mini-
mum and maximum service times and minimum and maximum rate of infrequent events

61
(e.g., machine breakdowns).

Trace Analysis
Due to the nature of discrete-event simulation, it may be difficult to locate the source of a
coding error. Most simulation packages incorporate an entity trace facility that is useful in
providing a detailed record of the life history of a particular entity. The trace facility can
show the events occurring for a particular entity or all events occurring during a particular
time frame. The trace analysis facility can pro- duce a large amount of output, so it is most
often used for detailed verification.

Animation Inspection
The animation facilities of simulation software packages provide a powerful tool in aiding
understanding of model behavior. The animation enables the model developer to see many of
the model components and their behavior simultaneously. A “rough-cut” animated drawing
should be sufficient at the testing stage for verification purposes. To aid understanding model
components can be animated, which may not appear in the final layout presented to a client.
The usefulness of the animation technique will be maximized if the animation software facilities
permit reliable and quick production of the animation effects. An animation inspection may be
conducted by following the “life-history” of an entity through the animation displayed at a slow
speed.

Documentation
It is important to document all elements in the simulation to aid verification by other
personnel or at a later date. Any general-purpose or simulation coding should have
comments attached to each line of code. Each object within a model produced on a
simulation system requires comments regarding its purpose and details of parameters and
other elements.

Verification with Arena

Arena provides a number of facilities for model verification. Animation facilities provide
immediate feedback to the user from which the model logic can be checked by following the
path of entities through the animated display. Errors can also be found by inspecting the
measure of performance (e.g., machine utilization, queue length) within the system and
comparing them to the estimated values made by using of rough-cut calculations. In testing
both the model logic and performance, particular attention should be paid to any behavior
at variance from the real system (taking into consideration simplifications and assumptions
made) or expected behavior of a system that does not yet exist. As was stated earlier,
verification is easiest with small simple models. With larger and more complex models, it
may be necessary to temporarily simplify model behavior for verification purposes. This
could be achieved by replacing distributions for arrival and process times with constant
values or only routing one entity type through the model to check the entity path. If the

62
model is not behaving in an appropriate manner, it is necessary to investigate the model
design. This can be achieved through inspection of the model code and by analyzing the
event calendar using the debugging facilities.

Validation
A verified model is a model that operates as intended by the modeler. However, this does
not necessarily mean that it is a satisfactory representation of the real system for the
purposes of the study. Validation is about ensuring that model behavior is close enough to
the real-world system for the purposes of the simulation study.
Unlike verification, the question of validation is one of judgement. Ideally the model
should provide enough accuracy for the measures required while limiting the amount of
efforts required to achieve this. Validation is difficult to do well and insufficient time may
be spent on this activity as there may be pressures to begin experimentation and analysis.
There are a number of issues to consider such as the sensitivity of model results to initial
run conditions and ensuring a correct under- standing of real-system behavior to which the
model can be compared. If the model fails the validation stage, then the conceptual
modeling stage should be revisited and model revisions made. As a last resort and if no
solution can be found to achieve model validation, then the modeling project should be
cancelled.
For most systems of any complexity validation can be achieved in a number of ways
and a key skill of the simulation developer is finding the most efficient way of achieving this
goal. Pegden et al. (1995) outlines three aspects of validation:
➢ Conceptual Validity—Does the model adequately represent the real-world
system?
➢ Operational Validity—Are the model generated behavioral data
characteristic of the real-world system behavioral data?
➢ Believability—Does the simulation model’s ultimate user have
confidence in the model’s results?

These three aspects reflect that model validation can be seen as proving that the assumptions
and structure of the model have led to a truthful representation of the real system but also
validation can be seen as being concerned with the accuracy of the predictions of the model.
In this latter approach the underlying assumptions and structure of the model are irrelevant
and this approach is often taken when validating data-driven analytics models. In reality most
simulation model developers will take both of these perspectives into account when conducting
model validation. The model results can be tested for reasonableness (operational validity) and
the model assumptions and structure can be examined in terms of their representation of the
real system (conceptual validity). Both the model developer and the model user (decision
maker) will need to be convinced that these tests have been passed (believability).

63
Chapter 5
Use Simulation for Descriptive, Predictive and
Prescriptive Analytics

This chapter covers the use of the simulation model to generate descriptive, predictive and
prescriptive analytics. As the final step in a simulation study and in order for the analysis
to have served its purpose it is usually necessary to implement actions deriving from the
simulation results.
When measuring performance using simulation one aspect deals with the variability
inherent in the simulation model that leads it to generate different values for its performance
metrics (such as queuing times) each time the simulation is executed over a time period. To
address this issue, the main method employed in simulation is to per- form multiple replications
of the simulation and construct confidence intervals of the metrics of interest. Additional
statistical tests, such as t-tests, may also be undertaken in which scenarios are compared. It
should be noted that these procedures deal with the statistical uncertainty associated with the
randomness and variability in the model. This does not necessarily mean that the model is
representative of the real system. Our domain knowledge may also be uncertain. This kind of
uncertainty is difficult to quantify as it may be about events that can not be considered random
and repeatable. This issue relates to the topic of model validity and emphasizes that it is
unlikely that a model will be wholly valid and certainly will not be a true representative of a
system that does not yet exist. However, the model results should not be viewed in isolation but
interpreted by people with the required knowledge of the model—its assumptions,
simplifications and scope and knowledge of the real system that is being modeled.
There are two types of simulation system that need to be defined, each requiring
different methods of data analysis.
Terminating systems run between predefined states or times where the end state
matches the initial state of the simulation. Most service organizations tend to be
terminating systems that close at the end of each day with no in-process inventory (i.e.,
people waiting for service) and thus return to the “empty” or “idle” state they had at the
start of that day. For example, a simulation of a retail outlet from opening to closing time.
Non-terminating systems do not reach predefined states or times. Most manufacturing
organizations are non-terminating with inventory in the system that is awaiting a process.
Thus even if the facility shuts down temporarily, it will start again in a different state to
the previous start state (i.e., the inventory levels define different starting conditions). Before
a non-terminating system is analyzed, the bias introduced by the nonrepresentative
starting conditions should be eliminated to obtain what are termed steady-state conditions
from which a representative statistical analysis can be undertaken.

64
Statistical Analysis of Performance Measures for a Simulation

This section will provide statistical tools to analyze either terminating or non- terminating
systems. For statistical analysis the output measure of a simulation model is a random
variable and so we need to conduct multiple runs (replications) of the model to provide us
with a sample of its value. When a number of replications have been undertaken, the sample
mean can be calculated by averaging the measure of interest (e.g., time in queue) over the
number of replications made. Each replication will use of a different set of random numbers
and so this procedure is called the method of independent replications. We will conduct our
statistical analysis of performance using confidence interval analysis for terminating
systems before considering analysis of non-terminating systems.

Statistical Analysis of Terminating Systems

To assess the precision of our results, we can compute a confidence interval or

range around the sample mean that will include, to a certain level of confidence, the
true mean value of the variable we are measuring. Thus, confidence intervals
provide a point estimate of the expected average (average over infinite number of
replications) and an idea of how precise this estimate is. Thus, a 95% confidence
interval means that if the experiment was repeated many times, then 95% of those
confidence intervals would contain the true value of the parameter. The formula for
the confidence interval involving the t-distribution is as follows:
t*s
μ=x± pffiffiffi
n
where
μ = half-width of the confidence interval
x = sample mean (from the replications)
t = t-distribution value tdf,α/2 where df = degrees of freedom (n-1) and α =
significance level
s = standard deviation (from the replications)
n = number of replications

Both the confidence interval analysis and the t-tests presented later for comparison analysis
assume the data measured is normally distributed. This assumption is usually acceptable if
measuring an average value for each replication as the output variable is made from many
measurements and the central limit theorem applies. However the central limit theorem
applies for a large sample size, and the definition of what constitutes a large sample depends
partly on how close the actual distribution of the output variable is to the normal
distribution. A histogram can be used to observe how close the actual distribution is to the
normal distribution curve.

65
Establishing Confidence Intervals at a Given Half-Width

There are instances when we wish to specify the half-width of the confidence intervals, thus
providing a predetermined measure of the statistical precision of our results. In general,
more replications conducted will lead to a narrower confidence interval, but the actual
number to achieve a particular level of precision cannot be calculated in advance as it is a
function of the sample size itself. An approximate value can be gained however by
obtaining an initial sample standard deviation (say from five to ten replications) and
solving for n as follows:
2
n = ðz1 − α=2 * so=hwÞ

where

z1 – α/2 = value from normal distribution table at a significance α

so = sample standard deviation
hw = required half-width of confidence interval

Note that t1 – α/2,df has been approximated to z1 – α/2 because df depends on n which is the
target value. When n replications have been completed the confidence intervals are
calculated in the normal way. If the half-width is greater than required then the current
value of s can be used to recalculate the number of replications, n, required.

Statistical Analysis for Non-terminating Systems

The previous section considered statistical analysis for terminating systems. This section
provides details of techniques for analyzing steady-state systems in which the start
conditions of the model are not returned to. These techniques involve more com- plex analysis
than for a terminating system and so consideration should be given to treating the model as
a terminating system if at all possible.
A non-terminating system generally goes through an initial transient phase and then
enters a steady-state phase when its condition is independent of the simulation starting
conditions.

This behavior could relate to a manufacturing system starting from an empty (no-inventory)
state and then after a period of time moving to a stabilized behavior pattern. A simulation
analysis will be directed toward measuring performance during the steady-state phase and
avoiding measurements during the initial transient phase. The following methods of
achieving this are discussed.

66
Setting Starting Conditions

This approach involves specifying start conditions for the simulation that will pro- vide a
quick transition to steady-state conditions. Most simulations are started in an empty state
for convenience, but by using knowledge of steady-state conditions (e.g., stock levels), it
is possible to reduce the initial bias phase substantially.

The disadvantage with this approach is the effort in initializing simulation variables, of
which there may be many, and when a suitable initial value may not be known. Also, it is
unlikely that the initial transient phase will be eliminated entirely. For these reasons, the
warmup period method is often used.

Using a Warmup Period

Instead of manually entering starting conditions, this approach uses the model to initialize
the system and thus provide starting conditions automatically. This approach discards all
measurements collected on a performance variable before a preset time in order to
ensure that no data is collected during the initial phase. The point at which data is discarded
must be set late enough to ensure that the simulation has entered the steady-state phase,
but not so late that insufficient data points can be collected for a reasonably precise
statistical analysis. A popular method of choosing the discard point is to visually inspect the
simulation output behavior of the variable over time. It is important to ensure that the
model is inspected over a time period, which allows infrequent events (e.g., machine break-
down) to occur a reasonable number of times.
In order to determine the behavior of the system over time and in particular to identify
steady-state behavior, a performance measure must be chosen. A value such as work-in-
progress (WIP) provides a useful measure of overall system behavior. In a manufacturing
setting, this could relate to the amount of material within the system at any given time. In
a service setting (as is the case with the bank clerk model) the WIP measure represents the
number of customers in the system.

Using an Extended Run Length

This approach consists of running the simulation for an extended run length, thus reducing
the bias introduced on output variables in the initial transient phase. This represents a
simple solution to the problem and can be applied in combination with the other
approaches.

Batch Means Analysis

To avoid repeatedly discarding data during the initial transient phase for each run, an
alternative approach allows all data to be collected during one long run. The batch means
67
method consists of making one very long run of the simulation and collecting data at
intervals during the run. Each interval between data collection is termed a batch. Each
batch is treated as a separate run of the simulation for analysis. The batch means method
is suited to systems that have very long warmup periods and so avoidance of multiple
replications is desirable. However, with the increase in computing power available, this
advantage has diminished with run lengths needing to be extremely long in order to slow
down analysis considerably. The batch means method also requires the use of statistical
analysis methods, which are beyond the scope of this module.

Comparing the Performance of Alternative Simulation Models

When comparing between alternative configurations of a simulation model, we need to test

whether differences in output measures are statistically significant or if differences could be
within the bounds of random variation. Alternative configurations that require this analysis
includes:
➢ changing input parameters (e.g., changing arrival rate)
➢ changing system rules (e.g., changing priority at a decision point)
➢ changing system configuration (comparing manual vs automated system)

Note that these changes imply that just a change to a single parameter value will create a new
simulation model in terms of our statistical analysis of performance measures. Irrespective of
the scale of the differences between alternative configurations, there is a need to undertake
statistical tests. The tests will be considered for comparing be- tween two alternatives, but can
also be used to compare for more than two alternatives. The following assumptions are made
when undertaking the tests:
➢ The data collected within a given alternative are independent observations of a random
variable. This can be obtained by each replication using a different set of random
numbers (method of independent replications).
➢ The data collected between alternatives are independent observations of a random
variable. This can be obtained by using a separate number stream for each alternative.
Note, however, that certain tests use the ability to use common random numbers for
each simulation run in their analysis (see paired t-test using common random
numbers).

The statistical tests are required so we can judge when comparing model results whether the
means (over the replications) do actually suggest that a statistical change in performance has
taken place. For instance, if the mean for the single queue scenario is 4.8 minutes and the mean
for the dual queue scenario is 5.1 minutes, do these measures represent a statistically significant
difference in performance or are they within the boundary of random variation? We now show
three ways of doing this analysis.

1. Comparing Confidence Intervals

The first is to compare the confidence intervals for the two means. If they do not overlap, we
68
can say that the difference between the means is statistically significant. However, this is an
informal test. In order to quantify how sure we are of this difference, we may need to
conduct a hypothesis test, which will provide us with a decision based on a chosen level of
confidence (e.g., 95% or 99%).

2. Hypothesis Testing
A hypothesis test makes an assumption or a hypothesis (termed the null hypothesis, H0) and
tries to disprove it. Acceptance of the null hypothesis implies that there is insufficient
evidence to reject it (it does not prove that it is true). Rejection of the null hypothesis,
however, means that the alternative hypothesis (H1) is accepted. The null hypothesis is
tested using a test statistic (based on an appropriate sampling distribution) at a particular
significance level α, which relates to the area called the critical region in the tail of the
distribution being used. If the test statistic (which we calculate) lies in the critical region, the
result is un- likely to have occurred by chance and so the null hypothesis would be rejected.
The boundaries of the critical region, called the critical values, depend on whether the test
is two-tailed (we have no reason to believe that a rejection of the null hypothesis implies that
the test statistic is either greater or less than some assumed value) or one-tailed (we have
reason to believe that a rejection of the null hypothesis implies that the test statistic is either
greater or less than some assumed value). We must also consider the fact that the decision
to reject or not reject the null hypothesis is based on a probability. Thus at a 5% significance
level, there is a 5% chance that H0 will be rejected when it is in fact true. In statistical
terminology, this is called a type I error. The converse of this is accepting the null hypothesis
when it is in fact false, called a type II error. Usually α values of 0.05 (5%) or 0.01 (1%) are
used. An alternative to testing at a particular significance level is to calculate the p-value,
which is the lowest level of significance at which the observed value of the test statistic is
significant. Thus, a p-value of 0.045 (indicating a type I error occurring 45 times out of
1000) would show that the null hypothesis would be rejected at 0.05, but only by a small
amount. Generally, the paired t-test is used to compare alternatives of real systems.

3. Analysis of Variance
One-way analysis of variance (ANOVA) is used to compare the means of several alternative
systems. Several replications are performed for each alternative and the test attempts to
determine whether the variation in output performance is due to differences between the
alternatives or due to inherent randomness within the alternatives themselves. This is
undertaken by comparing the ratio of the two variations with a test statistic. The test makes
the assumptions of independent data both within and between the data sets, observations
from each alternative are drawn from a normal distribution and the normal distributions
have the same variance. The first assumption implies the collection of data using indepen-
dent runs or the batch means technique, but precludes the use of variance reduction
techniques (e.g. common random numbers). The second assumption implies that each
output measure is the mean of a large number of observations. This assumption is usually
valid but can be tested with the chi-square or Kolmogorov- Smirnov test if required.

69
The third assumption may require an increase in replication run-length to decrease the
variances of mean performance. The F-test can be used to test this assumption if required.
The test finds if a significant difference between means is apparent but does not indicate if
all the means are different, or if the difference is between particular means. To identify
where the differences occur tests such as Tukeys HSD test may be used. Alternatively
confidence intervals between each combination can provide an indication.

Further Statistical Analysis

Factorial Designs

In our simulation experimentation we may wish to analyze the effect of making changes to
a variety of input variables and/or making a number of changes to a particular input variable
and to see the effect on a number of output variables. In experimental design terminology
experiments are conducted by changing a factor (input variable) in the model that has a
level (value). A particular combination of levels is called a treatment. The output variable of
interest is called the response. The standard way to investigate changes in a level of a factor
would be to make a simulation run for each level and record the response. There are two
issues with this approach:
➢ To investigate a number of factors requires a relatively high number of simulation runs.
➢ The effects of interaction (i.e., whether the effect of one factor depends on the levels of
the other).

Thus, the objective of factorial design is to determine the impact of factors on a response.

Two-Level Full-Factorial Design

A two-level full-factorial design is when each factor is set at just two (high and low) levels.
The levels could be different values of an input variable or different configurations of a
model (e.g., different scheduling rules for a queuing system). No rules are provided as to
what the levels of the factors should be, but levels need to be set that are different enough
to show how the factor affects response, but within normal operating conditions for that
factor. For a full-factorial design, each possible combination of factor levels is tested for
response. This means for k factors there are 2k combinations of low and high factor levels.
These possible combinations are shown in an array called the design matrix.

The advantage of following the factorial design approach over varying a single fac- tor at a
time is that the effect of interaction effects can be assessed. The ability to assess the impact
of one factor change when the level of a second factor changes is important in finding the
best system performance because the effect of two factors changing may not be the same as
the addition of the effect of each factor change in isolation.

70
Optimization

Optimization involves choosing the “best” results rather than reporting the result for a
particular scenario or comparing results between scenarios. Optimization thus provides a
prescriptive analytics capability by recommending a choice of action. Many simulation
software packages incorporate optimization software such as OptQuest (www.opttek.com)
that use the machine-learning technique of genetic algorithms.

Visual Analytics

For experimentation with systems that are complex we may have many input parameters
that we can set to many different values. In order to analyze these systems, we need to
execute simulation runs for different parameter combinations. Often in this situation, it is
difficult to interpret the results of these experimental designs simply using statistics.
Visualization or visual analytics simultaneously presents simulation results for a number
of parameter combinations to the user. Thus, the technique uses the visual display of
information and the domain knowledge of the user to observe the results of parameter
combinations that may be selected for further investigation. Alternatively visual analytics
may use analytics techniques such as a clustering algorithm that classifies the simulation
experiments as a way of synthesizing large amounts of data and helping to reveal patterns
and relationships between variables that might otherwise be hidden or difficult to find.
Clustering methods can be used to explore simulation output data by treating each object
in a cluster as a single simulation run allocated on selected parameter results. For example,
in a two-dimensional analysis, the variables cycle time and throughput time may be used.
Once the clusters are mapped out visually, analysts can investigate which input settings led
to the corresponding systems performance measures that define this cluster. A limitation
of visual analytics is that the identification of relationships using visual inspection may
be less precise and more open to interpretation than traditional approaches to simulation
output analysis. Furthermore, visual analytics may require the training in and use of new
analytics software and analysis methods by simulation practitioners.

71
Reference
Greasley, A. (1998) “An Example of a Discrete-Event Simulation on a Spreadsheet”, SIMULATION,
70(3), pp. 148–166.
Hill, A. and Hill, T. (2012) Operations Management, Third edition, Palgrave McMillan.
Kim, D. H. (1992) “Systems Archetypes: Diagnosing Systemic Issues and Designing High-Leverage
Interventions,” Toolbox Reprint Series: Systems Archetypes, Pegasus Communications,
pp. 3–26.
Maani, K. E. and Cavana, R.Y. (2007) Systems Thinking, System Dynamics: Managing Change and
Complexity, Second Edition, Pearson Education, New Zealand.
O'Neil, C. (2016) Weapons of Math Destruction: How Big Data Increases Inequality and Threatens
Democracy, Penguin Random House
Pegden, C.D., Shannon, R.E., Sadowski, R.P. (1995) Introduction to Simulation using
SIMAN, Second Edition, Mc-Graw-Hill.
Peppard, J. and Rowland, P. (1995) The Essence of Business Process Re-engineering, Prentice Hall.
Robinson, S. (2014) Simulation: The Practice of Model Development and Use, Second Edition,
Palgrave Macmillan.
Senge, P. M. (2006) The Fifth Discipline: The Art and Practice of The Learning Organization, Second
Edition Random House, London.
Slack, N, and Brandon-Jones, A. (2019) Operations Management, Ninth Edition, Pearson Education
Ltd., Harlow.

72
73
74