Powersim Studio 2003

User’s Guide
Foreword

Making business decisions

Today, business simulations are used in making business decisions. Faced with a decision on where
to build natural gas storage containers, and how big to build these containers, one oil company used
Powersim to build a simulation of the wells, the pipelines, and the markets. Once the simulation
was built, decision-makers at the oil company could try different storage container scenarios. The
consequences of the different scenarios could be understood in a risk-free environment before any
investments in construction of storage containers were made.
Business simulations are also used in explaining business decisions to others in the organizations.
For example, a major telecommunications company saw many of their competitors making risky
forays into interactive television, adventures that were garnering much publicity. It was not enough
to just focus on their core business; they also had to explain to the employees why investing in
interactive TV and similar ventures were bad business decisions. This telecommunications
company built an interactive simulation of how they perceived their business. Over 1200
employees used this simulation in a series of management training workshops, where employees
had an opportunity to run the simulated business and make investment decisions. The employees
gained new appreciation for the corporate strategy, and a better understanding of how the thought
leaders in their organization saw the dynamics of the next few years unfolding.
Both of these simulation success stories were achieved using Powersim software. In each case a
team of modelers and thought leaders from the business collaborated to build a custom model of
the business problem. The models contained enough of the important details and complexities of
the real world situation to be useful and believable.

Business simulations more common

The use of simulation in business is not yet an everyday occurrence, but it is spreading. In 1990,
both of these examples would have been quite unusual. Although simulation technology was well
understood by 1990, and the cost of the necessary software tools had become modest, very few
business decisions were supported by simulating the consequences before taking action. It simply
was not part of standard business practice. Now things are changing, and the use of business
simulation is becoming more common.
At Powersim Software AS, we are committed to supporting the emerging practice of business
simulation. We appreciate that the process of business decision-making is different from that of
decision-making in public policy. Our tools are focused on modeling business problems, not on
public policy or academic issues.

-5-
From models to simulators
Why this sudden change in business practices? Business models are not substantially easier to build
now than they were at the before. Nor has there been a watershed event that has made the business
community suddenly more receptive to simulation. From the viewpoint of us in the business
modeling community, building simulation models for years, the change is welcome, but a little
mysterious. However, from the viewpoint of our business customers, there has been a technology
revolution of sorts. Once a model is built, it is easy to build a high-quality user interface around it,
and turn it into a business simulator. Anyone can run the business simulator, trying different
decision scenarios and understanding the resultant behaviors.

The joy of simulators

Who has the time and interest to play with a business simulator? Everyone seems to make time.
Some commercial simulators – like SimCity and The Sims – are so much fun to play that they are
sold as entertainment. While few business executives might admit to playing games, they will
gladly play with a custom business simulator, crafted to model an aspect of their current
environment. One very serious senior executive remarked that playing a custom simulator of his
telecommunications company was "a joyful experience." Another senior executive at the same
organization spent 10 hours over two evenings playing the simulation.
Crafting a joyful experience can mean the difference between 4 people understanding your model
and its implications--and 1000 people understanding it. And joy is impossible if casual users must
struggle with arcane controls, obscure commands, or cryptic numerical displays.

The value of simulators

Joy alone is not enough in a business context: there must also be value. What is the value of
business managers playing a business simulator? Do the people playing the simulation make better
decisions? Do they learn more about the world in which they must succeed?
For many years, we in the business modeling community have approached the problem of creating
value by teaching business people a new way of thinking. We have taught them the basics of
building models, and when that was too difficult, we taught "systems thinking," how to see their
world the way a modeler might see it. Systems thinking is undoubtedly valuable for anyone who
chooses to undergo the education--but it has certain shortcomings as a discipline that can be put to
use within a business. Some people will never learn how to think systemically; and in fact, business
people expect something less abstract and more tangible for their investments in time and money.
Provide them a simulated experience – with their new situation, in their industry, in their context,
with their assets and constraints – and you’ve empowered them to solve existing problems, and
avoid potential ones.
Our goal at Powersim Software AS is to support you in "publishing" custom models of your
business, making them available to everyone who might need to know or care. And there are a lot
of people who might care: existing employees, new recruits, suppliers, distributors, customers, and
investors.

Learning from simulators

By playing a simulator, by trying different combinations of decisions, a business manager can
understand a simulation in a kind of kinesthetic way. Just as you don’t need to understand the
physics of skiing to learn how to ski, in the same way it is possible to play a simulation model and
learn the dynamics of a business situation – without understanding the details of the underlying

-6-
mechanics. Through simulation, a businessperson can thus acquire experience not only without
"field" work, but also without necessarily understanding details of the model behind it.
Powersim Studio supports two approaches to building business simulators: building a simulator
within Powersim itself, and building a simulator in a general-purpose, interface-building language
and using Powersim as a simulation engine through the available Powersim Studio Software
Development Kit (SDK).
Within Powersim Studio, a modeler can use a host of interface objects. You can present the state of
the simulation by a gauge, by a graph of an array, and by many other means. You may also
program events that will occur when certain situations occur in your simulation.
As more and more organizations build business simulators to understand their world, and
communicate it to others within their organization and outside, your job will become more exciting
and more important. We at Powersim are committed to supporting you in your efforts to build
business simulators.
Good luck with your purchase of Powersim Studio 2003!

The Powersim Software Development Team

-7-
Table of Contents

Foreword............................................................................................................................ 5

Table of Contents .............................................................................................................. 9

Getting Started ................................................................................................................ 25

What’s new in Powersim Studio 2003? ......................................................................... 28

Introduction to System Dynamics.................................................................................. 32

The tools and rules of System Dynamics ............................................................ 35
Building the models ............................................................................................ 41
Building blocks in Powersim Studio................................................................... 44
Creating a model ................................................................................................. 47

Importing models from Powersim Constructor 2.51 ................................................... 48

About variable conversion .................................................................................. 48
About diagram conversion .................................................................................. 51
About simulation conversion .............................................................................. 52
Convert the ASSIGN function ............................................................................ 53
Convert constrained expressions......................................................................... 56
Convert the SHIFT functions .............................................................................. 57
Convert the LIMIT functions .............................................................................. 61

Working with help in Powersim Studio ........................................................................ 63

Topic layout ........................................................................................................ 63
The help window................................................................................................. 64

-9-
Getting Started in Powersim Studio
If you are new to Powersim Studio, the following steps will help you get started creating simulation
models. If you want more detailed instructions on how to create models and simulations, please refer
to the tutorials available in the Learning Lab.
Although the list below gives a linear view of the modelling process, you will soon experience that it
is rather an iterative process where you revisit previous stages many times before you have finished
the model. The steps below are only meant as an overview of the steps required.

1. Create a new simulation project

Powersim Studio is always opened with a new project (unless you started Studio by
opening an existing simulation project, that is). The project contains one component by
default, and you can add as many components that you need. All components and
views are available in the Project Window that you can see on the left of the
application. Each component can contain many simulations, with various simulation
settings. Each component has one simulation by default.

2. Create units of measurement

In the real world, most quantities are expressed using units of measurement. This is
also the case in Studio, where all variables will usually have a unit. To be able to
define the variables, you will therefore have to create the units first. You can always go
back and create more units as you find necessary!

Tip! It is easiest to create and use global units, which are available to all the
components in the project.

3. Create variables, links, and flows

You create variables and link them together using links and flows in Constructor
diagrams. The diagrams offer a good visualization of your model, and it is easy to see
the feedback loops that make up the system.

Tip! You can create as many diagrams you want per component. All variables
do not have to be part of each diagram. You can see all your variables in the
Details Window or the Equations View.

- 25 -
4. Define variables and flows

Once you have created the variables, it's time to define them. This is done in the
Definitions property page, available when you double-click the variable (or hit
ALT+ENTER). The variable definition expresses how the variable relates to other
variables in the model. Flows are defined by the flow rate that controls them, and they
are added to the level according to the level's flow definition.

Tip! You will probably often create variables and define them at the same time,
performing this and the previous step simultaneously.

Note! You must always make sure that (continuous) flow rates have a "rate"
unit, that is, it has to have a unit compatible with the level's unit divided by time.

5. Set up the simulation

Before you can run your simulation, you must specify the simulation settings to be
used by the component. The simulation setup is essential to make your simulation
behave the way that you want to. The time horizon of a simulation will vary with the
system your are studying. If you are studying a human population, as an example, your
time horizon will probably be about a century or two, with a resolution of a year or
two. If you are studying the life cycle of a product, however, you are more likely to
require a time horizon of a few years, with weeks or months as the time step.

6. Create data in- and output objects

It is not very useful to run a simulation unless you can view the results of your
simulation while it runs. You can use auto reports and controls to display the
simulation results. You have time graphs, charts, tables, slider bars, and gauges at your
disposal for displaying simulation results.
In many cases you also need to provide input for your constants while the simulation
runs. This is also achieved by the use of controls. You can create buttons or sliders as
tools for data input.

7. Add navigation capabilities and documentation

When your simulation behaves the way you want to, it's time for the final touch. Add
navigation to your simulation by the use of hyperlinks and bookmarks, and add
documentation to your model diagrams using freeforms and frames with text. If you
need to further document your simulation, you can create your own help system in
HTML that can be started from the project.

8. Create and save reference data

It is often useful to be able to compare a simulation run to data from other simulation
runs. This is possible in Studio by using reference parameters and reference data. You
can also save a simulation run and activate it again later.

- 26 -
 9. Set up the Presentation Mode

Finally, you can use the Presentation Mode to present the simulator interface you have
created to the user. The Presentation Mode removes all the development-related tools
and buttons, and displays only the simulator itself.

You can use the Presentation Mode as a "simulation browser," which makes your simulation easily
available for your end-users.

- 27 -
What's New in Powersim Studio 2003?
Powersim Studio 2003 contains a range of new features and functionality that makes your simulations
easier to create and more powerful to analyse. The most important features are described below. You
can also read a brief listing of the new features and functionality.

Risk Assessment
Studio 2003 now includes the Risk Assessment task — one of the four tasks earlier available in
Powersim Solver. Factors that are external to your own organization, such as the inflation rate, or
other values that are difficult to determine, represent risk factors if they seriously affect the results
from your simulation. Through the Risk Assessment task in Studio, you can now investigate how
changes in these assumptions affect your results. The result of such a simulation is the likelihood of
achieving a certain result. You can also use Risk Assessment to find variables that are leverage points
for improving performance.

The Risk Assessment task let you analyze the sensitivty of your model.

Hierarchical Models
With the introduction of model hierarchy, Powersim Studio 2003 allows you to divide your
simulation model into smaller submodels that hide away unnecessary implementation details. As each
submodel can contain its own diagram book, modelling these submodels is just as easy as creating
any other model. In addition, submodels can be created from components in any simulation project.
This makes the duplication of existing model structures extremely easy! Utilizing model hierarchy
allows you to make more abstract models, divide your model into subsections that are easier to
maintain and model, and start reusing model structures from project to project.

- 28 -
 Hierarchical models allow implementation details to be hidden inside submodels, making your
models easier to read and understand.

Improved simulation capabilities

While you in earlier version of Studio could only have one simulation for each component in your
simulation project, you can now add as many simulations as you wish. Each simulation will maintain
its own set of variable values, but the model structure will be identical between each simulation. A
change to the Shared Diagrams of the component will always be reflected in all simulations. By using
Private Diagrams, however, you can create diagrams that are unique to the simulation you are
working on.
Improvements have also been done to time series, where you are now allowed to record summary
information for periods. You can select between several summary types, such as average,
accumulated, max, min, etc.

Summary types enable you to present your simulation results summarized over a period. You can
display summaries in time graphs and time tables.
The improvements also include optimisations to simulation speed; new and better views for
equations, units, co-models, and ranges; and index variables.

New modelling features

The property pages for variable definitions are greatly improved and simplified, especially when
implementing large variable definition expressions. A new function, the VBFUNCTION, allows you
to create your own custom functions by programming them in VBScript.
Variable expression validation will now be performed when you edit a variable’s expression directly
in the diagram. You will also experience new user-friendliness when working with connectors (links
and flows) in Constructor diagrams, including more flexible point editing and useful messages on the
status bar while you work. Moving symbols between diagrams is also greatly improved.

- 29 -
Anonymous flows allow you to gather all the flows from a submodel to another variable (level or
submodel) in your diagram in one common flow. An anonymous flow doesn’t have a flow rate,
obviously, but is a tidy way of showing how the flows run through your system.
When you have finished your simulation model and are working on documenting it, you can copy
model structures directly into your documents as graphics.
Other improvements and new features include improved delete and undo functionality; text search in
tree views, such as the Equations and Units views; the ability to organize your components and
datasets into your own folders; and much more.

Connectivity
Datasets have gone through a complete overhaul since Studio 2001. The user interfaces are now user-
friendlier, and now allow you to drag-and-drop variables from models into the datasets. The internal
workings of the datasets are also vastly improved. The new SAP® SEM dataset allows Powersim
Studio 2003 to run stand-alone and still connect to the SAP systems. You can also create multiple
instances of the dataset within Studio. The user interface for the Spreadsheet dataset is also
redesigned, and makes export and import of data from and to Microsoft® Excel a lot easier.
Studio 2003 introduces the new Studio Dataset, where you can record your time series to internal data
storages inside your simulation project. The Studio Dataset allows you to store data for later use; edit
time series that are used in the simulation manually; share simulation data between several
components; and more.
You have greater control over your external connections in Studio 2003 than in earlier versions of
Studio.

Presentation of simulations
The improvements to time series mentioned above, makes it easier than before to create interesting
presentations of simulation results. Combined with improvements in graphs and tables available in
Studio, this constitutes a powerful means of conveying your message. And you can now even copy
the history of a time graph to the Windows Clipboard!
Hyperlinks can now be made relative, allowing the same simulation interface to be used for all the
simulations within a component.
The introduction of index variables makes it possible for you to create controls that select between the
dimensions of array variables without having to implement this by the use of model variables.

Index variables can be used as parameters by input controls. You can in turn use the index variable to
index the parameter of output controls, greatly simplifying the user interface of complex simulations.
It is now easier to add parameters to your controls and you can even use controls to set simulation

- 30 -
details, such as the start and stop times. Automatic detection of the states of the Switch control
simplifies the creation of buttons that control index variables and logical variables.
Other new features include field codes; rotation of tick labels; bar and step graphs in the Time Graph
control; the possibility to save and load external data (when using datasets) from Presentation Mode;
and much more.

- 31 -
Introduction to System Dynamics
Simulations in Powersim Studio are based on system dynamics. System dynamics is a computer-
based simulation modeling methodology developed at the Massachusetts Institute of Technology
(MIT) in the 1950s as a tool for managers to analyze complex problems. Its primary audience is still
managers, although it has spread widely in academia, where professors and students use it to model
systems from every conceivable discipline ranging from history and literature to biology, physics, and
economics.
The word "dynamic" implies continuous change and that is what dynamic systems do - they
continuously change over time. Their position, or state, is not the same today as it was yesterday and
tomorrow it would have changed yet again.
Using system dynamics simulations allows us to see not just events, but also patterns of behavior over
time. The behavior of a system often arises out of the structure of the system itself, and behavior
usually changes over time. Sometimes the simulation looks backward, to historical results. At other
times it looks forward into the future, to predict possible future results.
Understanding patterns of behavior, instead of focusing on day-to-day events, can offer a radical
change in perspective. It shows how a system's own structure is the cause of its successes and failures.
This structure is represented by a series of causally linked relationships. The implication is that
decisions made within an organization have consequences, some of which are intentional and some
are not. Some of these consequences will be seen immediately while others might not be seen for
several years.
System dynamics simulations are good at communicating not just what might happen, but also why.
This is because system dynamics simulations are designed to correspond to what is, or might be
happening, in the real world.

Solving Problems
When confronted with problems or new situations, we can react to them in several possible ways. The
approach we select is based on prior experience and our knowledge of the problem at hand. As
humans, the most common approach to new problems and situations is to take them apart and
examine their pieces. We do this in the hope that by understanding the pieces we will also be able to
understand the entire problem or situation at hand. We are taught this method in our youth and it is
reinforced almost daily. This approach helps us manage the incredible amount of data, stress,
problems, and chaos that bombards us every day. If we didn't have this ability, all but the simplest
problems would appear overwhelming.
Although this method is a good approach in some situations, it can be inappropriate or even
dangerous under different circumstances. More sophisticated approaches are usually required when

- 32 -
investigating corporate problems. If a company is experiencing a serious threat to its survival, be it
declining market shares or disagreements with the labor union, resources are mobilized to deal with
the problem. The company might already be divided into "parts", such as the accounting department,
the sales department, and so on. Problems affecting the entire company are often blamed on a
department, as when a loss in market shares causes executives to target the sales department for
investigation or punishment. The reason for the problem might seem obvious. The company must be
losing market shares because the salespeople are not selling the product. What is often lost in the
picture is the fact that the sales department depends on many other departments to do its job.
Deficiencies may be in any or all of them. Perhaps the management information services department
has not provided the salespeople with the computer support they need. Or maybe manufacturing has
been suffering from poor scheduling of orders and a backlog has developed. This will in turn make it
harder for the salespeople to sell the product to customers who want an immediate delivery. A number
of factors may be the cause of the problem, which may come to light only when the interactions
among all parts of the corporation, and not just the parts themselves, are examined.

Connecting the Pieces

We are viewing a system when we look at a group of individual parts, as well as the connections or
interactions among these parts. A corporation is an example of such a system, composed of many
departments that in turn act as systems themselves. When we study the parts and the interactions
between them, we in fact study the entire system.
The study of systems is not new. It dates back to the 1920s when researchers from different
disciplines began to see that many patterns were common to all fields. A new field, general systems
theory, developed around the notion that no matter how different the parts of individual systems were,
they were put together using a set of rules common to all. Systems theory suggests that knowledge of
one type of system can be applied to many other types. By studying interactions and connections
between the pieces of a system, we can gain knowledge that can be useful when confronted with other
problems.
Systems theory expands further to include two major fields in management science: systems thinking
and system dynamics.

Systems Thinking and System Dynamics

The ideas we have presented thus far are important in both systems thinking and system dynamics.
Systems thinking involves changing paradigms about the way the world works, the way corporations
function, and the human role in each. In systems thinking we look for interrelationships among
system elements. We do this to avoid placing blame in favor of finding the true, long-term solution to
a problem. Seeing the interrelationships can also help us find leverage points within a system (places
where a slight change will have a tremendous effect on the system's behavior). Gaining awareness
about how the system is built up and how it works can also help us avoid solutions that only treat the
symptoms of an underlying problem without curing the problem itself.
Systems thinking is powerful because it helps us see our own mental models and how these models
color our perception of the world. In many cases, it is difficult for us to alter our mental models. There
are always some beliefs or viewpoints that we are not willing to change, no matter what evidence is
presented against it. This causes a certain resistance to new concepts. Problems can occur, however,
when a rigid mental model stands in the way of a solution that might solve a problem. In such
situations, adherence to mental models can be dangerous to the health of the organization.
We all use mental models every day. Our minds do not contain real economic or social systems.
Instead, they contain representations - models - of reality. We use these models in all aspects of
decision-making. Being explicitly aware of our mental models can help us understand why we make
the decisions we do and how we can improve our decision-making processes. If everyone's mental

- 33 -
models are brought to light in the context of an organization, we can begin to see where, how, and
why the models diverge. This is the first step in building a shared understanding within an
organization. As long as mental models remain hidden, they constitute an obstacle to building shared
understanding.
System dynamics is closely related to systems thinking. System dynamics was invented to give
managers a tool to understand the complex systems that they were charged with controlling. The
methodology uses computer simulation models to relate the structure of a system to its behavior over
time. Viewed in this way, system dynamics can translate the understanding gained by systems
thinking into a computer simulation model. By experimenting with this prototype of the system at
hand, we can gain further knowledge about the system. System dynamics is capable of creating a
learning environment - a laboratory that acts like the system in miniature.
Even if building a learning organization - an organization with a high degree of shared understanding
and knowledge about how the organization works - isn't the goal, systems thinking can be a very
valuable tool at the outset of a system dynamics study. It helps bring together the people necessary to
the success of the system dynamics study, and get them in a frame of mind that is open to new ideas,
and allow an evolution of mental models. For change to be successfully implemented, people must be
motivated to learn and able to act on what they've learned, and they must be in an environment of
open and honest exchange. Systems thinking, by helping people in an organization see what the
problems are and how their mental models contribute to the problems, sets the stage for a successful
system dynamics study.
When we conduct a systems thinking or system dynamics study, we must base it on existing
information. The information we can use exists on several levels. The largest and most complete
information available to us is our mental information; everything we carry in our heads. In sheer size,
this information database is the largest and most complete available to us. Next is the written
database, which may be smaller by a factor of a hundred or even a thousand. It represents all the
information we have on paper or stored electronically. Finally, we have a numerical database,
representing all information that is stored as numbers and constituting another hundred- or thousand-
fold loss in the amount and richness of the information. Obviously, the place to find the most
complete information about a situation is in the mental database. What we do with that information is
another matter. The human mind is a brilliant storage device, but we do have trouble relating cause
and effect, especially when they are not close in time. In such cases, we cannot reliably predict the
outcome of any but the simplest situations with the simplest inputs. This is one of the reasons why
computer simulation can be a useful addition to the method of systems thinking.
A systems thinking study usually produces causal-loop diagrams to map the feedback structure of a
system, and generic structures to illustrate common behavior. System dynamics takes the information
about a system's structure that normally remains hidden in mental models and formalizes it into a
computer model. The behavior generated by that particular structure is revealed when the model is
simulated. It constitutes a powerful tool for understanding complex problems. Instead of trying to
relate pieces of information in our heads, we can use the computers to formalize our ideas and
assumptions and then simulate them through time. That is the beauty and power of system dynamics
models.

- 34 -
The Tools and Rules of System Dynamics

System dynamics simulations are based on the principle of cause and effect, feedback, and delay.
Some simple simulations will incorporate only one or two of these principles. More sophisticated
simulations will use all three to produce the kind of behavior we encounter in the real world.

Cause and Effect

Cause and effect is a simple idea, but some simulations based on methodologies other than system
dynamics don't use it. The idea is that actions and decisions have consequences. Price affects sales.
Births affect the size of a population. Advertising affects market awareness.
If we examine these cause and effect relationships isolated, they are usually very easy to understand.
However, when they are combined into long chains of cause and effect, they can become complex.
This is one reason for using simulations. The human mind is good at developing intuition around
complex problems, but poor at keeping track of dozens, hundreds, or even thousands of
interconnections and cause and effect relationships.
We can create causal-loop diagrams, as are often used in systems thinking to illustrate cause and
effect relationships. In such diagrams we use arrows to indicate the relationships. Sometimes,
information about the way in which the relationship works is also included in the diagram. An added
"o" in the diagram implies a "change in the opposite direction". The relationship between price and
sales is such a relationship, where an increase in price leads to a decrease in sales. The relationship
between births and population is of another character. When births increase, so does the population.
This is a situation where a change leads to a "change in the same direction". Adding an "s" to the
arrow in the diagram indicates this.
Figure 1 shows a simple causal-loop diagram. In this diagram, which we will discuss closer in the
next section, price has a negative effect on sales, which in turn has a negative effect on unit costs,
which in turn has a positive effect on price.

Figure 1: A simple causal-loop diagram illustrating connections between price, sales, and unit costs.

Feedback
Feedback is a concept that most people associate with microphones and speakers. A microphone that
isn't properly set up will pick up the sound coming from its own speaker. This sound gets amplified
further by the speaker and picked up by the microphone again. This process keeps going until the
speaker is producing the loudest sound it can or the microphone cannot pick up any louder sound.
If the microphone and the speaker were set up correctly, the system would work linearly. The
loudness of the sound going into the microphone would only affect the loudness of the sound coming
out of the speaker. Because of the misplacement of the microphone, however, the loudness of sound
coming out of the speaker also affects the loudness of sound going into the microphone. Cause and
effect feed back on each other.

- 35 -
This is the general principle of feedback - that some causal chains are linked together so that cause
and effect feed back to each other. This happens everywhere in real world in all kinds of systems,
though people are often not aware of it.
Competitive pricing is one example of such feedback. Competitors set their prices based on our price
based on our competitors' prices.
Infrastructure of network effects is another example. Buyers on eBay are attracted to the large number
of goods available for sale. Sellers on eBay are attracted to the large numbers of buyers. Thus, the
more buyers, the more sellers and the more sellers, the more buyers.
Epidemics are yet another example. Viruses spread when a member of an infected population comes
into contact with someone, who is uninfected, but susceptible. This person then becomes part of the
infected population, and can spread the virus to others. The larger the infected population, the more
contacts, the larger the infected population.
The simple causal-loop diagram presented above illustrates feedback as seen in a price and sales
example. If we used a cost-based pricing strategy, then we could show that as sales increase, the unit
costs for the product goes down. As the unit costs go down, the price can go down. As the price goes
down, the sales go up.
The causal-loop diagram of figure XX shows that the price we charge today will affect what we
charge in the future. A low price will increase sales and reduce unit costs, making it possible to
further reduce price in the future. A high price will reduce sales and increase unit costs, making it
necessary to increase price in the future. This is obviously not the whole story. This structure is only
one part of a larger system and the level of price and sales are also subject to influences from other
variables in the system. But still, this isolated feedback loop is easy to understand.
Feedback relationships can produce a variety of behaviors in real systems and in simulations of real
systems. Figure 2 illustrates four common behaviors created by various feedback loops.

Figure 2: Four common behaviors created by various feedback loops.

„ Exponential growth. (Examples: sales within a start-up; a population of rabbits.)
„ Goal-seeking behavior. (Examples: the number of employees in a workforce and the number of
new hires; the number of potential customers and new sales.)
„ S-shaped behavior. (Examples: Product lifecycle dynamics; the acceptance of a revolutionary
scientific idea.)
„ Oscillations. (Examples: Business cycles; inventory oscillations.)

- 36 -
Delays
Not all cause and effect relationships occur instantaneously. Sometimes the consequences of an action
or decision are not apparent until several days, months, or even years after an event has taken place.
Often the relationship between cause and effect is obscured by separation in time. It is difficult to
understand a system when the consequences cannot be seen in close proximity to the behavior. A
child touching a hot stove will immediately understand the consequences of that behavior and will
likely never repeat the mistake. Many decisions have outcomes that cannot be known for years and
may never be linked to early mistakes.
Delays occur everywhere in the real world. A project may for example have cascading side effects
when critical-path tasks are delayed. New investments can have limited "windows of opportunity" for
making a return on investment. Introducing new products or services sometimes has first mover
advantages.
Delays can produce interesting and complex behavior in systems even when those systems have no
feedback and limited cause and effect complexity. To illustrate this let us look at the price and sales
example once again.
Revenue for a product line is determined by multiplying price by sales. So increasing either price or
sales will result in higher revenue, as long as nothing else changes. Figure 3 shows this relationship.

Figure 3: A diagram illustrating the effect of sales and price on revenue.

In this view of the world, it makes sense to always increase price because revenue will always be
higher. In economic terms, demand is perfectly inelastic with respect to price in this example (there is
no relationship between price and sales). The behavior of the variables can be seen in Figure 4.

Figure 4: The behavior of the sales, price, and revenue variables. Notice that even though price and
revenue change during the simulation, sales does not change.
However, demand is usually affected by price, though often not immediately. Customers may take
time to adjust their consumption patterns even after a price change. This may be because they have
imperfect information about alternatives, or some infrastructure is temporarily forcing them to a

- 37 -
specific option.
For example, in the early 1970s, gas prices around the world increased quickly and substantially. Car
owners couldn't reduce their demand for gas instantly - they owned an expensive asset (i.e., their big
car with high gas consumption per mile) that couldn't be immediately replaced. Also, a lot of travel
that car owners do isn't optional, for example work commute. Moreover, not many cars with low gas
consumption per mile existed in the early 1970s. It took consumers years before they could adjust
their consumption patterns.
This linkage between price and sales is now similar to the original link we saw in the causal-loop
diagram, except that we have introduced a delay, as shown in Figure 5.

Figure 5: The effect of changes in price on sales is illustrated by a delay.

Because there is no delay between price and revenue, when the price shock occurs, the first thing to
happen is that revenue goes up. Eventually, however, customers are able to adjust their consumption
patterns and sales will go down, resulting in lower revenue. Whether the new revenue is lower or
higher than it was before the price shock depends on how sensitive sales is to price. The behavior of
the price shock and the subsequent changes in sales and revenue can be seen in Figure 6.

Figure 6: When a sudden change in price occurs, it causes an immediate increase in revenue.
Because of the delay involved, sales will not decrease immediately. After a certain time, however,
sales will drop, resulting in a decrease in revenue as well.
Even though this system contains no feedback and only three cause and effect linkages, the behavior
of the system is already becoming complicated. This is due to the delay in the system. This is also a
good example of how structure creates behavior. Through one additional delayed linkage in the
system, the results are vastly different.
Understanding the concepts of cause and effect, feedback loops, and delays provides a good
foundation when beginning to uncover the intricacies of a system's nature - what elements are acting
on other elements and whether the interaction is positive or negative. However, feedback loops alone
do not indicate what the entire system's behavior will be. It is hard to anticipate the behavior of a
system from a causal-loop diagram representing the feedback structure of a system alone. Such
diagrams are useful when it comes to isolating the feedback structures. When creating a system

- 38 -
dynamics simulation, however, we represent the feedback structures by the use of levels and flows.
Levels and flows are the main building blocks of computer simulation models in Powersim Studio.
Every element in feedback loops, and therefore every element in a system, is either a level or a flow.
Levels are accumulations and flows represent the changes to these levels. Flows fill up or drain the
levels, much as the flow of water into a bathtub fills it, and the drain at the other end (another flow)
empties it. This action of flows being accumulated in levels is the cause of all dynamic behaviors in
the world.

Levels
Things accumulate or pile up around us every day. Many managers deal with the issue of inventory,
one of the most important accumulations in many companies. System dynamics refer to such
accumulations as levels. Levels give a snapshot view of reality. Their values tell us how the system is
doing at any given point in time. If time suddenly stops, levels would remain and be observable and
measurable. Levels can best be understood as the "nouns" of a system; they are somewhat static or
inactive. Levels have several distinct characteristics. They have memory, which means they do not
change instantaneously. They do change, of course, through the actions of flows, but it takes time. If
our inventory is 6,000 units and we want to increase it to 7,500 units in anticipation of an increase in
demand, we must allow some time for those additional 1,500 units to be placed in our warehouse. A
delay is involved in changing any level, be it inventory or the amount of water in a bathtub. A level
may seem to be changing instantaneously, but there is nonetheless a delay, no matter how small.

Flows
If levels are the nouns of a system, flows are the "verbs". Flows are action variables, creating
dynamics when they accumulate in levels. Dynamics are not created by feedback loops, even though
it may seem that feedback (transmission and return of information) is causing the changes to occur.
Dynamic behavior can occur with absolutely no feedback, because it is simply the result of flows
accumulating in levels. Without flows, levels would never change, and there would be no dynamic
behavior. Thus, flows represent the system's activity and are dependent on the values of the levels.
Because levels are increased or decreased only by their associated flows and flows depend on levels,
an alternating level-flow structure must exist for all systems.

Decisions and Policies

Many people intuitively understand the difference between decisions and policies. However, we often
do not realize that every decision we make is somehow governed by a policy. Even the system of a
swinging pendulum can be described in terms of its "decisions" in the face of governing policies (the
rules of physics).
Within corporations, the distinction between the two is extremely important. Managers must often
make decisions based on limited information and their own previous experience. They may have little
or no control over what information they receive, what form it is in, when they receive it, or how
much they receive. When their decisions turn out to be wrong, they are often blamed for
misinterpreting the data. Sometimes the conclusion is that the manager simply didn't have enough
information to make the correct choice. Unfortunately, the actual problem is usually much deeper.
As mentioned earlier, the behavior of a system is a consequence of its structure. Therefore, the only
real changes we can make to a system are changes to the structure. Other changes to the system will
soon be canceled out through the actions of negative feedback loops. Providing the managers with
more and more information is not necessarily the correct solution, since too much detail or complexity
might do more harm than good. It is often a better idea to examine the structure of the organization.
This way we can gain knowledge and insight about the policies of the company; the rules of the

- 39 -
organization, spoken or unspoken, implicit or explicit, that provides the setting for decisions.
We can illustrate this idea easily with the concept of inventory. Companies that must keep inventory
have specific rules for when more inventory is ordered. No matter who orders the inventory, the
decision of when and how much to order will always be the same in the sense that it fulfills the
inventory policy. If the policy is that inventory must not drop below 6,000 units, and it takes four
weeks to receive orders from suppliers, decisions regarding inventory can be made accordingly.
When problems with inventory develop - there is too much or too little inventory or the amount has
fluctuated wildly in the past six months - a common reaction is to look for the person who has been
"ordering the wrong amount". This can be described as a reaction-driven or "open-loop" approach.
However, it is often more appropriate to examine the pattern of behavior and compare it to a desirable
pattern. If the inventory pattern is essentially chaotic while the desired pattern is a flat line, we must
make changes to the structure - the policies - of the organization, not just the decisions.

Decision-making Process
Decisions must always be based on observable variables. In a system dynamics model, this means that
decisions must be based entirely upon levels, as flows are never instantaneously observable and
therefore can never affect instantaneous decision-making. In the example above, the decision on how
much inventory to order must be based on the present value of the level of inventory. Levels can
represent the actual state of the system at a given point in time (current inventory) or the desired state
of the system (desired inventory). When there is discrepancy between actual and desired conditions,
corrective actions are usually taken to move the actual state closer to the desired state.
The first attempt to solve a complex problem rarely succeeds. This is not surprising, given the
complex cause and effect relationships and feedback loops that exist in most systems we are in
contact with. Usually, corrections change the system and lead to a total redefinition of the problem.
Decisions are attempts to move the system toward our goals. Actual conditions are continuously
compared to the desired conditions and action is taken according to the discrepancy between them.
This is an iterative process. In the context of a corporate model, decisions could be how many orders
to submit to the supplier to replace inventory, how many workers to hire, or when to replace capital
equipment.
A decision to replenish inventory should be based on the present level of inventory (a level) and not
on the rate of sales (a flow). Levels should be the only inputs to decisions; decisions control the flows
to and from levels, and the flows determine the change in the levels. As Jay Forrester states it: "Only
rates [flows] resulting from decisions change levels. Only levels control decisions and rates of flows.
In other words, decisions control all processes of change". Decisions are governed by policies.
Therefore, the way decisions control change is through policies. Flows are defined by equations, and
these equations are statements of system policy. Policies describe how and why decisions are made.
Specifically, it is the policy statement that attempts to move the system toward a goal. It provides the
connection between information inputs and the resulting decisions stream.
Policies may be informal, such as a consequence of habit, intuition, personal interest, and social
pressures and power within the organization. They can also be explicit, with a formal awareness of the
reasons of action. In the latter case, participants know exactly what policies are guiding their
decisions and are able to anticipate the actions of others in a similar situation. Informal policies can be
hazy, but the system dynamics model attempts to make them explicit. In such a model, informal
policies are treated with as much concern as explicit policies. They are considered equally important
in understanding the behavior of a complex system. To truly capture the problematic behavior of a
system, a system dynamics model must represent the basic policy structure of that system. The model
can then be used to try out various policies before implementing them in the real system. In this way,
effective policies can be developed to provide a proper guiding framework for the average manager.
The ultimate goal, if real change is sought, is to find the optimal mix of policies that create the desired
behavior (smooth growth of revenue, constant inventory, etc.), no matter who is in the decision-

- 40 -
making process.
Corporations can be designed so that only highly skilled professionals will be able to navigate the
company through the ups and downs of the business world without creating the symptoms of the
corporation's problems. However, with improved policies and structures, managers will have the
tendency to make correct decisions because the structure has been built with an autopilot in mind
(goal-seeking feedback loops, for example). Just as non-pilots in crisis situations have been able to
land planes because of the design of the plane, so can managers more easily "pilot" a well-designed
company.
This doesn't mean that every manager should be thinking in terms of corporate design instead of
corporate management. The people who should be involved in corporate design are those who have
the power to truly change the structure of the company - the executives. According to Forrester,
corporate executives should be corporate designers. They should create the necessary policies so their
managers will be able to make good and informed decisions. Changing the structure - the policies - of
the corporation can eliminate problematic behavior of the corporation, and the only people in a
position to do so are the executives.

Building the Models

In the previous pages we have introduced the concepts and principles of system dynamics. For many
of us, using these concepts represents a new way of viewing the world around us. By using Powersim
Studio, we can formalize these concepts and views of the world into a computer simulation model.
Let us take a closer look at the stages of the modeling process. Although we will go through these in a
certain order, we should always keep in mind that creating simulation models is an iterative process.
Usually, when creating a model, we will not create it in a linear fashion. Instead, we will advance one
step, then take three steps back and reevaluate everything we've already done.
This is the art of modeling: it is subjective, frustrating at times, and in the end we can never say that
the model is "correct" or even finished. It is simply one representation of reality, built to explain a
particular problem. You may find that you learn more in the process of creating the model than in
manipulating it after it is finished.

Problem Definition
The modeling process begins with defining a problem. The problem definition is the keystone of the
entire activity. Although it might sound like the easiest part, it is not enough to have a vague notion
about the problem behavior. Defining the problem is essentially defining the purpose of the model.
The problem should therefore be defined as precisely as possible. This definition is the basis of all our
future efforts and our guide in decisions concerning boundaries and validity of our model. The
narrower our focus, the easier it will be to resist the temptation to overdo the structure.
Numbers are a useful tool in this stage of the process. If we can use numbers to define the problems,
such as real inventory data to illustrate the problem of inventory fluctuations, we will be better
equipped to define the problem. If no real data are available, it is extremely useful to draw the shape
of the behavior against time. If the problem concerns the interactions of variables, such as the effect
of seasonal fluctuations in demand on the level of inventory, it is necessary to map the relevant
variables against each other. This way we can build an understanding of how each of the various
variables affects each other. We should always keep in mind that system dynamics models are not
concerned with the behavior of individual variables. The main focus is on how each variable interacts
with the other variables to produce the system's behavior.

- 41 -
Model Boundaries
Given the problem definition, we can start to set the boundaries of the model. Creating boundary
diagrams can be useful at this stage of the process. Such diagrams will help us identify the variables
to include in the model, and whether these variables will be endogenous or exogenous. A boundary
diagram is shown in Figure 7.

Figure 7: The boundary diagram illustrates the scope of the model. Exogenous variables are usually
parameters to the model, while the behavior of endogenous variables are generated within the model.
Endogenous variables are those whose behavior is generated within the model. Exogenous variables
are essentially parameters to the model. Their values come from outside the model, and they can
usually be considered constant. The model should only contain the structure that is required to
generate the problem behavior. We should always attempt to eliminate structures that don't contribute
a significant difference to this behavior. Likewise, if the behavior of the model changes wildly when
parameters are changed, the model is not generating the behavior internally. It is simply too
dependent on outside influences. In such cases the model boundaries should be extended to make
room for these exogenous variables to become endogenous.

Variables and Reference Modes

The next stage in the process is to make reference modes for all relevant variables. Reference modes
graph the behavior of variables over time. It is essential that we have a reference mode of the problem
variable itself. However, we do need reference modes for all other variables as well. This may sound
tedious, but there is a good reason for including them. When we simulate our model and discover that
it isn't working the way we anticipated, we can compare the behavior of the variables to the reference
modes to identify missing or incorrect structures in the model.
We can create reference modes in various ways. One quick solution is to quickly sketch the behavior
of the variables based on your own experience or anticipation. If you are experimenting with
Constructor, or creating small models to illustrate decisions or facts, this can be a desirable way of
working. Another more time consuming and costly way is to base the reference modes on real data.
Sometimes these data are readily available in the organization, but often we will have to gather the
system data ourselves. The process of data collection will of course vary from system to system. No
matter how we collect the data, the purpose is to understand the structure and behavior of the

- 42 -
problem.
When we are comfortable with the collected data, we must identify the key concepts and components
to include in the model. It is important to refer back to the problem definition for the model. This
helps us structure our information, and to start generating names and units of measurement for
variables. The list of variables usually becomes very long. From this list, we should identify primary
system variables. Again, we can use the problem definition and boundaries of the model to identify
these primary system variables. We can throw out the variables that are irrelevant to the purpose of
the model and set aside the variables that we are not sure of. The latter ones might become helpful
later, when we arrive at the stage of model design.

Conceptual Model Design

Now that we have reference modes that we have confidence in, have a clear view of the domain of the
problem, and have generated a list of primary system variables, we are ready to create a conceptual
model design. This is the stage at which we will make causal-loop diagrams and diagrams using
levels and flows. Which comes first is a matter of personal preference, and it might be useful to
experiment with both to find the most comfortable. At this stage we are simply trying to formulate the
problem, using the primary system variables, in terms of feedback loops or levels and flows. It is
usually a good idea to go down the list of variables first and identify levels, flows, or constants.
We can begin to determine which variables influence each other and how they can be connected by
physical connections (flows to levels and levels to flows) or information links (all other connections).
The units of measurement can provide a good check in this stage.

Model Formalization
With the conceptual design in place, we can start creating the Constructor diagrams and defining the
equations of the model. Although Powersim Studio takes care of the calculus, we need to relate the
variables in much the same way as we would create cell expressions in a spreadsheet. The units of
measurement for each variable will now provide an instant check for you, since Powersim Studio
requires the units of measurement to be consistently defined throughout your model.
If we look at a simple inventory model, we would probably have a 'Production Rate' (a flow) filling an
'Inventory' (a level). The flow rate would be dependent on the inventory level, since production will
depend on how much inventory is available. In addition, the flow would also be dependent on two
constants, namely the desired level of inventory ('Desired Inventory') and the time delay in adjusting
the production rate ('Inventory Adjustment Time'). The units of measurement of such a system could
be 'widgets' for 'Desired Inventory' and 'Inventory', and 'weeks' for 'Inventory Adjustment Time'. The
equation defining 'Production Rate' (the flow rate) must combine the variables it depends on in a form
that yields a rate equation with the unit 'widgets/week', as shown below.
('Desired Inventory' - 'Inventory')/'Inventory Adjustment Time'

In addition to specifying the equations, this stage also involves choosing parameter values. We should
always keep the real system in mind when we choose constants. We might already know the values
from the system data we collected at the beginning of the study, or we might have to estimate
reasonable values. The key is to be consistent. Initial values of levels should also be realistic for the
particular system. In the beginning, however, it might be a good idea to stick to round numbers, to
make it easier to evaluate the outcome of the model.
Sometimes we find it difficult to relate variables directly using strict, mathematical equations. In these
cases it is often better to create a graph that expresses the relationship between the variables, and use
graph functions available in Powersim Studio to define them. Graph functions are most useful when
representing nonlinear relationships between variables. These graphs are also specified in this stage.

- 43 -
Simulation
We are now ready for the simulation stage of the system dynamics modeling. When we have put our
conceptual model into the computer using Powersim Studio, and all the variables and equations are
well defined, we can simulate the model and view its behavior over time. It is often useful to try a few
"mental simulation" exercises before simulating the model. We should try to imagine what the model
should do when it is simulated. When the model is simulated, we will see whether the actual behavior
differs from our expectations - it most probably will - and thereby have a starting point in figuring out
why. It could be that the structure of the model is in error. It could be that we forgot to take certain
variables into account and that our expectations of the behavior were wrong.
When we simulate the computer model, we must set up appropriate simulation settings for the model.
The two most important are the time horizon and the time step. The time horizon represents the period
of time we want our model to simulate. It is specified by a start and stop time given relative to the
selected calendar. The time horizon will vary from model to model, and we will usually select it so it
matches the time frame of the problem behavior. The time step represents the time interval that the
simulation progresses for each calculation. The shorter the time step, the more calculations Studio will
perform, and the slower the model will run.
Once we have determined the time horizon and time step of the simulation, we will be able to
simulate our model under different conditions and observe the results. To truly understand the model,
we must relate the structure we have created to the behavior that results from simulating the model.
If we cannot get the behavior we want we must go back and reexamine the structure of the model and
try to determine why it is creating the unwanted behavior. The causal-loop diagram is often useful in
this regard. When we understand why the model generates a certain behavior, we can experiment with
changes in the structure to generate the actual problem behavior as we described it in the early stages
of the model creation. When the model adequately represents the real problem, we can use it for
policy analysis and experimentation. We now have a mini-laboratory in which to simulate the effects
of various policy changes before implementing them in the real system.

Building Blocks in Powersim Studio

Powersim Studio is a modeling environment based on the science of system dynamics. Studio allows
us to model systems - with all their cause and effect relationships, feedback loops, and delays - in an
intuitive graphical manner. Symbols representing levels, flows, and "helper" variables (so called
auxiliaries) are used to create graphical representations of the system in Constructor diagrams. Flows
and information links represent relationships and interconnections. The entire structure of a system,
no matter how complex, can be represented in Studio by the use of these variable types and
connections.

Levels and Flows

In a system dynamics model, the structure of the system is represented mathematically. A level is the
accumulation (or integration) of the flows that causes the level to change. In integrating a function, we
are simply measuring the area underneath the function by dividing it into equal-width partitions and
then summing up the area of all the partitions. This is illustrated in Figure 8.

- 44 -
Figure 8: When integrating a function, the area underneath the function is measured. This is done by
dividing it into equal-width partitions and summing up the area of all the partitions.
When creating a simulation model graphically in Studio, connecting the variable symbols generates
the integral (flow) equations. Every variable in the model is defined by an equation, in the same way
as cells in a spreadsheet are defined.
In Studio, boxes represent levels. Double arrows represent the flows, and the flow is controlled by a
flow rate. The flow rate is defined in the same way as auxiliaries (see below). Figure 9 shows a simple
model when created graphically in Studio.

Figure 9: A simple model created in the graphical modeling language of Powersim Studio.
Notice the cloud-like symbol to the left of the first flow and to the right of the second flow. These are
the source and sink of the structure, respectively. The cloud symbol indicates infinity and marks the
boundary of the model. For instance, in the simple structure illustrated in the figure, the level is the
'Workforce', measured in people, which is increased by the 'Hiring Rate' (flow) and decreased by the
'Firing Rate' (flow). The clouds tell us that in this model we are not concerned with where the hired
people come from or where the fired people go. That information is beyond the model boundaries.
If we were interested in including this information, we could add another level to the left of the hiring
rate and one to the right of the firing rate extending the model boundary. This is shown in Figure 10,
where we have the hiring rate draining a level of applicants, and the firing rate adding to a level of
former employees.

Figure 10: The model of Figure 9 with extended model boundaries

Auxiliaries
While it is possible to create an entire model with only levels and flows, Studio has a few more tools
to help us capture real-world phenomena in a model. To achieve a certain level of detail or to aid in
the formulation of flow rate equations, it is sometimes necessary to model a variable as an auxiliary.
In Studio, a circle represents auxiliaries, as shown in Figure 11.

- 45 -
 Figure 11: Auxiliary
An auxiliary is used to combine or reformulate information. It has no standard form; it is an algebraic
computation of any combination of levels, flow rates, or other auxiliaries. Although auxiliary
variables may appear to be accumulations, they have no memory, unlike levels. Auxiliaries are used
to model information, not the physical flow of goods, so they change with no delay, instantaneously.
They can be inputs to flows, but never directly to levels, because flows are the only variables that
change their associated levels. Levels, however, can be inputs to auxiliaries. Note that flow rates and
auxiliaries are defined in exactly the same manner. The difference is that the flow rate is connected to
the flow valve, and thereby controls the flow directly.

Constants
Constants are, unlike ordinary auxiliaries, constant over the time period of the simulation. A diamond
represents these constants, as shown in Figure 12.

Figure 12: Constant

A constant is defined by an initial value, and maintains this value throughout the simulation, unless
the user changes the value manually (by using a slider bar, for example). For instance, in a one-year
simulation, a company may have an essentially fixed workforce that can be represented as a constant
auxiliary. If the simulation were to expand to 20 years, however, workforce would most likely
become a level and be allowed to vary over time. Once again we return to the important issues of
problem definition and model boundaries. Without a clearly defined problem to model, we will be
unable to set the proper boundaries.
Sometimes we find ourselves confused about whether an element of the system should be included as
a constant auxiliary or as a level. In these situations we should try to rethink the problem. We should
think of the time period of the problematic behavior and whether or not it is reasonable to expect the
element to change over that period. We will then be in a better position to decide what elements
should be constants and what elements should be allowed to vary during the simulation.

Information Links
Connections are made among constants, auxiliaries, and levels by means of information links. These
links appear as thin connectors in the Constructor diagram, as shown in Figure 13.

Figure 13: Information links connects various variables.

Information links show how the individual elements of the system are put together. In a sense they
close the feedback loops. We have already seen how flows change the levels by filling them or
draining them. Information links can transfer the value of the level back to the flow, indicating a

- 46 -
dependence of the flow on the level, as well as the obvious dependence of the level on the flow, as
seen in Figure 14.

Figure 14: A closed feedback loop representing the interest earned from an account in a bank.
For a Constructor diagram to be consistent, the equation that defines a variable must contain all the
variables that are linked to the variable itself. In the simple model structure in Figure 14, this means
that for the model to be consistently defined, the definition of Interest must include Account and
Interest Rate. In this example, the definition of Interest is simply
'Interest Rate' * 'Account'

Creating a Model

The previous chapters have introduced you to the field of system dynamics and the principles behind
building computer simulation models. To help you create your first simulation models we have
created a Learning Lab that will guide you through the steps of modeling and simulating a model. The
Learning Lab also contains finished models that you can study.

- 47 -