OT Networking

Reference Architecture
usa.siemens.com/industrial-networks

© 2023 by Siemens AG, Berlin and Munich

The OT Networking Reference Architecture – Table of Contents

Table of Contents
1 Introduction 3

2 Understanding the current state 4

2.1 The trends of today and tomorrow 4
2.1.1 Reliably interconnecting systems 4
2.1.2 Being ready for future changes 4
2.1.3 Insuring system safety and availability with cybersecurity 4
2.2 The Undesigned Network or the status quo of IACS networks 5
2.2.1 The Evolved Network 5
2.2.2 The IT-OT Entangled Flat Network 7

3 Components of an IT / OT Network 8
3.1 Hardware 8
3.1.1 Switches 8
3.1.2 Routers 8
3.1.3 Firewalls 9
3.1.4 Industrial Wireless Local Area Network (IWLAN) 10
3.2 Network Software 10
3.2.1 Network Management System (NMS) 10
3.2.2 Network Services Software 10
3.3 Cybersecurity Software 11
3.3.1 Secure Remote Access 11
3.3.2 SIEM 11
3.3.3 Intrusion Detection System/ Intrusion Prevention System (IDS/IPS) 11
3.3.4 Zero Trust Concept 11

4 Categorizing Future-Ready OT Networks 12

4.1 Small OT Networks – Connecting Isolated Systems 14
4.1.1 Key Considerations 14
4.2 Medium Networks – Connecting Distributed Systems 16
4.2.1 Key Considerations 17
4.3 Large Networks – Connecting Multiple Facilities 18
4.3.1 Key Considerations 18

5. Key Steps 21

Resources 22

Figure list
Figure 1 – The Evolved Network 6
Figure 2 – IT-OT Entangled Flat Network 7
Figure 3 – Small Network Reference Architecture 14
Figure 4 – Medium Network Reference Architecture 16
Figure 5 – Large Network Reference Architecture 18

2 Version 2, 03 -2023
 The OT Networking Reference Architecture – Introduction

1. Introduction
Operations Technology (OT) networks require different structures and skills to properly design, deploy, maintain, protect,
and improve. These skills differ from standard Information Technology (IT) Networks due to the requirements of the OT
environment. These differences include how the data is used, expected lifecycles of the equipment, acceptable latencies
in data transfer, and operational reliability and availability requirements of an operating plant, and many more. This
document provides an overview of the steps needed to implement a cybersecure plant-wide OT network.

When discussing OT networking, it is important to understand the current state of the systems and the desired goals. With
this assessment, we can design a more complete solution that meets the needs with hardware, software, and services while
considering current and future roles and training for personnel. Commonly overlooked factors during network improvements
are the potential future requirements of the plant and broader enterprise. This is why we advocate building a scalable network
which considers not just the current needs but also creates a flexible solution that can easily adapt to future trends. By
building future-ready networks, we are prepared to embrace the latest advancements in technology, enabling easier
adoption of new manufacturing concepts, and at a lower total cost of ownership.

The purpose of this document is to inform on OT networking best practices and reference OT network architectures. Delving
into specific technologies and how to implement them is left to consultants, technical manuals, and further training.

Version 2, 03 -2023 3
The OT Networking Reference Architecture – Understanding the Current State

2. Understanding the current state

A critical step in being able to design a solution that meets the current needs and considers the trends of tomorrow is to
understand the current state of the network.

Every network begins by having a need for one system to communicate with another. In the past, this was accomplished by
implementing proprietary and physically complicated wiring schemes. The systems of today and of the future meet this need
with a network based on an open standard called Ethernet. By utilizing Ethernet, every vendor can enable their devices to
communicate with another vendor’s devices. It is on top of Ethernet that common industrial protocols like PROFINET, Modbus
TCP/IP, EtherNet/IP and EtherCAT function. The use of Ethernet creates a universal platform for automation. The architectures
proposed in this document support all standard Ethernet-based automation protocols.

2.1 The trends of today and tomorrow

Most trends in the networking space of Industry 4.0 can be summarized in the following three categories.

2.1.1 Reliably Interconnecting Systems

The days of isolated systems are coming to an end due to the cost advantages of being able to automate a greater
number of processes. Gathering data from field level and utilizing it for better planning is but one example of how
manufacturing is changing from this explosion of data. This shift in our expectations can only be addressed by having
a plant-wide network capable of transferring all the data generated by our systems. When these automated processes
become part of our day-to-day operation, reliability becomes critical. We must design our OT network to be reliable
and resilient.

2.1.2 Being Ready for Future Changes

Industry 4.0, Industrial Internet of Things (IIoT), and Edge computing have been driving immense changes in our OT
networks. The pace at which we will need to embrace these changes has never been this fast, and yet, it will never be
this slow in the future. We must therefore design our networks not just for the current needs, but to be flexible and
ready for the inevitable change.

2.1.3 Ensuring System Safety and Availability with Cybersecurity

Cybersecurity has become synonymous with safety in the world of increasingly connected digital assets. Now that
life-safety systems are increasingly operating on Ethernet networks, the availability and reliability of our network has
become mission-critical to our manufacturing plants. Due to increasing portions of our operations becoming digital,
we increase the costs incurred when these digital systems fail. This creates a situation that makes cybersecurity
attacks more profitable for bad actors. It is therefore imperative to become educated on the risks that exist.

Many standards and models have been created to provide best-practices in terms of cybersecurity. IEC 62443 is an
internationally recognized series of standards that provide a framework to design and operate cyber-secure Industrial
Automation and Control Systems (IACS). Another prevalent OT cybersecurity standard is NIST SP 800-82 which is used
more often in critical infrastructure industries (e.g., electrical power distribution, water/wastewater, oil and gas
pipelines), but is also being adopted in more typical manufacturing operations (e.g., chemicals, automotive, food &
beverage). Both standards have been in development, in various forms, for about two decades and are now very
similar in their basic recommendations. This document will rely most directly on IEC 62443 as it has a more general
applicability and greater adoption internationally. The architecture drawings included will make references to the
Purdue model as it provides a model that segments devices and equipment into hierarchical functions and layers.

This document discusses some of the technical controls for securing an OT or Industrial Control System (ICS) network.
However, it is extremely important to develop and implement a dedicated OT Cybersecurity Program to be able to
fully assess, design, implement, maintain, and continuously improve a secure OT network. While the design and
implementation of a cybersecurity program is beyond the scope of this document, the IEC 62443 standard series
contains a section (Part 2-1) which describes requirements for OT cybersecurity programs.

4 Version 2, 03 -2023
 The OT Networking Reference Architecture – Understanding the Current State

If we think of how safety programs developed over the 1990s and 2000s, we realize that without a continuous
improvement program and a well-structured organization to carry out the program’s policies and procedures, a safety
program would quickly become an afterthought and the safety posture of your plant would quickly decline. The same
is sure to happen with new cybersecurity initiatives if there is no OT-specific cybersecurity program with dedicated
resources and well-defined management responsibilities.

2.2 The undesigned network or the status quo of IACS networks

Many legacy OT networks are struggling to keep up with the demands of digitalization, including the multiple
connections to business-level systems, outside service providers, machinery or packaged OEMs systems, and the large
number of Ethernet devices which have been added over the last three to five years. This is often because these
networks were not designed for this level of connectedness, and often, they were never actually “designed” at all.
They often grew haphazardly as required. Over the years and decades, automation engineers or production personnel
added switches and connections to get the data where it was needed but rarely had time or budget to take a step back
and improve the overall architecture. This led to two common, but problematic situations – the Evolved network and
the Flat network.

2.2.1 The Evolved OT Network

Many companies have Ethernet networks that grew organically over time, from mere replacements for the serial
connection between a PLC and an HMI to interconnecting multiple cells or areas and then to providing data for
higher level systems. Then the I/O and instrumentation layer began to incorporate Ethernet as an easy and
standardized approach to replace multiple proprietary field busses. Eventually, these networks morphed into a vast
web of interconnected instruments, drives, controllers, HMIs, SCADA, remote access, databases, recipe systems,
batch managers, Manufacturing Execution Systems (MES), Enterprise Resource Planning (ERP) systems, and various
cloud-based functions (e.g. machine learning, data analytics, data warehouses, etc.).

These networks (as depicted in Figure 1) are characterized by a mix of multiple architectures (line, star, ring) and
typically have little to no redundancy, often with multiple single points of failure which could bring down large
portions of the network. These networks also often contain unmanaged switches which limit diagnostics and
security capabilities. Additionally, evolved OT networks may be physically separate (also known as air-gapped) from
the business network but are more typically connected via a single firewall controlled by IT. Documentation is also
often lacking and typically out of date.

QUICK SIDE NOTE on the myth of security via “Air Gapped” Networks: There really is no such thing as security by
air gap in industry. Any control system requires maintenance, updates, and probably expansion. Any time a USB
flash drive or laptop from a non-dedicated source is connected to the network or a device on the network it
represents a breach of the air gap. Due to the lack of documentation and the freeform nature of these networks,
they are difficult to troubleshoot and require significant additional time and effort to expand or modify. They
often exhibit intermittent connectivity or data throughput issues with no efficient methods to detect the exact
location or cause of the issues.

Version 2, 03 -2023 5
The OT Networking Reference Architecture – Understanding the Current State

Typical Evolved Network

Enterprise Network
Core

WAN WAN IT Core External Internet

Router Firewall
May or May
Not be an
IT/OT Firewall

Direct Connection to
Business Network

Main OT
Main Control Room

Switch

Operator
Stations

Servers

Control Room

Production Backbone
Switch
Distribution

Operator Operator
Station Station

Line 3 Backbone

Switch Switch

Ring

Switch Switch

Production Network
Cell 1-1-1 Cell 2-1-1 Cell 2-1-2 Cell 3-1-1 Cell 3-1-2

Firew

Tree
Switch Switch
Switch Switch Switch

Production Cells
HRP
Access

Switch Switch

Operator Switch Operator

Station Station

Figure 1 – The IT-OT Evolved Network

2.2.2 The IT-OT Entangled Flat Network

Another path to Ethernet on the plant floor is the one that is an extension of the business network into the
Operational Technology (OT) space. This is where IT either desired to maintain control over all Ethernet networks
in the enterprise or was tasked with helping their OT colleagues create and maintain an Ethernet network.
Thesecnetworks are known as “flat” networks because there is no protection scheme between production
systems and even the rest of the corporate network.

These flat networks may be well designed but use IT-style line or star topologies which can be more efficient
from a cabling and port count perspective but lack the resiliency of a ring style. Because it is one big network,
the OT devices are often intertwined with the IT devices. A flat network presents challenges to both the OT and
IT users as problems on one type of equipment can directly affect the other types. Additionally, IT typically
enforces IT-centric policies and procedures across the full network. This is because there is no segregation which
would allow the OT areas to be managed differently.

6 Version 2, 03 -2023
 The OT Networking Reference Architecture – Understanding the Current State

Flat networks can cause production equipment slowdowns and outages due to non-OT traffic (e.g. video or
large file transfers), network reconfigurations, or firmware updates. Broadcast storms are more common on
large non-segmented networks and can act like a Denial-of-Service (DoS) attack. It is also easier to inadvertently
create a loop in a large flat network which could then also cause communication issues even taking down parts
of the network until resolved. Identifying, isolating, and resolving issues is an immense challenge as the lack of
segmentation forces troubleshooting efforts to consider the entire system rather than smaller sections.

Another underlying issue with IT managed OT networks can be the service level agreements (or lack thereof)
between OT and IT. If a switch goes down or a network problem is suspected during non-office hours there may
be no quick route to troubleshooting and repair of the network. As production relies more and more on Ethernet
based networks, it is imperative that support structures be created which meet the near instant response times
required on the production floor.

IT-OT Entangled Evolved Network

Enterprise Network
Core

WAN WAN IT Core External Internet

Router Firewall
No
IT/OT Firewall

Direct Connection to
Business Network

Main OT ANDON Board

Switch
Main Control Room

Operator IT Workstation
Stations ERP/Email/Web
Servers

Control Room

Production Backbone
Switch
Distribution

Operator Operator
Station Station

Line 3 Backbone
Manufacturing Office

ANDON Board

ANDON IT Workstation IT Workstation

Board ERP/Email/Web ERP/Email/Web Switch Switch
Ring
File
Servers Switch Switch

Manufacturing Office Switch

Production Network
Cell 1-1-1 Cell 2-1-1 Cell 2-1-2 Cell 3-1-1 Cell 3-1-2

Switch Switch
Switch Switch Switch
Production Cells

HRP
Access

Switch Switch

Switch
Operator Operator
Station Station

ERP Station ERP Station

Figure 2. - IT-OT Entangled Flat Network

Version 2, 03 -2023 7
The OT Networking Reference Architecture – Components of an IT/OT Network

3. Components of an IT/ OT Network

When designing an OT network, it is critical to select components that are meant to handle the environmental conditions as well
as the application requirements. As an example, IT equipment is usually placed in climate controlled and dust free environments;
their fans would fail quickly should be they be subjected to the hot and dusty OT environments. This unexpected downtime can
easily cost more than the savings of using components that weren’t designed to operate in OT environments. The selection of
components can be simplified by having industry specific certifications like Class 1 Div. 2 for explosive environments found in the
oil industry. It is also recommended to look for third party certification of the manufacturer’s claims. Designing networks for
hazardous environments is beyond the scope of this document; please consult a trained and experienced control systems
engineer if you have needs for hazardous area networking systems.

3.1 Hardware
The functions of networking components commonly found in plant-wide OT networks are outlined below along with the
recommended Siemens offering.

3.1.1 Switches
There are two main categories of switches, unmanaged and managed. Unmanaged switches offer simple connection of
Ethernet devices with little or no configuration options, but this simple operation comes with significant opportunity
cost. Their unmanaged nature means network diagnostics, security, and network redundancy features are unavailable.
Unmanaged switches are not recommended in a plant-wide network for the reasons above.

Managed switches allow the connection and networking of ethernet devices in a manner that can be configured and
monitored to fit the needs of the network in a secure manner. Noteworthy management functions include redundancy
mechanisms (RSTP, ring networks, etc.), security functions (disabling ports, MAC or IP filtering, and user management),
network segmentation through VLANs, and some managed switches even support Network Address Translation (NAT)
to allow integration of devices into differing IP network structures. In addition to management, monitoring functions
allow users to access diagnostic information that assist in troubleshooting and network optimization. Managed
switches are recommended to be used in a plant-wide network. All switches shown in the figures below are managed.

The recommended Siemens switches are the SCALANCE XC-200, XC-300, XR-300, XM-400 (layer 3 capable), and
XR-500 (layer 3 capable).
Click here for information on SCALANCE X

3.1.2 Routers
Routers move network traffic between different Internet Protocol (IP) subnets and Virtual Local Area Networks (VLANs).
Network segmentation (using VLANs and/or subnetting) is highly recommended in plant-wide networks as it creates
different security cells that can only be accessed with the help of a router or firewall. Routers enable communication
across different subnets and VLANs when one area of the network needs to communicate with a separate segment of
the network. For example, a server may need access to many different production lines that are segmented – in this
case a router will be needed to traverse these boundaries. Routers are recommended and generally required in the
upper layers of the plant-wide network. Routers are placed in the production backbone in the figures below.

The recommended Siemens Layer 3 switches with routing capabilities are the SCALANCE XM-400 and XR-500.
Click here for information on SCALANCE X

8 Version 2, 03 -2023
 The OT Networking Reference Architecture – Components of an IT/OT Network

3.1.3 Firewalls
Firewalls are utilized for restricting communication and access to various areas of the network. A firewall restricts
access by following a set of user-defined rules which describe the allowed communication between specific
devices (IP Addresses) or segments (VLANs or subnets) in the network. Most firewalls can also act as routers
allowing communication between network segments while restricting which specific IP addresses can
communicate to each other and which protocols may be used. Another commonly used function is Network
Address Translation (NAT).

The most common area firewalls will be found are between the internet and the company network. This is
to protect the company network from threats existing on the internet. In a plant-wide network, firewalls are
recommended between the OT and IT network divide and between the various cells / lines and the backbone
for zone/area protection. Recommended types of firewalls are as follows.

3.1.3.1. MAC Firewall (Layer 2)

In areas where additional protection is needed within an IP subnet – Layer 2 firewalls can be utilized to restrict
communication between devices based on MAC addresses. These devices will typically be found within the
Production Cell section in the below figures. In more recent times, Layer 2 or MAC firewalls have been mostly
superseded by Layer 3 firewalls due to easier management over time. If a device is replaced it may be set with
the same IP address but will likely have a different MAC address so the Layer 2 firewall rules would have to be
revised. A Layer 3 firewall relies on IP addresses, so it’s rules would not need revision if a device were replaced
with a different MAC address but the same IP address. Many firewalls can be both Layer 2 and Layer 3 firewalls
at the same time, it is a matter of configuration.

3.1.3.2. Stateful Inspection Firewall (Layer 3)

These devices examine each packet to determine if they meet predefined criteria for permitted
communication, which are defined by rules based on IP addresses and TCP/IP ports. This type of firewall will
often serve as a zone/area firewall and is located between the Production Cell and Production Backbone areas
in the figures below.

Siemens firewalls are capable of both Layer 2 and Layer 3 functionality. The recommended Siemens
firewalls are the SCALANCE S615, SCALANCE SC-600, and SCALANCE M-800 (the M-800 series are Mobile
devices meaning they have wireless interfaces in addition to the wired connections).
Click here for information about Siemens Firewalls

3.1.3.3. Next Generation Firewall (NGFW)

This type of firewall expands on the functionality of Stateful Inspection Firewall with Deep Packet Inspection (DPI).
It may include additional security tools such as an Intrusion Detection System / Intrusion Prevention System
(IDS/IPS), malware filtering and antivirus. The defining feature, DPI, goes beyond the information in the packet
header and evaluates the packet payload. An example of this advantage is that a PLC can be protected from
program downloads from an HMI, but still allow the HMI to read and write data in the PLC. NGFW firewalls are
recommended between the company network and internet and recommended for use between the IT and OT
networks, which are labeled as IT/OT firewalls in the figure below.

For Next Generation Firewalls, Siemens has partnered with leaders like Fortinet, Palo Alto and CheckPoint
to offer their best-in-class software solutions with Siemens’ hardware.
Click here for information about Next Generation Firewalls

Version 2, 03 -2023 9
The OT Networking Reference Architecture – Components of an IT/OT Network

3.1.4 Industrial Wireless Local Area Network (IWLAN)

Industrial wireless networks differ from wireless seen in the IT space. Industrial wireless systems experience
more demanding physical environments and more challenging communication requirements. For this reason,
specialized equipment has been designed for the requirements in the OT environment such as deterministic
communication mechanisms, fast roaming, and features that provide expanded operability in control networks.
In figures below, IWLAN is shown as connected to one of the production cells and can be connected in many
other areas of the OT network. This document does not focus on the architecture required for IWLAN. Please
refer to the additional resources section when considering IWLAN.

The recommended Siemens solution for industrial wireless are the SCALANCE W family of radios.
Click here for information about SCALANCE W

3.2 Network Software

3.2.1 Network Management System (NMS)
Network management systems are utilized for centralized monitoring, management, and configuration in networks of
all sizes. Commonly found in enterprise (IT) networks, the NMS provides several key functions in industrial networks
including – fault, configuration, inventory, performance, security, firmware, and backup management. The
centralized nature of an NMS system limits the need of individual device management and saves a substantial amount
of time in industrial network administration.

The recommended Siemens NMS solution is SINEC NMS – industrial NMS for OT networks.
Click here for information about SINEC NMS

3.2.2 Network Services Software

The recommended Siemens solution for Network Services is SINEC INS, which includes all the services mentioned
below in a single software platform.
Click here for information about SINEC INS

3.2.2.1. Syslog Server

Syslog is a standard networking protocol for centralized message logging. Syslog servers collect status,
fault, and other messages from distributed devices for evaluation.

3.2.2.2 Network Time Protocol (NTP) Server

NTP is a networking protocol for time synchronization between different devices to within a few
milliseconds. The NTP server provides the reference time that various end point devices in an industrial
network, such as PLCs, industrial PCs, or network switches, will synchronize with. Synchronized time allows
more effective troubleshooting and logging.

3.2.2.3 Dynamic Host Configuration Protocol (DHCP) Server

DHCP is a network management protocol for assigning IP addresses within a network from a DHCP server.
The DHCP server assigns unique addresses in a predefined range, which helps prevent duplicate and
misconfigured IP addresses.

3.2.2.4 Remote Authentication Dial-in User Service (RADIUS) Server

RADIUS is a networking protocol that provides centralized authentication, authorization, and
accounting (AAA). The server manages the users and access levels granted to the users.

10 Version 2, 03 -2023
 The OT Networking Reference Architecture – Components of an IT/OT Network

3.2.2.5 SSH File Transfer Protocol (SFTP) Server

SFTP is a networking protocol that provides file access, transfer, and management in a network. The SFTP
server is commonly used in industrial networks for storing and managing device configuration and backup files.

3.2.2.6 Domain Name Server (DNS) Server

DNS servers are the phone books of the internet by providing IP addresses for human readable domain
names (e.g. www.google.com -> 142.251.32.46)

3.3 Cybersecurity Software

3.3.1 Secure Remote Access
Secure remote access systems are utilized to allow secured access to industrial network components using
technologies such as Virtual Private Networks (VPN). Remote access is typically needed by the original equipment
manufacturer (OEM), service providers, and the end user’s engineering team for quickly responding to system issues
affecting production and operation. The remote access software authorizes and grants access to various network end
points based on user credentials and preconfigured communication relationships.

The recommended Siemens solution is SINEMA Remote Connect – secure remote access for OT networks.
Click here for information about SINEMA RC

3.3.2 SIEM
Security Information and Event Management systems monitor log data (typically in a Syslog server) and look for
specific Indicators of Compromise (IoCs), send alerts when suspicious activity is detected, and in some versions may
provide playbooks and integrations to helpdesk ticketing systems to help organize the response to an incident.

3.3.3 Intrusion detection System/ Intrusion Prevention System (IDS/IPS)

An IDS is a software and/or device that monitors a network for malicious activity or policy violations. A detection is
reported to the network administrator or to a Security Information and Event Management (SIEM) system. IDS systems
are typically passive and only notify of an intrusion on the network.

An IPS is an expansion in functionality of an IDS. The “prevention” occurs by responding to a “detected” network
threat by blocking or dropping the packets from the network.

For IDS/IPS, Siemens has partnered with leaders like Nozomi, Claroty, SecureNOK, Palo Alto and Fortinet to offer their
best-in-class solution running on Siemens hardware.
Click here for information about Next Generation Firewalls

3.3.4 Zero Trust Concept

Zero Trust is one of the latest buzzwords that tries to condense several cybersecurity technologies into a single
concept. The overarching idea is that perimeter defenses (firewalls, VPN, IDS, IPS, etc.) are not enough. There is too
great a chance that an external threat will still be able to get into the network and there are many internal threats that
are just as dangerous as the external threats. Thus, there should be no trust of a device or user just because they are
inside the perimeter defenses. This means that every device and user should be authenticated for every connection
initiated on the network. This extends to the protocols and applications that talk between HMI and PLC, engineering
laptop and PLC, or MES and data historian. Zero Trust is a difficult concept to fully apply to an ICS as many devices are
older and have no certificate authentication capability or do not have enough on-board processing power or memory
to implement additional security protocols. However, Siemens current SCADAs, HMIs, PLCs, switches, and security
devices are capable of being configured for use in a Zero Trust environment.
Click here for information about the SCALANCE LPE and Zscaler Zero Trust

Version 2, 03 -2023 11
The OT Networking Reference Architecture – Categorizing Future-Ready OT Networks

4. Categorizing Future-Ready OT Networks

With the infinite possibilities that exist in networks, assessing the state of a network can be a difficult task. The IEC62443 standards
provide two different classification models: maturity levels and security levels. Maturity levels describe the efforts put towards
having documented systems and repeatable procedures and are used to assess the organizational maturity of an OT cybersecurity
program. Security levels (SL) describe the technical risks and the level adversarial capability and motivation faced. The SLs allow
the current state of a network and future target states to be unambiguously defined which then enables a detailed gap analysis
and the identification of necessary steps to prioritize and mitigate identified risks. It is important to note that no cybersecurity
system is perfect and that there will always be some level of risk that must be tolerated. However, the decision to determine what
level of risk is acceptable to the business is reserved for the highest level of management.

The goal of this document is to provide an overview of the technical controls one should incorporate when designing a future-ready
OT network. We will be summarizing the requirements of different maturity and security levels for the following three network sizes:
1. Small – where connecting systems is the highest priority
2. Medium – where building a scalable platform is the highest priority
3. Large – where protecting against advanced threats is the highest priority

Every network will inherently go through phases and transformations throughout its lifetime. It is for this reason that we must
always begin with proper planning. We want to design a scalable system that can adapt to the changes without requiring a
complete redesign.

The core of our network will be used to connect multiple production cells, which will typically increase in number over time.
By following the fundamentals outlined here for the foundational network, expanding the network in the future will proceed
in an efficient manner.

Given the explosion of connected devices and experts

forecasting that the Industrial Internet of Things (IIoT) market
will grow at 24% annually through 2023 – it’s a good bet
that industrial networks will need to be capable of processing
and transmitting large amounts of data from many connected
devices. Preparing for the future of industrial networking
will be essential to remaining competitive.

12 Version 2, 03 -2023
 HOW FIT IS YOUR COMMUNICATION NETWORK?

Consider an Industrial
Network Health Check!
Industrial Network Health Checks typically include:
• Assessment of the current status of the network
• Benchmark testing to identify potential network issues like:
- Packet collisions
- Problematic network architecture
- Sub-optimal device configuration
• Creation of a network asset inventory
• A detailed report with recommendations to achieve a reliable network ready for
future challenges
Receive a free consultation by e-mailing us at siemensci.us@siemens.com

Version 2, 03 -2023 13
13
The OT Networking Reference Architecture – Categorizing Future-Ready OT Networks

4.1 Small OT Networks – Connecting Isolated Systems

New and smaller facilities should begin with the reference architecture below as a baseline that will properly apply
fundamentals of OT networking. This will make for efficient expansion of the industrial network as it grows and
requires more capacity. A small-size network can be classified using the following criteria:
• Less than 50 end point devices
• Network disruption above production cells will not result in substantial financial impact
• A network spanning up to a few lines (composed of production cells) or a small sized facility

Enterprise Network
Core

WAN WAN Core External Internet

Router Firewall

IT / OT IT / OT
Firewalls Firewalls

HMI and SCADA

Industrial DMZ

Historian and Reports

Network Services
Secure Remote Access

Production Backbone
Network Management
SIEM and IDS/IPS Servers
Distribution

Note: Optional, but recommended – Redundant

redundant routers and firewall Connections

Router Router

Production Network
Redundant
Connections

Cell 1-1 Cell 1-2 Cell 1-3

Layer 3
VRRP NAT
Firewall
Firewall

Switch Firewall Firewall

Tree / Star

Production Cells
Ring
Switch
Access

Switch Switch
Line
Industrial
WLAN

Figure 3. - Small Network Reference Architecture

4.1.1 Key Considerations for Small OT Networks

1) Topology
• Ring Networks
o A ring network is found in the Cell 1-1. A ring network is recommended in automation and OT networks to
add resiliency via connection redundancy while maintaining quick recovery times in the millisecond range.
o The most common failure mode is a disruption in physical connection, which makes ring redundancy a
best-practice and an efficient method of increasing resiliency for industrial networks

14 Version 2, 03 -2023
 The OT Networking Reference Architecture – Categorizing Future-Ready OT Networks

• Star Networks
o A star topology network is found in the Production Cell 1-3. A star network is typically less efficient than a ring
network and does not offer redundancy. This introduces many single points of failure into the network, which
include each cable, each end device, and the network devices aggregating the connections.
• Line Networks
o A line topology network is found in Production Cell 1-2. A line topology network offers the lowest possible
level of resiliency in a network due to interdependence on the upper-level devices in the line topology. For
example, if the first device in the line fails – all other devices downstream will lose connection. It is not
recommended to utilize a line topology when uptime is critical.

2) Redundancy
Several layers of redundancy are possible in an OT network that will improve resiliency and tolerance of failure
in the network. Redundancy is not necessary in all situations but is highly recommended in critical connections
that would impact operation if lost.
• Ring redundancy, highlighted in the “Topology” section above, is a form of connection redundancy and is
commonly used in industrial networks to mitigate loss of physical connections between devices.
• Virtual Router Redundancy Protocol (VRRP) can be configured for redundant communication between separate
subnets or VLANs, such as the connections seen between Production Cell 1-1 and the routers in the Production
Backbone routers. In the event of a failure in one router or physical connection, the remaining functional
connection will be detected and utilized as the communication path. VRRP is configured in the redundant router
pair – such as the firewalls in Production Cell 1-1, the routers in the Production Backbone, and the redundant
IT/OT firewalls between the Production Backbone and Enterprise Network.

3) Recommended Software
• Secure Remote Access software is recommended for small networks to enable access by OEMs, service providers,
or engineers to the systems when not physically present.
• NTP server software is recommended to synchronize time across the devices for efficient troubleshooting and
transparency in log timestamps.

4) Additional Security Considerations

• End Point Hardening is a critical first step in making sure the overall system is secure. Changing default passwords,
keeping up with security updates, and following security recommendations in the user manuals must be done.
• Segmentation of the network sets the foundation for creating cells and allowing only the necessary communication
between different components. Segmentation is accomplished using VLANs and subnetting. These require routers
between different parts of the network. Segmentation can be augmented by firewalls to restrict communication
to the specific IPs and Ports needed for the application.
• Firewalls should be placed at strategic entry and exit points to protect assets from external and internal threats.
It is highly recommended to have a firewall between the IT network and OT network as the IT network is usually
connected to the internet. This protects not only the OT network but also protects the IT network from threats
that could come from the OT network.
• A Demilitarized Zone (DMZ) exists to provide for a controlled set of services that are allowed to communicate
from the OT side to the IT side and vice versa. There should not be any communication from the field devices
into the IT side and vice versa.
• In the diagram above, Secure Remote Access can be configured to allow a remote user to only be able to access
a single cell. This can greatly reduce the risk of allowing third-party users access to the OT network as
they are only capable of communicating with a small section of the network or even just a single IP address.
• Documentation and backups are necessary for troubleshooting any issues with the network. Network
drawings, Endpoint information such as IP addresses, MAC addresses, firmware versions, and procedures
should be created.

Version 2, 03 -2023 15
The OT Networking Reference Architecture – Categorizing Future-Ready OT Networks

4.2 Medium Networks – Connecting Distributed Systems

As operations grow and evolve over time, the scale, capacity, and services required of the OT network will
correspondingly increase.
A medium sized network can be classified using the following criteria:
• More than 50 end point devices
• Critical operation where network disruption will result in substantial financial impact
• A network spanning multiple lines (composed of production cells) or a medium-size facility

Enterprise Network
Core

WAN WAN Core External Internet

Industrial DMZ

Router Firewall
IT

Servers

IT / OT IT / OT
Firewalls Firewalls
HMI and SCADA
Industrial DMZ

Historian and Reports

Network Services
Secure Remote Access
Network Management
SIEM and IDS/IPS Servers

Production Backbone
Redundant
Connections Aggregation / Backbone
Distribution

Router Router Switch

Switch
Ring Ring

Switch Switch Switch Switch

Industrial
WLAN

Production Network
Redundant
Connections

Cell 1-1 Cell 1-2 Cell 1-3 Cell 1-4

Layer 3
VRRP
Firewall
Layer 3 NAT / Firewall
Firewall
Firewall Firewall Firewall Firewall
Tree / Star

Production Cells
Ring Ring
Switch
Access

Switch Switch Switch Switch

Line

Figure 4 - Medium Network Reference Architecture

The above medium reference network architecture will build on the key concepts of the small reference network architecture.
Each of the key considerations listed below are in addition or an expansion of the considerations covered in the small
network architecture

16 Version 2, 03 -2023
 The OT Networking Reference Architecture – Categorizing Future-Ready OT Networks

4.2.1 Key Considerations for Medium Networks

1) Topology
• Ring Networks
o As the complexity and importance of the network continues to grow – the presence of ring topologies
should be more prevalent to further improve the resiliency of communication. This can now be seen in
Cell 1-2 in addition to Cell 1-1.
o The implementation of a Backbone / Aggregation ring will reduce potential impact of a network device failure
by further distributing the Production Cell level connections amongst several Production Backbone devices.

2) Redundancy
As the complexity and importance of the network continues to grow – various methodologies of redundant
communication will become necessary to maintain reliable operation.
• As mentioned above in ring networks, additional layers of redundancy will improve the overall resiliency of
the network by distributing the connections amongst several devices. This will reduce the impact on various
production cells that could be caused by a device failure.
• At this scale it is absolutely recommended to implement RSTP or redundant routing protocols to achieve:
o Redundant enterprise connections via IT/OT firewalls
o Redundant core routers in the Backbone / Aggregation layer
o Redundant ring network in the Backbone / Aggregation layer
• At this scale it is recommended to implement:
o Redundant connections from the Production Cell layer to the Backbone / Aggregation layer

3) Recommended Software
• Network Management software is recommended in medium scale networks to efficiently manage and monitor the
larger number of devices. The NMS system will also allow tracking of the user(s) changes to network configuration.
• RADIUS Server is recommended as it eliminates sharing of passwords and accounts. User actions are uniquely
identifiable and password policies are consistent across all devices.
• Syslog Server is recommended as it provides centralized logging and accountability across all devices. This can
greatly reduce troubleshooting time should any errors occur.

4) Additional Security Considerations

• The interconnection between OT and IT is now handled via redundant pairs of firewalls; we are no longer
sharing ownership and accountability of any firewalls. Demilitarized Zones (DMZ) exist on both the IT and OT
side to allow for a very precise set of firewall rules that restrict communication. There should not be any
communication from either the IT or OT side that is initialized outside of the DMZ-to-DMZ.
• Next-Generation Firewalls (NGFW) can be deployed as the IT / OT Firewalls for their advanced threat detections
and Deep Packet Inspection (DPI) capabilities. It is possible to also deploy N -GFWs at the cell firewall level so that
the management of all firewalls can be centralized. The cost analysis should value a simplified set of procedures,
a reduction in user errors and training requirements, in addition to the advanced capabilities gained.
• Backups should be validated, and disaster recovery procedures should be part of yearly exercises to reduce
the financial impacts of network outages.
• Signing up for firmware update alerts and performing them in a timely manner becomes increasingly important
to reduce risk.
• Replacement hardware should be stocked and kept up to date with firmware updates

Version 2, 03 -2023 17
The OT Networking Reference Architecture – Categorizing Future-Ready OT Networks

4.3 Large Networks – Connecting Multiple Facilities

As operations expand to multiple facilities across a campus, the scale, capacity, and service required of the OT
network will correspondingly increase. A large sized network can be classified using the following criteria:
• More than 250 end point devices
• Critical operation where network disruption will result in significant financial impact
• Multiple facilities across a campus, each consisting of multiple production lines, or a large-size single production facility

Enterprise Network
. WAN External WAN Core External Internet
Core

Industrial DMZ

Firewall Router Firewall

IT

Servers

IT / OT IT / OT
Firewalls Firewalls

HMI and SCADA

Industrial DMZ

Historian and Reports

Network Services
Secure Remote Access
Network Management
SIEM and IDS/IPS Servers Backbone Backbone +2

Routers Redundant
Routers
Connections

Production Backbone
Distribution

Redundant Connections

Aggregation 1 Aggregation 2

Switch Switch Switch Switch

Ring Ring

Switch Switch Switch Switch Switch Switch

Industrial
WLAN

Production Network
L3 Redundant
Connections
Redundant
Connections

Cell 1-1 Cell 1-2 Cell 1-3 Cell 1-4 Cell 2-1 Cell 2-2

Layer 3
VRRP Firewall
Firewall Firewall

Firewalls

Production Cells
Firewall Tree / Star
Firewall Firewall
Ring
Access

Switch Switch Switch

Line
Ring
Line

Tree / Star

Figure 5 – Large Network Reference Architecture

The above large reference network architecture will build on the key concepts of the small and medium reference network
architecture. Each of the key considerations listed below are in addition or an expansion of the considerations covered in the
small and medium network architecture.

4.3.1 Key Considerations for Large Networks

1) Topology
• Ring Networks
o As the complexity and importance of the network continues to grow – ring topologies should be implemented
in multiple layers in the production backbone. This is now seen as Aggregation rings 1 and 2.
o The corresponding Aggregation rings will individually consolidate communication for several production cells
and lines. The Aggregation rings should then connect to a redundant router ring to further increase resiliency
and reduce interdependencies of production areas.
18 Version 2, 03 -2023

 The OT Networking Reference Architecture – Components of an IT/OT Network

2) Redundancy
As the complexity and importance of the network continues to grow – various methodologies of redundant
communication will become necessary to maintain reliable operation.
• As mentioned above in ring networks, additional layers of redundancy will improve the overall resiliency of
the network by distributing the connections amongst several devices. This will reduce the impact on various
production cells that could be caused by a device failure.
• Several routers located in single or multiple Backbone rings manage the network traffic load efficiently across
many IP subnets. This also provides a higher level of redundancy in layer 3 communications.
• At this scale it is absolutely recommended to implement RSTP and redundant routing protocols to achieve:
o Redundant enterprise connections via IT/OT firewalls
o Highly redundant core routers in the Backbone layer
o Multiple redundant rings in the Aggregation layer
• At this scale it is recommended to implement:
o Redundant connections from the Aggregation layer to the Backbone layer
o Redundant connections from the Production Cell layer to the Aggregation layer

3) Recommended Software
An IDS should be part of the cybersecurity program. To be able to detect intrusions, network traffic is cloned by
using a managed switch’s port mirroring function. This mirrored traffic is ingested by detectors that can use
traffic signatures and anomaly-based detection to raise alarms. This information can be centralized in a controller
to allow for multi-site capabilities. From an architecture perspective, this system should operate on a physically
separate network in very large networks. It is common to have the central controller as the bridge to the rest of
the enterprise network.

4) Additional Security Considerations

When discussing the security considerations of a large network, our OT cybersecurity program must be thorough
and precisely follow the procedures laid out in the specific framework that we have elected to follow. To assist
with this matter, Siemens has created an Industrial Network Security Architecture white paper that is linked in
the references of this document. It provides a breakdown of the considerations in five separate categories.
• Network Segmentation is performed using zone-based concepts and such zones are defined by the
needs of Safety Instrumented Systems and Functions (SIS & SIF) requirements, real-time communication
requirements, functional relationships, and risks
• Asset and Network Management functions are defined by policies and procedures. These functions are
augmented using tools such as Network Management Systems like SINEC NMS.
• Network Protection functions require advanced authentication, authorization, and accounting. RADIUS,
certificate-based authentication, and logging is prevalent. A SIEM receives information from these
systems, as well as from IDS/IPS solutions, and centralized firewall management systems.
• Secure Remote Access functions are integrated into the network protection scheme and can leverage
concepts such as Zero Trust to provide an advanced secure solution.
• Training and Awareness is integral to the functioning of the OT cybersecurity program. A continuous
training plan ensures staff knowledge is up to date. Threats against the integrity of the system, as well
as how to respond to cybersecurity incidents are documented, well understood, and regularly updated.

Version 2, 03 -2023 19
 The OT Networking Reference Architecture – Components of an IT/OT Network

CERTIFY YOUR PLACE IN THE FUTURE OF NETWORKING

Industrial Networks
Education
In our Industrial Networks Education program, you will learn the fundamental skills of planning,
implementation monitoring and securing of wired and wireless industrial data networks and
connect them to a corporate network. For those with a high degree of networking knowledge,
we also offer advanced classes to take you to the next level in your skill set.
usa.siemens.com/yourcertification

20 Version 2, 03 -2023
 The OT Networking Reference Architecture – Five Next Steps

5. Next Steps
Now that the key considerations for each OT network size has been provided, do you feel confident that you can build a
Future-Ready Network?

While this document provides an overview of the possible considerations to make, determining how each consideration matters
to your specific application requires plenty of research. It is recommended that technical decision makers understand the
different technologies listed in this document. A great resource to start is with our 269-page technical overview of industrial
networking concepts.

If you are interested in the cybersecurity solutions mentioned in this document and would like to further delve into cybersecurity
considerations for large networks, please read through our Industrial Networking Security Architecture document.

If you’d like some help with making sure your network is Future-Ready, our experts are ready to schedule a free consultation.
Please fill out our form.

Version 2, 03 -2023 21
The OT Networking Reference Architecture – Resources

Resources
Further Readings

White Paper: Industrial Network Security Architecture

www.usa.siemens.com/network-security-architecture-paper

IT/OT Collaboration
www.usa.siemens.com/it-ot-collaboration-article

Network Concepts for Factory Automation

www.usa.siemens.com/automation-network-concepts

Future ready networks

www.usa.siemens.com/future-ready-networks

Industrial Cybersecurity Solutions

www.usa.siemens.com/network-security

Industrial Wireless
www.usa.siemens.com/industrial-wireless

Siemens can help you!

Free consultation request
www.usa.siemens.com/cybersecurity-consultation

Siemens Professional Services

www.usa.siemens.com/networking-services

22 Version 2, 03 -2023
Legal Manufacturer This document contains a general description of available technical options
Siemens Industry, Inc. only, and its effectiveness will be subject to specific variables including field
100 Technology Drive conditions and project parameters. Siemens does not make representations,
Alpharetta, GA 30005 warranties, or assurances as to the accuracy or completeness of the content
United States of America contained herein. Siemens reserves the right to modify the technology and
product specifications in its sole discretion without advance notice.
Telephone: +1 (800) 241-4453
usa.siemens.com/industrial-networks
Order No. NTBR-REFARC-0323

© 03.2023, Siemens Industry, Inc.

Version 2, 03 -2023 23