1. Introduction
Modern factory automation systems are based on distributed architectures in which communication networks are exploited to connect industrial controllers and field devices, like sensors and actuators, located throughout a plant, possibly at considerable distances [
1]. During operation, the components of these systems execute tasks (e.g. data acquisition and analysis, control, data fusion, etc.), which have to be appropriately scheduled to achieve the expected plant behavior [
2,
3]. In this scenario, task execution times represent key variables, mainly determined by two components. The first one is concerned with the local (internal to a device) elaboration, which typically comprises the CPU time that is necessary to execute the protocol stack code, as well as the latencies due to data buffering and queuing. The second one is related to the communication times necessary for transmitting data to the components distributed over the plant. Thus, for example, the instance of a control task in a networked system comprises, not only the time necessary to execute the control algorithm on a local controller, but also both the time necessary to acquire the feedback data, obtained from a message sent by remote sensors connected to the network, and the time to send control commands to remote actuators, again via the network [
4]. Communication times play a critical role in task scheduling, since their values can be substantially higher than those due to local elaboration and, more importantly, they may be strongly variable. A typical cause of such variability is represented by a failure in delivering a packet, which necessarily requires a retransmission: in this case, the actual task communication time results at least doubled with respect to the time necessary to a single transmission attempt. This problem is even more evident when wireless networks are used, since they are inherently error prone [
5,
6].
The usage of wireless technology is of strategic importance to enable the Industrial Internet of Things (IIoT), which claims for the connection of industrial equipment (things), making them reachable from anywhere, at any time, using any service [
7]. IIoT is part of the Industry 4.0 plan, which is fostering a significant transformation within the field of industrial automation and control. Some of the leading issues of this paradigm are represented by the need of interconnection, integration, and interoperability among possibly heterogeneous industrial systems [
8].
The IEEE 802.11 Wireless LAN [
9], universally known as Wi-Fi, already demonstrated its ability to cope with the tight requirements of the industrial field of application [
10]. Moreover, it is easy to use, flexible, and able to provide the required interoperability [
11]. Additionally, Wi-Fi has an attractive feature, namely the Multi–Rate Support (MRS), which revealed to be definitely advantageous in environments subjected to frequent fluctuations of the Signal–to–Noise Ratio (SNR), such as the industrial ones [
12]. Indeed, the use of suitable Rate Adaptation (RA) Algorithms (RAAs) allows to dynamically change the packet transmission rate to better cope with the channel status. Roughly speaking, the most common RAAs select low transmission rates for low values of the SNR, since lower rates adopt more robust modulation schemes.
Several RAAs have been designed over the years and some of them were specifically conceived for industrial applications [
12,
13]. These latter ones have demonstrated their effectiveness, in that they are able to rapidly adapt to the SNR behavior, and revealed to be able to ensure low communication times, along with less variability and limited packet loss ratios. However, the computational complexity and execution times of such algorithms have not been effectively addressed yet. This is an aspect of considerable importance which may have a serious impact on the overall tasks execution times, especially when devices with limited resources, such as industrial sensors/actuators are used.
This paper moves from the above considerations and focuses on the computational aspects of rate adaptation algorithms, as well as of protocol stacks, adopted for industrial real–time applications, as will be described in the following.
2. Related Work and Contribution
The actual impact of task execution times on the performance of distributed automation systems has only rarely been addressed by the scientific literature. More commonly, theoretical analyses and/or experimental measurements on prototype systems have been carried out. In this respect, an essential review of related works, mostly concerned with Wi-Fi systems, is provided in the following.
In [
14], the authors provide an interesting survey that addresses the timeliness of the IEEE 802.11 MAC mechanisms. Another survey is given by [
10], where the authors focus more specifically on the deployment of Wi-Fi for time critical and reliable industrial control applications. In [
15], the authors investigate the adoption of frequency–domain based access protocols to improve the performance of legacy Wi-Fi. In [
16], the features of IEEE 802.11ax are analyzed in detail to achieve efficient scheduling (i.e., an effective assignment of users to subcarriers in order to improve network throughput). The design and practical implementations of real–time networks based, to different extents, on IEEE 802.11, are provided in [
17,
18], respectively. In [
17], the authors propose a new MAC protocol based on a Time Division Multiple Access (TDMA) technique that relies on the IEEE 802.11 Distributed Coordination Function (DCF), capable of providing deterministic performance. A prototype implementation of the protocol actually demonstrated its effectiveness. In [
18], a modification of the IEEE 802.11 physical layer is introduced in order to achieve very low packet delivery times, which were comparable with those of the wired counterparts. In [
19], the authors describe WIA–FA, a communication system for factory automation based on Wi-Fi, standardized by the International Electrotechnical Commission (IEC) [
20], along with some interesting case studies and examples of application to real manufacturing plants. Real–time industrial multimedia applications, particularly video streaming, are addressed in [
21], where the authors show the capability of Wi-Fi to handle such a kind of traffic. A valuable example of calculation of protocol stack execution times is given in [
22]. Here, the authors not only describe a possible measurement procedure, but also provide the results of some experimental sessions aimed at evaluating the execution times of both the TCP/IP and UDP/IP protocol stacks for a given embedded device.
The aforementioned studies represent interesting and valuable contributions that, however, do not address the real behavior of devices, typically determined by non-negligible computational requirements and times that may reveal of paramount importance in the real-time industrial communication context [
23]. Focusing on Wi-Fi, this aspect might be exacerbated by the adoption of RAAs that, undeniably, require an undefined amount of time for the execution of their code. Hence, there is the risk that the benefits achieved with the adoption of MRS on the communication side, in particular the reduced packet delivery times, are vanished by the possible increase of the local elaboration times, caused by the execution of the rate adaptation algorithms. These considerations fostered the preparation of this paper whose main contribution can be summarized, as follows:
Implementation of rate adaptation algorithms for industrial real-time applications and experimental assessment of their execution times;
Evaluation of the impact of rate adaptation algorithms on the protocol stacks execution times and on the communication times;
Performance assessment of Wi-Fi–based industrial communication networks with rate adaptation algorithms enabled.
In detail, the paper is structured, as follows.
Section 3 provides some essential background about real–time industrial networks.
Section 4 presents the rate adaptation algorithms conceived for real-time communication, to give the foundations to evaluate the impact of their execution times.
Section 5 describes the implementation of the algorithms and introduces the setup adopted for the experimental sessions.
Section 6 presents the tests that were executed and discusses the obtained results. Finally,
Section 7 concludes the paper and outlines some future research activities.
3. Background: Real–time Industrial Networks
Real-time industrial networks have been extensively addressed by the scientific literature [
1,
24]. They are communication systems that are specifically designed to effectively interconnect industrial devices, such as controllers, sensors, and actuators. This type of communication implies the timely transmission of limited amounts of data (some tens of bytes, typically) carrying, for example, set–points values or process data variables. Industrial networks have often to comply with tight timing requirements due to the peculiarity of the applications for which they are deployed. Indeed, they have to grant for the precise scheduling of periodic messages, as well as to ensure that aperiodic messages are delivered within a-priori specified deadlines. Notably, packet transmission periods and deadlines can be as low as some hundreds of microseconds. The introduction of wireless networks in this challenging context needed careful analyses, due to the well known problems that may affect wireless communication, such as fading, path loss, and collisions, which inevitably lead to delays in packet delivery or, even worse, to packet losses [
25,
26]. This topic has been addressed by several research activities in the past years, that led to the latest state of play in which wireless networks are widespread in the industrial scenario, particularly for process automation applications [
27,
28]. Moreover, their deployment is expected to grow significantly in the next future [
29].
Suitable protocols have been adopted in order for both types of industrial networks (i.e., wired and wireless) in order to fulfill the aforementioned requirements. The most widespread are based on techniques, like TDMA (Time Division Multiple Access), Master–Slave, and Producer–Consumer [
30,
31]. In general, these protocols introduce an ordered access scheme to the transmission medium that allows for properly handling the message flows generated by the applications. For example, in a Master–Slave configuration, if a temperature sensor (the Slave) has to transmit its value with a given periodicity to a controller (the Master), then it will be periodically granted by the Master with the right to access the network to transmit the temperature value. In this way, the desired timing behavior can be achieved. Additionally, with such protocols packet collisions are definitely reduced, since stations are not allowed to initiate a transmission autonomously and, hence, to transmit contemporaneously.
4. Rate Adaptation Algorithms for Wi–Fi–Based Real Time Industrial Networks
The MRS feature of IEEE 802.11 can be deployed to dynamically change the transmission rate of a packet. Indeed, the adopted rate can be selected from a set of available rates before each transmission.
Definition 1 (Set of available rates)
. The set of the available rates, R, for the transmission of a packet is defined, as follows. Basically, the operation of a RAAs consists in the selection of an ordered set of rates, called Retransmission Chain, to be used for the transmission of a packet.
Definition 2 (Retransmission chain)
. Consider the transmission of a packet for which a maximum of N attempts can be carried out. The Retransmission Chain, RC, is defined asmeaning that the first transmission attempt is scheduled at rate , the second at rate , the N-th at rate 4.1. ARF: Automatic Rate Fallback
Automatic Rate Fallback (ARF) [
32] is one of the most popular rate adaptation strategies for Wi-Fi, designed for general purpose applications.
The behavior of ARF can be briefly described, as follows. Consider a set
R of available data rates. Starting from the generic rate
, a station implementing ARF moves to the lower rate
after
K consecutive failed transmission attempts. Conversely, the rate is increased to
after
N successful attempts. Also, if a failure occurs at the first transmission attempt after the rate has been increased, then the rate is immediately lowered down to
(this feature is known as "Probing transmission"). Typical values of the parameters
K and
N for the ARF algorithm are 2 and 10, respectively [
33].
Unfortunately, ARF demonstrated to be rather ineffective for real–time industrial communication. However, its behavior allowed to derive two new algorithms able to reduce both the number of retransmissions and the delivery time of a packet, that revealed suitable for the industrial scenario. These are, namely, Static retransmission rate ARF (SARF) and Fast rate reduction ARF (FARF) [
12].
4.2. SARF: Static Retransmission Rate ARF
SARF [
12] is a variant of ARF in which each retransmission attempt is carried out at rate
, i.e., the lowest rate in the set
R. In practice, if the first transmission attempt fails, then for all the subsequent retransmissions we will have
with
. However, a successful retransmission carried out at the lowest rate is not considered as an event that resets the consecutive failures counter in order to grant SARF with the same sensitivity to channel variations of ARF. More precisely, only the original (first) transmission attempt is considered, and if
K of these attempts fail then the rate will be lowered anyway, even if they were interleaved by one or more successful retransmissions. SARF ensures a very high success probability in packet transmission, since the lowest transmission rate uses the most robust modulation. This allows to limit the number of retransmissions, with the consequent reduction of the randomness. The pseudo-code of SARF is described by Algorithm 1.
Algorithm 1 SARF Algorithm |
- 1:
/* Initialization - 2:
▹ Number of consecutive transmission failures - 3:
▹ Number of consecutive transmission successes - 4:
▹ Available rates - 5:
▹ Select the lowest transmission rate in set R - 6:
▹ Set transmission rate to the lowest value - 7:
/* End Initialization - 8:
while true do - 9:
if Request of a new transmission then - 10:
if Previous attempt was failed then - 11:
Perform transmission attempt at the lowest rate - 12:
else - 13:
Perform transmission attempt at the current rate - 14:
if Success then - 15:
▹ Reset # of consecutive failures - 16:
▹ Increase counter of consecutive successes - 17:
if then ▹ If success threshold reached - 18:
if then ▹ If highest rate not reached - 19:
; - 20:
▹ Increase rate - 21:
end if - 22:
end if - 23:
else - 24:
/* (If failure) - 25:
▹ Reset # of consecutive successes - 26:
▹ Increase consecutive failures - 27:
if then ▹ If failure threshold reached - 28:
if then ▹ If lowest rate not reached - 29:
- 30:
▹ Decrease rate - 31:
end if - 32:
end if - 33:
end if - 34:
end if - 35:
end if - 36:
end while
|
4.3. FARF: Fast Rate Reduction ARF
FARF is an algorithm that represents a customization of ARF in which and . In practice, just a single failed attempt is sufficient to set the new transmission rate to the lowest value in the set R, (i.e., ). Conversely, the increasing of the rate follows the same rules of ARF, and happens after N consecutive successful attempts. Analogously to SARF, with . From a certain point of view, FARF is a more conservative algorithm than SARF, since it intrinsically assumes that a failure is an event that requires to maintain low rates for a certain number of subsequent packet transmissions. The pseudo-code for FARF is described by Algorithm 2.
Algorithm 2 FARF Algorithm |
- 1:
/* Same code as SARF Algorithm (lines 3 to 7) - 2:
while true do - 3:
if Request of a new transmission then - 4:
Perform transmission attempt - 5:
if Success then - 6:
/* Same code as SARF Algorithm (lines 16 to 22) - 7:
else - 8:
/* (If failure) - 9:
▹ Reset # of consecutive successes - 10:
▹ Set transmission rate to the lowest value - 11:
end if - 12:
end if - 13:
end while
|
4.4. RSIN: Rate Selection for Industrial Networks
RSIN is a rate adaptation algorithm specifically conceived for the industrial environment, proposed in [
13], and based on two fundamental assumptions. Specifically, (i) a station wishing to send a frame (of a given length) has to be aware of the SNR perceived by the receiver and (ii) the station has to know the relationship between SNR and packet error rate (PER) for all the rates of the transmission set
R. For each packet to be transmitted, let
D be its deadline. Subsequently, the outputs of RSIN are the number of retransmissions (
), the retransmission chain (
) and the residual error rate probability for the transmission of that packet in a time less than
D. More formally, RSIN can be described by a constrained optimization problem.
Definition 3 (RSIN constrained optimization problem). For each packet transmission let be:
- D
the transmission deadline;
- s
the signal–to–noise ratio;
- l
the payload of the packet to transmit; and,
the maximum number of retransmission attempts.
Let us define:
- N
the number of allowed retransmission attempts;
the residual packet transmission error probability, given N retransmissions at rates ; and,
the transmission time.
Subsequently, the constrained optimization problem is to find an optimal number of retransmissions attempts , and a retransmission chain , which minimize the residual packet transmission error probability, while ensuring that the deadline is not missed. That is RSIN is described by Algorithms 3 and 4. One of the core sections of Algorithm 3 is represented by lines 18–20. At this stage, a vector of retransmission chains is built, and each element stores the chain ensuring the lowest transmission error probability for that specific number of retransmission attempts, ranging from 1 to
. This is achieved with the invocation of Algorithm 4. Subsequently, lines 22–33 select the optimal retransmission chain among the
chains of the
vector, as the one associated with the lowest transmission error probability. It is worth noting, in Algorithm 4, how the selection of the retransmission chain is handled in case more retransmission chains ensure the same transmission error probability. As can be seen (lines 9–13), in this case Algorithm 4 selects the retransmission chain that has the lower transmission time. A further assumption has been adopted to limit the execution time of RSIN. Specifically, the construction of the array of retransmission chains by Algorithm 4, has been carried out under the condition
that limits the retransmission chains to be considered.
Algorithm 3 RSIN Algorithm |
- 1:
N
| ▹ number of retransmission attempts |
- 2:
/* Inputs */
|
- 3:
| ▹ max number of retransmission attempts |
- 4:
| ▹ set of available rates |
- 5:
D
| ▹ Packet deadline |
- 6:
s
| ▹ SNR value |
- 7:
/* Definitions */
|
- 8:
| ▹ packet transmission time |
- 9:
| ▹ residual error probability for the optimal rate sequence |
- 10:
/* Definitions – Vectors of size */
|
- 11:
| ▹: residual error probability when |
- 12:
| ▹: transmission time when |
- 13:
| ▹ Vector of rate chains. |
- 14:
/* The i-th element, , contains the retransmission chain of length that ensures the lowest transmission error probability*/
|
- 15:
/* Outputs */
|
- 16:
| ▹ optimal number of retransmissions |
- 17:
| ▹ optimal retransmission chain |
- 18:
for to do
|
- 19:
Running Algorithm 4.
|
- 20:
end for
|
- 21:
/* Calculation of and */
|
- 22:
|
- 23:
|
- 24:
|
- 25:
for to do
|
- 26:
if then
|
- 27:
|
- 28:
|
- 29:
|
- 30:
else
|
- 31:
Do nothing
|
- 32:
end if
|
- 33:
end for
|
Algorithm 4: Selection of the retransmission chain ensuring the lowest transmission error probability for each value of N |
- 1:
repeat - 2:
Generate a retransmission chain of size i, - 3:
calculate - 4:
if then - 5:
if then - 6:
- 7:
- 8:
- 9:
else if then - 10:
if then - 11:
- 12:
- 13:
- 14:
else - 15:
Do Nothing - 16:
end if - 17:
else - 18:
Do Nothing - 19:
end if - 20:
else - 21:
Continue - 22:
end if - 23:
until there are sequences of rates , of length , meeting condition 2
|
Finally, in [
34] a variant of RSIN, namely RSIN with Estimation (RSIN–E), has been introduced. RSIN–E has been devised to address the cases in which the SNR is not made available by the receivers. Thus, a learning algorithm has been designed to provide an estimation of the SNR, as better detailed in [
34]. The learning algorithm is executed cyclically, with update period
. Clearly, the introduction of a further algorithm has the effect of increasing the execution time of RSIN–E, as will be investigated in the following of the paper. In particular, the lower the update period, the higher the impact on the execution time.
5. Algorithms Implementation and Experimental Set–Up
The RAAs described in the previous section, with the exclusion of ARF, have been practically implemented on a Personal Computer (Dell Optiplex 960) equipped with an Ubuntu Long Term Support Linux distribution, based on Linux kernel version 4.1.0-040100-lowlatency. The PC is equipped with two commercial off-the-shelf Wireless Network Interface Cards (WNICs), namely TP-LINK model TL-WN851ND, in order to implement an IEEE 802.11n network, as described in
Figure 1. The used WNICs are based on the AR9287 chip, which adopts the open–source
ath9k driver. This choice, along with adequate modifications to the
mac80211 module, following the technique outlined in [
35], allowed to implement and test the designed rate adaptation algorithms.
Clearly, the use of a PC with two WNICs does not reflect a real industrial network set-up at all. However, the first set of experiments we executed do not need to be carried out on an actual industrial network, as it will be better detailed in the following. Indeed, the performed analysis is aimed at evaluating the execution times of the RAAs and their impact on the MAC layer execution times. Such times are related to the algorithms implementation and do not depend on the network configuration. Moreover, having the two WNICs installed on the same PC revealed definitely advantageous, because it allowed for exploiting a unique clock source for the experiments, ensuring high accuracy of the results, since there was no need to synchronize the two boards.
For the specific case of the RSIN algorithm, we developed two different implementations. In the first one, the constrained optimization problem (Equation (
1)) is run before the transmission of each packet. In the second implementation, referred to as “RSIN Light” (RSIN–L) in the following, a set including all of the optimal retransmission chains, for each possible SNR value, is determined at an early initialization phase, and is stored within a look-up table. Thus, the selection of the retransmission chain, which happens just before a single packet transmission, simply involves access to that table. This strategy clearly reduces the execution time of the algorithm. However, because the constrained optimization problem is not solved dynamically, RSIN–L implicitly assumes that (i) the relationship between SNR and PER does not vary over time, (ii) the deadline is the same for all the transmitted packets, (iii) the packets have fixed length. The actual assessment of such conditions depends on the communication channel status as well as on the specific application.
The choice of using a Personal Computer for the implementation of the algorithms, as well as of the subsequent measurement campaign, has been suggested by the flexibility ensured by such device. Indeed, it has been possible to adequately modify the MAC layer protocol stack, as well as to enable/disable and tune the rate adaptation algorithms during the execution of the tests. However, in real applications different devices may be used, since factory automation systems make use of Programmable Logic Controllers (PLCs), sensors/actuators and, possibly, Soft PLCs. All of these devices, typically, do not grant full access to the MAC layer functions and, hence, could have not been used for our purposes. Nevertheless, and more importantly, rate adaptation algorithms do exist and are (or may be) used by such kind of devices. In this respect, we believe the results of the practical experiments we carried out, which will be discussed in the next Section, represent a meaningful indication of the impact of rate adaptation algorithms on the performance of industrial devices.
The RAAs have been implemented within the context of the Linux
mac80211 framework. In agreement with this approach, we developed the set of required functions for each algorithm. A test application has been subsequently designed, in which the data exchange described in
Figure 2 was triggered periodically, thus resembling a typical industrial polling sequence. A Master–Slave relationship has been set-up where one board (the Master), configured as Access Point, sends a request packet to the other one, configured as Station (the Slave), which responds with a data packet. Each transaction ends when the Master receives the packet with the requested data. The complete list of the settings used for the experiments is summarized in
Table 1. They have been selected to comply with the most common features/requirements of Wi-Fi based real-time industrial networks and their relevant rate adaptation algorithms, as described in [
25,
36,
37,
38].
We introduced adequate timestamp points within the relevant parts of the protocol stack code in order to measure the RAAs execution times.
Figure 3 provides a representation of the timings related to the transmission of a packet. As can be seen, the packet is initialized at time
and delivered to the MAC layer at time
. The execution of the RAA takes place in the interval
. The packet is eventually delivered to the physical layer at time
. Notably, the reference clock shared by the two WNICs allowed to precisely timestamp the events concerned with code execution.
7. Conclusions and Future Directions of Research
In this paper, we addressed the impact of rate adaptation algorithms on the task execution times of distributed automation systems, based on Wi-Fi. The analysis has been carried out gradually, via practical experiments as well as by simulations. We first evaluated the execution times of the algorithms, then we determined their impact on the MAC protocol stack and, eventually, on the task execution times. The obtained results clearly indicate that the adoption of rate adaptation algorithms in real-time industrial communication systems should be encouraged, since they bring considerable benefits in terms of timeliness and reliability with a limited impact on the task execution times and, hence, on industrial control scheduling policies.
Future activities may be envisaged in two directions. The first one involves the development of new rate adaptation algorithms for the industrial scenario, and/or the enhancement of the existing ones. To this regard, the behavior of RSIN could be made more effective, in order to further limit its execution time. For example, some Retransmission Chains that are not able to meet a deadline could be a priori excluded, reducing in such a way the number of iterations performed in Algorithm 3.
The second direction refers to the execution of more comprehensive experimental assessments. Particularly, measurements should be carried out on real industrial devices, in order to achieve a complete characterization of their behavior.