GR1010686B - Integrated incident management system using health and security devices - application thereof to critical infrastructures - Google Patents

Abstract

The invention refers to an integrated event management system in critical infrastructures consisting of a set of subsystems organized in containers in a loosely-coupled architecture with the support of a message management subsystem (8). The subsystems (1), (2) and (3) compose the system arrangement (hardware, software and container management platform) on which the containers of the proposed system operate. The main objective of the system is to support operational procedures for the protection of critical infrastructures. Therefore, there are two main categories of users: a) Security guards in critical areas who carry wearables (such as smart watches and monocular glasses 4b and 11a respectively) and portable devices (such as smart phones 4a) and are supported by the subsystems BSafe (4) and SafeAssist (11), b) the control center operators who are responsible both for monitoring the communication systems, including the facility's computer systems and networks, and for guiding the guards in incident management. Control center operators are supported by GSafe (5), decision making (6) and CCAssist (10 and binocular glasses 10a) subsystems. The Privacy/Security subsystem (9 and 9a) has a global role in the integrated system. Finally, the data model of the system is composed of two more subsystems, that of the database (7) and the message management subsystem (8) through which all the above subsystems communicate with each other, but also gain secure access to the database data. The integrated system of the present invention connects both wired and wirelessly (portable and wearable devices) to support the critical infrastructure security business plan.

Description

DESCRIPTION

Integrated incident management system in critical infrastructures using health and safety devices

The present invention relates to an integrated incident management system in critical infrastructures that causes the restructuring and improvement of existing incident management processes, as well as the design of new advanced ones.

Existing incident management systems in critical infrastructures are based on the projection of a large number of videos on a small number of screens in an operations center where the operator monitors the videos on a 24-hour basis and tries to identify any incident. In the event that the operator determines that there is an incident at a certain point, he communicates by radio with the patrol to rush to the scene of the incident and gives them verbal instructions. The patrol transmits information via radio to the operator in order to form a picture of the incident and decide on the final actions.

The purpose of the present invention is to eliminate the disadvantages of previous systems. Specifically, the proposed system supports multi-parametric authentication of guards, where a combination of biometric data is made with equipment elements, which are resistant to physical cloning, as well as the production of personalized encryption keys that can be varied over time, providing more security to the critical infrastructure, as the operator cooperates with authenticated guards whose wearable devices can connect and transmit information only after being properly authenticated. The fact that the authentication is based on the technology of physically non-clonable functions makes the process completely reliable. Additionally, continuous monitoring of the health and location of guards in the field during patrol is supported, which helps the operator at the center to know the status and location of the guards at all times and to be able to manage both the human material and the incidents more effectively, sending the closest and best physically fit guard to the point where the incident occurs. The control center operator will no longer detect incidents by eye through the video surveillance of dozens of cameras, but will be supported by a system that will detect incidents in the infrastructure areas and immediately inform the operator. The operator will be able to send the appropriate guards to the scene of the incident, giving them instructions, augmenting their perception of the incident, and requesting them to send video via their wearable devices, simultaneously augmenting their own view of the incident for the best possible visual surveillance of the incident. Finally, the system, upon closing the incident management, can provide a full report on the incident.

The system that is the subject of the present invention consists of a set of subsystems with clear and distinct functionality. These subsystems are distinguished in Figure 1 which illustrates the physical layering of the integrated system, as well as its interface with the external devices carried by the users. A loosely coupled architecture was adopted whose subsystems are organized in containers. Subsystems (1), (2) and (3) of Figure 1 compose the system arrangement (hardware, software and container management platform of the server respectively) on which the containers of the proposed system operate. The main objective of the system is to support the operational procedures for the protection of critical infrastructures. Therefore, there are two main categories of users: a) The guards in the critical areas who carry wearables (4b and 11a) and mobile devices (4a) and are supported by the BSafe (4) and SafeAssist (11) subsystems, b) the control center operators who are responsible for both monitoring the communication systems, including the computer systems and networks of the facility, and for guiding the guards during incident management. The control center operators are supported by the GSafe (5), decision-making (6) and CCAssist (10 and 10a) subsystems. The Privacy/Security subsystem (9 and 9a) has a universal role in the integrated system. Finally, the data model of the system is composed of two more subsystems, that of the database (7) and the message management subsystem (8) through which all the above subsystems communicate with each other, but also gain secure access to the database data. The system will initially be described by briefly listing the operation of each subsystem.

The BSafe subsystem (4) is a device for collecting, processing and storing data via smart wearable devices (4b, such as a smartwatch), smart portable devices (4a, such as a mobile phone) carried by the guard and location beacons (4b) installed at various points in the critical infrastructure. The data collected by the system are: 1) health (such as heart rate, oxygen level, etc.), 2) location, 3) movement (such as accelerometer data), and 4) danger alerts activated by the guard for emergency situations. The data is transmitted encrypted, while it is stored in the system database (7) in its normal form and anonymized. The guard has access to his data in real time via the smart portable device.

The SafeAssist subsystem (11 and 11a) is an Augmented Reality system that supports critical infrastructure guards in the field. Specifically, it integrates and disseminates to the monocular glasses (10a) the data (such as time, space and type of incident, as well as a list of incident management instructions) that have been generated by the other subsystems, augmenting the reality perceived by the guard in the field. At the same time, the guard is given the opportunity to transmit, through the integrated camera of the imaging glasses, a video stream from the point where the incident takes place to the control center in real time. And all this without requiring the guard's attention to be distracted from performing his duties. Finally, intuitive interfaces are provided with guards to minimize problems related to concentration and distraction of guards in the field. At the same time, the possibility of guard interaction through voice instructions or special hand gestures is also supported.

The CCAssist subsystem (10 and 10a) is a Mixed Reality system provided to the critical infrastructure control center operator. CCAssist integrates and disseminates to the binoculars (10a) the data generated by the other subsystems, augmenting the reality perceived by the control center operator and supporting the supervision of activities taking place in the field in real time. The user interface is based on the visualization of multiple screens with which the user interacts (data charts, incident data, incident management instructions). The CCAssist subsystem focuses on incident data management and is used by the operator in addition to the GSafe subsystem. More specifically, CC-Assist completes: (a) the data related to each incident, (b) the video streams (from cameras and the SafeAssist subsystem), (c) the photos related to the incident, and (d) the list of incident handling instructions.

The GSafe subsystem (5) is a device for collecting, processing, storing and disseminating information that enters the system and/or is generated by it. In real time, it receives and processes health and movement data of the guards transmitted by BSafe, data generated by the decision-making subsystem (6) when an incident is detected, as well as video streams received from the devices of the SafeAssist subsystem (11a) that the guards carry during the management of an incident. In addition, it receives from the privacy and security subsystem the result of the authentication process of the guards who take up a shift in the critical infrastructure and allows or does not allow, depending on the result, the connection of their wearable devices to the system. GSafe disseminates in real time the status and location of each guard, notifications generated by the decision-making subsystem (6) upon incident detection or directly by the guards (danger alerts) only to authorized users. It also stores part of the above data in the database of the integrated system. Finally, it supports the management of system users and the matching of guards with wearable devices.

The decision-making subsystem (6) receives and processes in real time as input visual data streams from the closed critical infrastructure monitoring system (12), as well as data from the wearable devices used by the guards in the field via BSafe (4, 4a, 4b). In the event that any of the events specified by the facility managers is detected/identified, the corresponding information (category, criticality classification, response instructions, image with the necessary spatial information) is disseminated to the control center operator via the GSafe subsystem. Finally, via GSafe, feedback is provided by the center operator regarding the correctness of the identification of the incidents by the subsystem, the knowledge base of the subsystem is updated, the subsystem is retrained and evolves, increasing the performance of the system.

The privacy and security subsystem (9 and 9a) supports the authentication and authorization of users and subsystems, access control to system resources and encryption of sensitive data. For the authentication, authorization and access control of the subsystems of the proposed system, the security mechanisms of the message management subsystem (8) are used. The authentication of the infrastructure users, as well as the generation of encryption keys, are based on the external PUF (Physical Unclonable Function) subsystem (9a), the operation of which is based on physical unclonable functions. The PUF subsystem (9a) is composed of an optical PUF reader, a fingerprint sensor and a microcontroller to perform basic functions. To perform the authentication of a guard, two procedures take place: a) the initial registration of the credentials of each guard and b) the verification of the credentials each time he takes over a shift. Each guard receives his own unique card (optical PUF) with a distinct PUF_CardID, which, after being associated with the unique GuardID, is registered in the database. The guard then enters his fingerprint into the sensor of the PUF subsystem, places the PUF card and types a code into the reader. These elements are combined and a unique response to the PUF is generated. This unique response from the guard is encoded in binary format of 128bit-1Kbit length and together with the extracted helper data and the PUF_CardID compose the message that is transmitted encrypted from the subsystem (9a) to the subsystem (9) via the message management subsystem (8). The message, after its content (binary response and helper data) is decrypted by the subsystem (9), is assigned to the database by associating it with the GuardID. In this way, the phase of registering the guard's credentials is completed. Each time the guard takes on a shift, the above process is repeated and the binary response received is compared with the one assigned to the database for the same guard (either with an error detection code or by examining the hamming distance) and if the proximity of the two responses is considered tolerable, the guard is authenticated, otherwise he is rejected. In any case, the result is disseminated to the center operator by transferring the result to the GSafe subsystem. When authentication is successful, the received response is combined with other data related to the guard, such as data from the wearable devices that the guard carries during his shift, to produce the symmetric keys that will be used to encrypt sensitive data transmitted by wearable and mobile devices with a symmetric cryptosystem (e.g., Advanced Encryption Standard).

The message management (8) and data storage (7) subsystems support the system data model. A message management arrangement was adopted to organize the data model, through which the subsystems can securely transfer data between themselves, both inside and outside the system. Data is transferred by exchanging messages using the appropriate protocol of the message management arrangement. Data is exchanged between a pair of subsystems, where one subsystem has the role of producer and the other has the role of consumer. The structure of the queue and the exchanger is used to transfer data from the producer to the consumer. The producer subsystem deposits one or more messages in the exchanger that manages the queue of the consumer subsystem, which then stores the message in this queue. The producer continues without waiting for a delivery confirmation from the exchanger. The consumer at any time receives from the queue the messages delivered for him by the producer. In this way, asynchronous producer/consumer communication is achieved. In Figure 1, all the interfaces between the subsystems are distinguished. In Table 1, the data classes managed by the system as a whole are shown. The database of the system (7) is relational.

The above integrated system was integrated into the security systems of a Marina, causing the restructuring of existing incident management procedures, as well as the design of new ones. The following choices were made for the development of the system: a) a server with sufficient computing and storage power, as shown in Figure 1, b) use of the containerization platform offered by Docker technology c) use of the Docker compose tool, d) use of Vuzix M4000 monocular glasses and Microsoft HoloLens 2 binocular glasses, e) Smart Watch TicWatch Pro 3 Ultra, LinTech Bluetooth Low Energy 4.2 Smart Beacon and mobile phones with Android 12, f) use of a semiconductor laser with visible emission (Optical PUFs), camera, thermal camera, microcontroller (Arduino, raspberry pi), PUF cards, g) use of the RabbitMQ Erlang/OTP message manager, and the Advanced Message Queueing Protocol (AMQP) protocol and h) development of the processing devices on Linux Ubuntu with Python, implementing the PyTorch, Scikit-Learn, Pandas, Numpy, OpenCV, SciPy, SQLalchemy libraries, use of OpenSSL and MySQL.

Q server together with the subsystem (9a) was placed in the local control center of the Marina and functioned complementary to the existing information system used by the Marina security services. The system was connected to the Marina subnet to which the Marina cameras are also connected, which are the sources of the video streams to the decision-making subsystem (6). The wearable and portable equipment of the Marina guards and operators were connected to the Marina's protected wireless network.

The system was then trained and tested in real time in handling ten incidents requested by the Marina security manager with remarkable success. These incidents are: a) bar breach, b) fence breach (security incident), c) detection of a suspicious object (security incident), d) aggressive behavior of a person (security incident), e) fire/smoke detection near boats (security incident), f) guard fall (safety incident), g) visitor fall (security incident), h) abnormal heart function in a guard (safety incident), i) detection of prolonged presence of a person/s near boats at night (security incident), j) breach of the main door of the Central Administration Building (security incident). A typical incident is the detection of a fence breach, where an unknown person or persons climbed over the fence of the facility and entered the Marina area. Before the integration of the proposed system, the operational plan included the following procedure:

1. The operator at the control center visually analyzes the information from the installation's cameras and identifies the violation,

2. an announcement is made over the loudspeakers to remove themselves from the area, and 3. a guard is ordered to approach the area and request their removal,

4; if their removal is not possible, the competent authority is notified upon order of the Marina Security Officer.

By integrating the proposed system into the Marina's security architecture, the incident response process in the operational plan is transformed as follows:

1. The decision-making system (6), examining in real time video streams from the Marina's cameras (12), detects the intrusion of a person or persons and notifies the control center operator via GSafe (5) or CCAssist (10a).

2. The operator processes, through GSafe (5), the health data of the guards and their location transmitted by the portable (4a) and wearable devices (4b) of BSafe (4).

3. The operator, via Very High Frequency (VHF), asks the guard monitoring the area to approach the point of violation while maintaining a safe distance.

4. The operator asks the guard to activate real-time video capture through the incident imaging glasses (11a).

5. The guard communicates via VHF with the control center.

6. The system (6) sends a list of actions in text form to the display glasses (10a and 11a).

7. The operator asks the guard via VHF to monitor the intruder at a safe distance, maintain visual contact with him and inform the center every time he changes direction.

8. If their removal is not possible, the competent authority is notified upon order of the Marina Security Officer.

Table 1: Data Classes appearing in the subsystems of the proposed system

Claims (8)

1. An integrated critical infrastructure incident management system consisting of: a) the BSafe subsystem (4), b) the SafeAssist subsystem (11 and 11a), c) the CCAssist subsystem (10 and 10a), d) the decision-making subsystem (6), e) the GSafe subsystem (5), f) the privacy and security subsystem (9 and 9a), g) the message management (8) and data storage subsystems (7), characterized by the hardware (1), system software (2) and container support platform (3) layers of the server that compose the system arrangement on which the above subsystems are integrated in the system of the present invention as a loosely-coupled architecture with the above subsystems being organized separately in containers. 2. The integrated critical infrastructure incident management system as referred to in claim 1 consists of the BSafe subsystem (4) which is characterized by an array of smart wearable devices (4b, such as a smartwatch), smart portable devices (4a, such as a mobile phone) carried by the guard and location beacons (4b) installed at various points in the critical infrastructure, for the encrypted collection, processing, dissemination to the GSafe subsystems and decision-making, and storage of part of the health and location data of the guards in the field in the database. 3. The integrated incident management system in critical infrastructures according to claims 1 and 2 consists of the decision-making subsystem (6) which is characterized by the real-time reception and processing of video streams from the closed critical infrastructure monitoring system (12), as well as health and position data of the guards in the field from the BSafe subsystem (4, 4a, 4b), the dissemination of a multitude of data (category, criticality classification, response instructions, image with the necessary spatial information) to the control center operator and the guard in the field through the GSafe/CCAssist subsystems (5/10 and 10a) and SafeAssist (11 and 11a) respectively in the event that any of the incidents for which the decision-making subsystem has been trained is detected/recognized, the storage of this data in the database (7) and the possibility of its retraining and development through the updating its knowledge base following feedback on the accuracy of incident identification provided by the center operator via GSafe (5). 4. The integrated incident management system in critical infrastructures according to claims 1 and 3 consists of the SafeAssist subsystem (11 and 11a) which is characterized by the real-time augmentation of the guard's reality in the field through the integration and diffusion in monocular glasses (11a) of the data generated by the decision-making subsystem, and the possibility of transmitting in real time, through the integrated camera of the glasses, a video stream from the point where the incident takes place to the GSafe (5) and CCAssist subsystems (10 and 10a). 5. The integrated incident management system in critical infrastructures according to claims 1, 2, 3 and 4 consists of the GSafe subsystem (5) which is characterized by the real-time reception and processing of health and movement data of the guards transmitted by the BSafe (4), data generated by the decision-making subsystem (6) when an incident is detected, video streams from the devices of the SafeAssist subsystem (11a) carried by the guards, the result of the authentication process of the guards who take over the shift in the critical infrastructure by the privacy and security subsystem, the real-time dissemination only to authorized users of the status and location of each guard, the notifications generated by the decision-making subsystem (6) when an incident is detected or directly by the guards (danger alerts) and the storage of part of the above data in the database of the integrated system. 6. The integrated critical infrastructure incident management system according to claims 1, 3, 4 and 5 consists of the CCAssist subsystem (10 and 10a) which is characterized by the real-time augmentation of the reality perceived by the center operator through the integration and diffusion in binocular glasses (10a) of the data generated by the decision-making subsystems (6), GSafe (5) and SafeAssist (11 and 11a). 7. The integrated critical infrastructure incident management system according to claims 1, 2 and 5 consists of the Privacy and Security subsystem (9 and 9a) which is characterized by authentication, authorization and access control procedures for all subsystems to the system resources that configure the security mechanisms of the message management subsystem (8), authentication procedures for the infrastructure users, as well as the generation of BSafe encryption keys (4 and 4a), which are based on the external PUF subsystem (9a, Physical Unclonable Function), the operation of which is based on physical unclonable functions and is composed of an optical PUF reader, a fingerprint sensor and a microcontroller to support the authentication process. 8. The integrated critical infrastructure incident management system according to claims 1, 2, 3, 5 and 7 consists of the database subsystem which is characterized by a relational database (7) which together with the message management subsystem (8) compose the data model of the system.