[go: up one dir, main page]

CN116827883B - SCADA system offline data caching method, device and storage medium - Google Patents

SCADA system offline data caching method, device and storage medium Download PDF

Info

Publication number
CN116827883B
CN116827883B CN202311080948.8A CN202311080948A CN116827883B CN 116827883 B CN116827883 B CN 116827883B CN 202311080948 A CN202311080948 A CN 202311080948A CN 116827883 B CN116827883 B CN 116827883B
Authority
CN
China
Prior art keywords
data
data blocks
data block
buffer area
blocks
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202311080948.8A
Other languages
Chinese (zh)
Other versions
CN116827883A (en
Inventor
李美松
包卿
王兴华
薛立俊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mingdu Zhiyun Zhejiang Technology Co Ltd
Original Assignee
Mingdu Zhiyun Zhejiang Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mingdu Zhiyun Zhejiang Technology Co Ltd filed Critical Mingdu Zhiyun Zhejiang Technology Co Ltd
Priority to CN202311080948.8A priority Critical patent/CN116827883B/en
Publication of CN116827883A publication Critical patent/CN116827883A/en
Application granted granted Critical
Publication of CN116827883B publication Critical patent/CN116827883B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/50Queue scheduling
    • H04L47/62Queue scheduling characterised by scheduling criteria
    • H04L47/625Queue scheduling characterised by scheduling criteria for service slots or service orders
    • H04L47/6275Queue scheduling characterised by scheduling criteria for service slots or service orders based on priority
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/10Flow control; Congestion control
    • H04L47/43Assembling or disassembling of packets, e.g. segmentation and reassembly [SAR]
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/50Queue scheduling
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L47/00Traffic control in data switching networks
    • H04L47/50Queue scheduling
    • H04L47/56Queue scheduling implementing delay-aware scheduling
    • H04L47/562Attaching a time tag to queues
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The application discloses an offline data caching method, device and storage medium of a SCADA (supervisory control and data acquisition) system, which are characterized in that the method comprises the steps of dividing unsent acquired data into a plurality of data blocks, storing the data blocks in a first cache area, and selecting corresponding complementary transmission modes according to requirements, wherein the first complementary transmission modes are used for sequentially capturing data blocks with the largest serial numbers in the data blocks which are not completed and are subjected to complementary transmission from the first cache area after network connection is restored, and the second complementary transmission modes are used for sequentially capturing data blocks corresponding to serial numbers positioned in the middle of serial numbers in continuous sections of each serial number and forming a data block queue in a second cache area, and transmitting the data blocks after network restoration; the data buffer and the data transmission supplementing modes are adjusted according to the network conditions by utilizing different data transmission supplementing modes, so that the influence of network interruption on data transmission of the system is reduced, and the stability of the SCADA system is improved.

Description

SCADA system offline data caching method, device and storage medium
Technical Field
The present application relates to the field of network technologies, and in particular, to a method, an apparatus, and a storage medium for offline data caching of a SCADA system.
Background
Because of the unstable network communication environment, the SCADA system often has the condition of data transmission failure in the process of remote connection and transmission, when the related data under the condition is subjected to the complementary transmission, the unstable network state can also cause the network connection to be interrupted for a plurality of times in the complementary transmission process, even the condition of complete network interruption occurs, so that the data to be complementary transmitted cannot be completely transmitted to the SCADA server in the required time, the normal work of the SCADA system is affected, in addition, the existing offline data caching and complementary transmission mode is single, and the data caching and complementary transmission modes cannot be adjusted according to the network condition.
Disclosure of Invention
The application provides an offline data caching method of a SCADA system, which aims at the defects in the prior art and comprises the following steps:
s1, dividing unsent acquired data into a plurality of data blocks, and storing the data blocks in a first buffer area, wherein the data blocks are provided with serial numbers distributed according to the data acquisition time sequence;
s2, acquiring a set data retransmission mode, and if the data retransmission mode is a first retransmission mode, sequentially capturing data blocks with the largest serial numbers in the data blocks which are not completed and are transmitted from a first buffer area after network connection is restored;
and S3, if the data block sequence number distribution section is in the second transmission supplementing mode, acquiring the sequence number distribution section of each data block stored in the first buffer zone, sequentially capturing the data blocks corresponding to the sequence numbers positioned in the middle of the sequence number continuous sections, forming a data block queue in the second buffer zone, and sequentially capturing the data blocks in the second buffer zone according to the sequence of the data block queue for transmission after network connection is restored.
Preferably, the step S1 includes: carrying out format processing and endowing unique serial numbers to each divided data block, wherein the data block format comprises serial numbers, historical data information and check codes, and the serial numbers of the data blocks are larger as the historical time stamps of the first data record contained in the corresponding data block are later; and if the historical data information contains alarm information, adding a priority identifier into the corresponding data block format.
Preferably, the step S2 includes: after monitoring that the network connection is restored, acquiring a set data supplementary transmission mode, searching whether a data block with a priority identifier exists in a first buffer area or not if the data block with the priority identifier exists in the first buffer area, and if the data block with the priority identifier exists, sequentially capturing the data block with the priority identifier and carrying out supplementary transmission on the data block to an SCADA server through a network; after the reissuing of all the data blocks with the priority identifiers is completed, the data blocks with the largest serial numbers in the data blocks which are not reissued are sequentially grabbed from the first buffer area and are transmitted.
Preferably, the step S3 includes:
if the supplemented option is a second supplemented transmission mode, searching whether the data blocks with the priority identifiers exist in the first buffer area, and if so, capturing the data blocks with the priority identifiers in sequence and supplementing the data blocks to the SCADA server through a network;
after the reissuing of all the data blocks with the priority identifiers is completed, acquiring the serial number distribution section of each remaining data block stored in the first buffer area;
after capturing the data blocks corresponding to the serial numbers positioned in the middle of the serial number continuous sections to the second buffer area, sequentially and repeatedly identifying each serial number continuous section corresponding to the rest data blocks in the first buffer area, capturing the data blocks corresponding to the serial numbers positioned in the middle of each serial number continuous section to the second buffer area until all the data blocks in the first buffer area are captured, and forming a data block queue in the second buffer area, wherein the data block queues are arranged according to the adding sequence of the data blocks;
and sequentially capturing the data blocks in the second buffer area according to the sequence of the data block queues, and reissuing the data blocks to the SCADA server through a network.
Preferably, the step S3 further includes: if a plurality of serial number continuous sections corresponding to the rest data blocks in the first buffer area exist, after a plurality of data blocks corresponding to the serial numbers in the middle of each serial number continuous section are identified, the data blocks are sequentially grabbed into the second buffer area according to the sequence of the serial numbers of the plurality of data blocks.
Preferably, the offline data caching method further comprises the following steps: monitoring the sending state of the real-time acquisition data in the process of the historical data supplementary transmission, acquiring a current data supplementary transmission mode if the network congestion feature occurs, switching the current data supplementary transmission mode to a second supplementary transmission mode if the current data supplementary transmission mode is in the first supplementary transmission mode, and reducing the supplementary transmission frequency of the data blocks in the second supplementary transmission mode if the current data supplementary transmission mode is already in the second supplementary transmission mode.
The application also discloses a data acquisition device, which comprises: the buffer memory module is used for dividing the unsent collected data into a plurality of data blocks and storing the data blocks in the first buffer memory area, wherein the data blocks are provided with serial numbers distributed according to the time sequence of data collection; the first supplementing module is used for sequentially grabbing data blocks with the largest serial numbers in the data blocks which are not completed and reissued from the first buffer area after network connection is restored to be transmitted; and the second supplementing module is used for sequentially grabbing the data blocks corresponding to the serial numbers positioned in the middle of the serial number continuous sections to the second buffer area to form a data block queue after acquiring the serial number distribution sections of each data block stored in the first buffer area, and sequentially grabbing the data blocks in the second buffer area according to the sequence of the data block queue for transmission after recovering network connection.
Preferably, the buffer module is further configured to perform format processing and assign a unique serial number to each divided data block, where the data block format includes a serial number, historical data information and a check code, and the serial number of each data block is greater as a historical timestamp of a first data record included in a corresponding data block is later; and if the historical data information contains alarm information, adding a priority identifier into the corresponding data block format.
Preferably, the first supplementing module is configured to acquire a set data supplementing mode after monitoring that the network connection is restored, if the supplemented option is the first supplementing mode, searching whether a data block with a priority identifier exists in the first buffer area, and if so, sequentially grabbing the data block with the priority identifier to supplement the data block to the SCADA server through the network; after the reissuing of all the data blocks with the priority identifiers is completed, the data blocks with the largest serial numbers in the data blocks which are not reissued are sequentially grabbed from the first buffer area and are transmitted.
The application also discloses a computer readable storage medium storing a computer program which when executed by a processor implements the steps of any of the methods described above.
According to the off-line data caching method, the off-line data caching device and the storage medium of the SCADA system, through setting a plurality of data complementary transmission modes, the SCADA system can adjust the complementary transmission sequence of acquired data according to requirements, and therefore the SCADA server can acquire important information as much as possible within limited time. The first transmission supplementing mode can sequentially grasp and send the data block with the largest serial number in the data blocks which are not completed and supplemented from the first buffer area, and the latest data collected in the mode can be firstly transmitted to the SCADA server; the second complementary transmission mode can sequentially grasp data blocks corresponding to the serial numbers positioned at the middle positions of the serial number continuous sections to form a data block queue in the second buffer area for transmission after network recovery, and the mode transmits the data in each acquired time area through intervals, so that the system can receive the data information in different time distribution areas as soon as possible in a short time, obtain the data information as wide as possible in a short time after network recovery, acquire the data change trend in the complementary data as soon as possible, and avoid the problem that the data change trend cannot be acquired from the data after the complementary transmission is completed due to the fact that the network is interrupted again before the complete data supplementation is completed when the first complementary transmission mode is adopted.
Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application and do not constitute a limitation on the application. In the drawings:
fig. 1 is a flow chart of an offline data caching method of a SCADA system according to an embodiment of the disclosure.
Fig. 2 is a schematic flow chart of step S1 according to an embodiment of the application.
Fig. 3 is a schematic diagram of a structure of a cut data block according to an embodiment of the present application.
Fig. 4 is a schematic flow chart of step S2 according to an embodiment of the application.
Fig. 5 is a schematic flow chart of step S3 according to an embodiment of the application.
Fig. 6 is a schematic structural diagram of a data acquisition device according to an embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present application. It will be apparent that the described embodiments are some, but not all, embodiments of the application. All other embodiments, which can be made by a person skilled in the art without creative efforts, based on the described embodiments of the present application fall within the protection scope of the present application.
Unless defined otherwise, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this application belongs. The terms "first," "second," and the like in the description and in the claims, are not used for any order, quantity, or importance, but are used for distinguishing between different elements. Likewise, the terms "a" or "an" and the like do not denote a limitation of quantity, but rather denote the presence of at least one.
The method for caching the offline data of the SCADA system disclosed by the embodiment can be used for various monitoring systems to cache the acquired data in an offline state, wait for network recovery and then send the data in a preconfigured mode, and as shown in the attached figure 1, the method specifically comprises the following steps:
step S1, the unsent collected data is divided into a plurality of data blocks and stored in a first buffer area, wherein the data blocks are provided with serial numbers distributed according to the time sequence of data collection.
Specifically, the SCADA system for offline caching and online transmission supplement comprises a hardware remote acquisition IO unit and an SCADA server at the site side. The remote acquisition IO unit is used for acquiring operation data of the industrial equipment, caching the acquired data when the remote acquisition IO unit is in offline communication with the remote SCADA server, and receiving and processing the acquired data and receiving the offline cached data which is fed back by the remote acquisition IO unit when the communication is recovered.
In this embodiment, as shown in fig. 2, the step S1 may specifically include the following.
And S11, after the connection between the remote acquisition IO unit and the SCADA server is lost, the acquired field data is stored into a local storage area.
And S12, capturing historical data from the local storage area by the remote acquisition IO unit, dividing the historical data into a plurality of data blocks, storing the data blocks in the first cache area, marking each data block, and endowing the data blocks with a unique identifier.
Step S12 may further include capturing a historical data packet of a corresponding size from the local storage area at regular time according to a set rule, and dividing the historical data packet into a plurality of data blocks of a corresponding size by a data dividing function, where the data blocks include a historical timestamp of the collection variable and a historical data value.
In the present embodiment, step S12 may further include the following.
Step S121, capturing the data stream with the set size from the local storage area storing the non-transmitted collected data at the set time interval.
Step S122, obtaining a historical network interruption parameter in a latest preset time span, obtaining a corresponding data block value according to the historical network interruption parameter, and dividing the data stream according to the data block value to form a plurality of data blocks, wherein the historical network interruption parameter comprises network connection accumulation time, network interruption times or network connection shortest duration.
The step S122 may include the following.
Querying a historical network outage parameter within a latest preset time span from a device network status record, wherein the historical network outage parameter comprises network connection accumulated time, network outage times or network connection shortest duration.
And inquiring the corresponding size of the data block to be cut from a data cutting configuration table according to the historical network interruption parameters, wherein a plurality of optional data block volumes and corresponding network interruption parameter ranges are recorded in the data cutting configuration table.
And cutting the data flow according to the volume of the data block obtained by inquiry, and equally dividing the data flow into a plurality of data blocks with corresponding volumes.
In this embodiment, step S122 may further include: and after the connection with the server is interrupted, calculating the duration of the current network connection, and if the duration of the current network connection is smaller than the shortest network connection time corresponding to the current data block segmentation volume, inquiring and acquiring the data block segmentation volume corresponding to the shortest network connection time which is smaller than or equal to the duration of the current network connection from a data cutting configuration table as a follow-up data block volume group, and selecting the volume maximum value in the follow-up data block volume group as the updated data block segmentation volume.
Further, acquiring a shortest duration time for keeping continuous network connection within a set time span, and if the shortest duration time is greater than a shortest network connection time corresponding to a current data block segmentation volume, searching and acquiring a corresponding data block volume from a data segmentation configuration table again according to the shortest duration time for updating the current data block segmentation volume.
And after carrying out format processing and endowing unique identifiers on each divided data block, storing the data blocks into a first buffer area, and carrying out complementary transmission after the network is recovered.
Wherein, this step S1 further includes: and carrying out format processing and assigning a unique serial number to each divided data block, wherein the data block format comprises serial numbers, historical data information and check codes, and the serial numbers of the data blocks are larger as the historical time stamps of the first data record contained in the corresponding data block are later. In this embodiment, if the history data information is found to include alarm information, a priority identifier is added to the corresponding data block format.
Specifically, each data block format is represented by a custom structure, as shown in fig. 3, where the data block format may include a sequence number, historical data information, and a check code, and the identifier is configured to calculate, by using a hash algorithm, a historical timestamp of a first data record included in the data block, the check code, and a hash value sequence number obtained by calculating the number of data records included in the data block, where the sequence number of each data block is greater as the historical timestamp of the first data record included in the corresponding data block is later.
The sequence number indicates the position of the chunk of data throughout the data stream and may be a sequence of ordered data beginning at zero. When data communication is interrupted, the confirmed sequence numbers can be used for determining the correct data packet position and retransmitting, and the size relationship of each sequence number is determined by the historical time stamp of the first data record of each data block, and the larger the sequence number of the data block, the later the time when the data block is collected is proved. Wherein the check code adds an extra bit to each data block to make the sum equal to a fixed value. If a data block is altered or corrupted during transmission, its check code will not match, thereby enabling the recipient, i.e., the SCADA system server, to determine whether the data block has been altered or lost.
Further, the sequence number may be further configured to be calculated by a hash algorithm according to the historical timestamp of the first data record included in the data block, the check code, and the number of data records included in the data block, where the historical timestamp of the first data record included in the corresponding data block is larger as the historical timestamp is later.
And S2, acquiring a set data retransmission mode, and if the data retransmission mode is the first retransmission mode, sequentially capturing data blocks with the largest serial numbers in the data blocks which are not completed and are transmitted from the first buffer area after network connection is restored.
Specifically, after the connection recovery between the remote acquisition IO unit and the SCADA server is monitored, the remote acquisition IO unit grabs corresponding data blocks from the first cache region according to a set updating rule and sends the data blocks to the SCADA server. And capturing the data block with the largest serial number in the first buffer area, and then, after receiving the confirmation number corresponding to the serial number of the data block fed back after the successful confirmation of the receipt by the SCADA server, sequentially capturing the data block with the next serial number, and then, retransmitting the data block to the SCADA server.
In the first transmission supplementing mode, the SCADA server grabs the data blocks to be supplemented in the first buffer area according to the sequence from the large sequence number to the small sequence number, namely grabs the data blocks according to the sequence from the near sequence number to the far sequence number, and in this case, the SCADE server preferentially receives the latest data, so that the SCADE server can make other decisions or adjustments according to the information contained in the latest data.
As shown in fig. 4, this step S2 may include the following.
Step S21, after the network connection is monitored to be restored, a set data transmission supplementing mode is obtained, if the data transmission supplementing option is a first transmission supplementing mode, whether data blocks with priority identifiers exist in a first buffer area is searched, and if the data blocks with the priority identifiers exist, the data blocks with the priority identifiers are sequentially grabbed and are subjected to transmission supplementing to the SCADA server through a network.
Specifically, the system may include a priority mechanism, when the collected data includes time-efficient data and urgent data, the remote collecting IO unit needs to transmit the collected data to the SCADA server in a short time, otherwise, if the information included in the data exceeds a certain time, the information included in the data will lose the transmission value and negatively affect the existing engineering, the collected data may be classified in level at this time, after the data block is cut, a priority identifier is added in the data block with a higher priority level, and when the network connection is restored, the collected data is transmitted preferentially, for example, when the data includes fault alarm information or early warning information, the data block is marked and transmitted preferentially, so that the SCADA system can receive the data at the fastest possible speed and perform subsequent adjustment according to the data, and smooth operation of the system is ensured.
Step S22, after completing the reissuing of all the data blocks with the priority identification, sequentially capturing the data blocks with the largest serial numbers in the data blocks which are not reissued and have not completed reissuing from the first buffer area for transmission.
Step S23, if the second transmission supplementing mode is adopted, acquiring serial number distribution sections of each data block stored in the first buffer, successively grabbing data blocks corresponding to serial numbers positioned in the middle of the serial number continuous sections, forming a data block queue in the second buffer, and after network connection is restored, sequentially grabbing data blocks in the second buffer according to the sequence of the data block queue for transmission.
Specifically, in the second transmission supplementing mode, the remote acquisition IO unit performs interval grabbing and sending on the data to be supplemented, so that the SCADA system can receive data points in different time periods of the data stream to be supplemented in the same time, and when the network is interrupted for a long time or completely terminated under the condition that the network state is extremely unstable, the SCADA system can perform estimation analysis on the overall condition of the data stream according to the received data point information in different time periods. The first complementary transmission mode focuses on the reference value of the latest data, and the second complementary transmission mode focuses on the reference value of the data and the regular searching of the data in the whole time range.
As shown in fig. 5, this step S3 may further include the following.
Step S31, if the complement option is the second complement transmission mode, searching whether the data block with the priority identification exists in the first buffer area, if so, capturing the data block with the priority identification in sequence and complement the SCADA server through the network.
Step S32, after finishing the reissuing of all the data blocks with the priority identification, obtaining the serial number distribution sections of the rest data blocks stored in the first buffer area.
Step S33, after capturing the data blocks corresponding to the serial numbers positioned in the middle of the serial number continuous sections to the second buffer area, sequentially and repeatedly identifying each serial number continuous section corresponding to the rest data blocks in the first buffer area, capturing the data blocks corresponding to the serial numbers positioned in the middle of each serial number continuous section to the second buffer area until all the data blocks in the first buffer area are captured, and forming a data block queue in the second buffer area, wherein the data block queues are arranged according to the adding sequence of the data blocks.
Specifically, after each time a data block in the middle position of a serial number continuous section is grabbed, the continuous section is naturally divided into two small continuous sections with the same length, the serial number grabbed next time is located in the middle of each of the two small sections and is symmetrically distributed, the data block in the middle position of the serial number continuous section is continuously acquired by adopting a second complementary transmission mode, the whole data stream is continuously and naturally divided into relatively uniform different areas, so that each time the grabbed data block is distributed in each area of the whole data stream, and information contained in the data stream is furthest transmitted. Further, if the number of the remaining data blocks included in a certain continuous sector in the first buffer area is even, any one of the two sequence numbers located in the middle of the continuous sector is randomly grabbed.
In this embodiment, the step S33 may further include: if a plurality of serial number continuous sections corresponding to the rest data blocks in the first buffer area exist, after a plurality of data blocks corresponding to the serial numbers in the middle of each serial number continuous section are identified, the data blocks are sequentially grabbed into the second buffer area according to the sequence of the serial numbers of the plurality of data blocks.
Specifically, when there are a plurality of continuous sections in the first buffer area, each data block may be sequentially stored in the second buffer area according to the sequence from the big sequence to the small sequence.
When the number of the data blocks in the middle positions corresponding to the remaining continuous sections is excessive due to continuous grabbing, other sequence arrangement methods can be adopted to store the grabbed data blocks into the second buffer area, and the specific operation steps are as follows: judging whether the number of the data blocks grabbed at the time exceeds a preset number, if yes, arranging the data blocks grabbed at the time according to a preset sequence number arrangement rule, and storing the data blocks in a second buffer area, wherein the preset sequence rule is configured to divide the sequence numbers of the data blocks by a specific number and arrange the data blocks according to the sequence from big to small of remainder, and when the remainder is the same, arranging the data blocks according to the size of the sequence numbers, by the method, the interval of sequence numbers of adjacent data blocks stored in the second buffer area each time can be enlarged, and the data flow area range spanned by the data blocks in a certain time is further enlarged.
And step S34, sequentially capturing the data blocks in the second buffer area according to the sequence of the data block queues, and reissuing the data blocks to the SCADA server through a network.
In this embodiment, after completing data reissue, after receiving a data block that is reissued by a remote acquisition IO unit, the SCADA server generates a corresponding acknowledgement number according to a unique identifier of the data block and feeds back the acknowledgement number to the remote acquisition IO unit, extracts a historical data value and a historical timestamp in the data block, and inserts the historical data value and the historical timestamp into a historical trend database for archiving and storing.
After receiving the data block reissued by the remote acquisition IO unit, the SCADA server extracts the serial number value of the data block, adds a corresponding timestamp, generates a confirmation number, saves the confirmation number to a local database, and feeds the confirmation number back to the remote acquisition IO unit for sending the data block; and acquiring a historical time stamp and a historical data value of each piece of collected variable historical data contained in the data block, reformatting according to a format of a historical trend database of the SCADA server, and then sequentially inserting the reformatted historical time stamp and historical data value into a time interval corresponding to the historical trend database.
Monitoring the sending state of the real-time acquisition data in the process of the historical data supplementary transmission, acquiring a current data supplementary transmission mode if the network congestion feature occurs, switching the current data supplementary transmission mode to a second supplementary transmission mode if the current data supplementary transmission mode is in the first supplementary transmission mode, and reducing the supplementary transmission frequency of the data blocks in the second supplementary transmission mode if the current data supplementary transmission mode is already in the second supplementary transmission mode.
In another embodiment, an autonomous regulation and control mechanism can be established according to the current network condition, so that the remote acquisition IO unit can autonomously select the first supplementary transmission mode or the second supplementary transmission mode when the network connection is recovered, manual regulation and control are not needed, the flexibility and the simplicity of the remote acquisition IO unit are improved, the manpower and material resources are saved, real-time monitoring is performed in the supplementary transmission process, and the supplementary transmission mode is automatically switched according to the monitoring condition. Which may include the following in particular.
Step S101, obtaining the network interruption times in a preset adjacent period, judging whether the network interruption times exceed a first preset value when the network connection is recovered, and selecting a second complementary transmission mode if the network interruption times exceed the first preset value.
When the number of network interruption times is too large and is higher than a set value, the network is proved to be in an extremely unstable state, the possibility of interruption again is high, the time spent for the full transmission of the data to be supplemented to the SCADA server exceeds the expectation, and the second supplementing mode is needed to enable the data received by the server in a short time to cover the information of sufficient breadth.
Step S102, if not, judging whether the data quantity to be reissued in the first candidate area exceeds a second preset value, if yes, selecting a second reissue mode, and if not, selecting a first reissue mode.
If the amount of data to be reissued in the first candidate area is too large, the time taken for the data to be reissued to be transmitted to the SCADA server is still longer than expected, and a second reissue mode is selected.
When the reissue flow starts, the remote acquisition IO unit monitors the current network congestion state in real time, and when congestion information is acquired, the current data reissue mode is switched or the data block reissue frequency is reduced so as to slow down the current network congestion state, and the data blocks are smoothly transmitted to the SCADA server.
According to the off-line data caching method of the SCADA system, through setting a plurality of data complementary transmission modes, the SCADA system can adjust the complementary transmission sequence of the acquired data according to the needs, so that the SCADA server can acquire important information with reference value as much as possible within a limited time. The first transmission supplementing mode can sequentially grasp and send the data block with the largest serial number in the data blocks which are not completed and supplemented from the first buffer area, and the latest data collected in the mode is firstly transmitted to the SCADA server; and the second transmission supplementing mode can sequentially grab the data blocks corresponding to the serial numbers positioned in the middle positions of the serial number continuous sections into the second buffer area to form a data block queue, and the data blocks are transmitted after the network is recovered. By the two data complement modes, the influence of network interruption on system data acquisition and application is reduced, and the stability of the SCADA system is improved.
In another embodiment, as shown in fig. 6, a data acquisition device includes a buffer module 1, a first supplementing module 2, and a second supplementing module 3. The buffer module 1 is configured to divide the unsent collected data into a plurality of data blocks, and store the data blocks in the first buffer area, where the data blocks have serial numbers allocated according to a data collection time sequence. And the first supplementing module 2 is used for sequentially grabbing data blocks with the largest serial numbers in the data blocks which are not completed and reissued from the first buffer area after the network connection is restored and sending the data blocks. And the second supplementing module 3 is used for sequentially grabbing the data blocks corresponding to the serial numbers positioned in the middle of the serial number continuous sections to the second buffer area to form a data block queue after acquiring the serial number distribution sections of each data block stored in the first buffer area, and sequentially grabbing the data blocks in the second buffer area according to the sequence of the data block queue for transmission after recovering network connection.
In this embodiment, the buffer module 1 may be further configured to perform format processing and assign a unique serial number to each of the divided data blocks, where the data block format includes a serial number, historical data information, and a check code, and the serial number of each data block is greater as a historical timestamp of a first data record included in a corresponding data block is later; and if the historical data information contains alarm information, adding a priority identifier into the corresponding data block format.
In this embodiment, the first supplementing module 2 may be further configured to obtain a set data supplementing mode after monitoring that the network connection is restored, if the supplemented option is the first supplementing mode, search whether a data block with a priority identifier exists in the first buffer area, and if so, sequentially grab the data block with the priority identifier and supplement the data block to the SCADA server through the network; after the reissuing of all the data blocks with the priority identifiers is completed, the data blocks with the largest serial numbers in the data blocks which are not reissued are sequentially grabbed from the first buffer area and are transmitted.
In this embodiment, the second supplementing module 3 may be further configured to search whether the data block with the priority identifier exists in the first buffer, and if so, sequentially grab the data block with the priority identifier and reissue the data block to the SCADA server through the network. And after the reissuing of all the data blocks with the priority identifiers is completed, acquiring the sequence number distribution section of each remaining data block stored in the first buffer area. And after grabbing the data blocks corresponding to the serial numbers positioned in the middle of the serial number continuous sections to the second buffer zone, sequentially and repeatedly identifying each serial number continuous section corresponding to the rest data blocks in the first buffer zone, grabbing the data blocks corresponding to the serial numbers positioned in the middle of each serial number continuous section to the second buffer zone until all the data blocks in the first buffer zone are grabbed, and forming a data block queue in the second buffer zone, wherein the data block queues are arranged according to the adding sequence of the data blocks. And sequentially capturing the data blocks in the second buffer area according to the sequence of the data block queues, and reissuing the data blocks to the SCADA server through a network.
It should be noted that, in the present description, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different manner from other embodiments, and identical and similar parts between the embodiments are all enough to refer to each other. For the data acquisition device disclosed in the embodiment, the description is relatively simple because the data acquisition device corresponds to the offline data caching method disclosed in the embodiment, and relevant parts only need to be referred to in the method section.
The offline data caching method described above, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the present application may implement all or part of the flow of the method of the above embodiment, or may be implemented by instructing related hardware by a computer program, where the computer program may be stored in a computer readable storage medium, and the computer program may implement the steps of each of the offline data caching method embodiments described above when executed by a processor. Wherein the computer program comprises computer program code which may be in source code form, object code form, executable file or some intermediate form etc. The computer readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer Memory, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), an electrical carrier signal, a telecommunications signal, a software distribution medium, and so forth. It should be noted that the computer readable medium contains content that can be appropriately scaled according to the requirements of jurisdictions in which such content is subject to legislation and patent practice, such as in certain jurisdictions in which such content is subject to legislation and patent practice, the computer readable medium does not include electrical carrier signals and telecommunication signals.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the application.
In summary, the foregoing description is only of the preferred embodiments of the present application, and all equivalent changes and modifications made in accordance with the claims should be construed to fall within the scope of the application.

Claims (9)

  1. An offline data caching method of an SCADA system is characterized by comprising the following steps:
    s1, dividing unsent acquired data into a plurality of data blocks, and storing the data blocks in a first buffer area, wherein the data blocks are provided with serial numbers distributed according to the data acquisition time sequence;
    s2, acquiring a set data retransmission mode, and if the data retransmission mode is a first retransmission mode, sequentially capturing data blocks with the largest serial numbers in the data blocks which are not completed and are transmitted from a first buffer area after network connection is restored;
    s3, if the data block is in the second transmission supplementing mode, acquiring serial number distribution sections of each data block stored in the first buffer zone, sequentially grabbing data blocks corresponding to serial numbers positioned in the middle of the serial number continuous sections to the second buffer zone to form a data block queue, and sequentially grabbing the data blocks in the second buffer zone according to the sequence of the data block queue for transmission after network connection is restored;
    monitoring the sending state of the real-time acquisition data in the process of the historical data supplementary transmission, acquiring a current data supplementary transmission mode if the network congestion feature occurs, switching the current data supplementary transmission mode to a second supplementary transmission mode if the current data supplementary transmission mode is in the first supplementary transmission mode, and reducing the supplementary transmission frequency of the data blocks in the second supplementary transmission mode if the current data supplementary transmission mode is already in the second supplementary transmission mode.
  2. 2. The method for offline data caching of SCADA system according to claim 1, wherein said step S1 comprises:
    carrying out format processing and endowing unique serial numbers to each divided data block, wherein the data block format comprises serial numbers, historical data information and check codes, and the serial numbers of the data blocks are larger as the historical time stamps of the first data record contained in the corresponding data block are later; and if the historical data information contains alarm information, adding a priority identifier into the corresponding data block format.
  3. 3. The method for offline data caching of SCADA system according to claim 2, wherein said step S2 comprises:
    after monitoring that network connection is restored, acquiring a set data supplementary transmission mode, searching whether a data block with a priority identifier exists in a first buffer area or not if the data supplementary transmission mode is a first supplementary transmission mode, and if the data block with the priority identifier exists, sequentially capturing the data block with the priority identifier and carrying out supplementary transmission on the data block to an SCADA server through a network;
    after the reissuing of all the data blocks with the priority identifiers is completed, the data blocks with the largest serial numbers in the data blocks which are not reissued are sequentially grabbed from the first buffer area and are transmitted.
  4. 4. A SCADA system offline data caching method according to claim 3, characterized in that said step S3 comprises:
    if the data supplementary transmission mode is the second supplementary transmission mode, searching whether the data blocks with the priority identifiers exist in the first buffer area, and if so, capturing the data blocks with the priority identifiers in sequence and carrying out supplementary transmission on the data blocks to the SCADA server through a network;
    after the reissuing of all the data blocks with the priority identifiers is completed, acquiring the serial number distribution section of each remaining data block stored in the first buffer area;
    after capturing the data blocks corresponding to the serial numbers positioned in the middle of the serial number continuous sections to the second buffer area, sequentially and repeatedly identifying each serial number continuous section corresponding to the rest data blocks in the first buffer area, capturing the data blocks corresponding to the serial numbers positioned in the middle of each serial number continuous section to the second buffer area until all the data blocks in the first buffer area are captured, and forming a data block queue in the second buffer area, wherein the data block queues are arranged according to the adding sequence of the data blocks;
    and sequentially capturing the data blocks in the second buffer area according to the sequence of the data block queues, and reissuing the data blocks to the SCADA server through a network.
  5. 5. The method for offline data caching of SCADA system according to claim 4, wherein said step S3 further comprises:
    if a plurality of serial number continuous sections corresponding to the rest data blocks in the first buffer area exist, after a plurality of data blocks corresponding to the serial numbers in the middle of each serial number continuous section are identified, the data blocks are sequentially grabbed into the second buffer area according to the sequence of the serial numbers of the plurality of data blocks.
  6. 6. A data acquisition device, comprising:
    the buffer memory module is used for dividing the unsent collected data into a plurality of data blocks and storing the data blocks in the first buffer memory area, wherein the data blocks are provided with serial numbers distributed according to the time sequence of data collection;
    the first supplementing module is used for sequentially grabbing data blocks with the largest serial numbers in the data blocks which are not completed and reissued from the first buffer area after network connection is restored to be transmitted;
    the second supplementing module is used for sequentially grabbing data blocks corresponding to the serial numbers positioned in the middle of the serial number continuous sections to the second buffer area to form a data block queue after acquiring the serial number distribution sections of each data block stored in the first buffer area, and sequentially grabbing the data blocks in the second buffer area according to the sequence of the data block queue for transmission after recovering network connection;
    the data acquisition device is also used for monitoring the sending state of the real-time acquisition data in the process of the supplement transmission of the historical data, acquiring a supplement module which operates currently if the network congestion characteristic occurs, switching the supplement module to a second supplement module if the supplement module is a first supplement module, and reducing the supplement transmission frequency of the data blocks in the second supplement module if the supplement module is a second supplement module in operation.
  7. 7. The data acquisition device of claim 6 wherein the caching module is further configured to format and assign unique sequence numbers to each of the partitioned data blocks, the data block format including a sequence number, historical data information, and a check code, wherein the sequence number of each data block increases as a historical timestamp of a first data record contained in a corresponding data block increases; and if the historical data information contains alarm information, adding a priority identifier into the corresponding data block format.
  8. 8. The data acquisition device of claim 7, wherein: the first supplementing module is configured to acquire a set data supplementing mode after monitoring that the network connection is restored, if the data supplementing mode is the first supplementing mode, searching whether a data block with a priority identifier exists in the first buffer area, and if the data block with the priority identifier exists, sequentially grabbing the data block with the priority identifier and supplementing the data block to the SCADA server through a network; after the reissuing of all the data blocks with the priority identifiers is completed, the data blocks with the largest serial numbers in the data blocks which are not reissued are sequentially grabbed from the first buffer area and are transmitted.
  9. 9. A computer-readable storage medium storing a computer program, characterized in that: the computer program implementing the steps of the method according to any of claims 1-5 when executed by a processor.
CN202311080948.8A 2023-08-25 2023-08-25 SCADA system offline data caching method, device and storage medium Active CN116827883B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311080948.8A CN116827883B (en) 2023-08-25 2023-08-25 SCADA system offline data caching method, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311080948.8A CN116827883B (en) 2023-08-25 2023-08-25 SCADA system offline data caching method, device and storage medium

Publications (2)

Publication Number Publication Date
CN116827883A CN116827883A (en) 2023-09-29
CN116827883B true CN116827883B (en) 2023-11-17

Family

ID=88114810

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311080948.8A Active CN116827883B (en) 2023-08-25 2023-08-25 SCADA system offline data caching method, device and storage medium

Country Status (1)

Country Link
CN (1) CN116827883B (en)

Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003188898A (en) * 2001-12-21 2003-07-04 Nippon Telegr & Teleph Corp <Ntt> File distribution reception buffer management method, apparatus for implementing the method, program, and recording medium recording the program
WO2007098702A1 (en) * 2006-03-02 2007-09-07 Huawei Technologies Co., Ltd. Data transmitting method and system with re-transmitting request and status report of receiver formatting method
CN101175012A (en) * 2006-11-01 2008-05-07 大唐移动通信设备有限公司 Method, system and device for frame media data transmission
CN101369967A (en) * 2007-08-14 2009-02-18 华为技术有限公司 Method for transmitting and receiving data, base station, terminal
KR20090044309A (en) * 2007-10-31 2009-05-07 한양대학교 산학협력단 How to send and receive multimedia data on demand
CN105405325A (en) * 2015-12-22 2016-03-16 深圳市时尚德源文化传播有限公司 Network teaching method and system
CN105974836A (en) * 2016-04-29 2016-09-28 郑州宇通客车股份有限公司 Bus monitoring host data acquisition method and bus monitoring host data acquisition system
WO2017022211A1 (en) * 2015-08-03 2017-02-09 パナソニックIpマネジメント株式会社 Transmission method, reception method, transmission apparatus, and reception apparatus
CN106452737A (en) * 2010-08-11 2017-02-22 安全第公司 Systems and methods for secure multi-tenant data storage
CN108052567A (en) * 2017-12-06 2018-05-18 吉旗(成都)科技有限公司 A kind of method that increment caches displaying with pulling time series data and exhaustive
CN109194451A (en) * 2018-08-17 2019-01-11 北汽福田汽车股份有限公司 The data of car-mounted terminal reissue method and car-mounted terminal
CN113031551A (en) * 2021-03-09 2021-06-25 科大智能物联技术有限公司 Intelligent arrangement method and system for automatic production line of workshop
CN113114589A (en) * 2021-04-28 2021-07-13 北京中宏立达信创科技有限公司 Cross-network data secure transmission system and method
CN113242165A (en) * 2021-03-31 2021-08-10 惠州市德赛西威汽车电子股份有限公司 New energy vehicle national standard data processing and uploading method
CN113934530A (en) * 2020-12-31 2022-01-14 技象科技(浙江)有限公司 Multi-core multi-queue task cross processing method, device, system and storage medium
CN113986862A (en) * 2021-10-27 2022-01-28 上海星融汽车科技有限公司 Client log collection method
CN113992471A (en) * 2021-10-26 2022-01-28 成都亿佰特电子科技有限公司 Communication system and method for multiple hosts and multiple slaves
CN114020440A (en) * 2020-12-31 2022-02-08 技象科技(浙江)有限公司 Multi-stage task classification processing method, device and system and storage medium

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11675785B2 (en) * 2020-01-31 2023-06-13 Oracle International Corporation Dynamic asynchronous traversals for distributed graph queries
CN113489575A (en) * 2021-06-25 2021-10-08 阿波罗智联(北京)科技有限公司 Data transmission method and device and electronic equipment

Patent Citations (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003188898A (en) * 2001-12-21 2003-07-04 Nippon Telegr & Teleph Corp <Ntt> File distribution reception buffer management method, apparatus for implementing the method, program, and recording medium recording the program
WO2007098702A1 (en) * 2006-03-02 2007-09-07 Huawei Technologies Co., Ltd. Data transmitting method and system with re-transmitting request and status report of receiver formatting method
CN101175012A (en) * 2006-11-01 2008-05-07 大唐移动通信设备有限公司 Method, system and device for frame media data transmission
CN101369967A (en) * 2007-08-14 2009-02-18 华为技术有限公司 Method for transmitting and receiving data, base station, terminal
KR20090044309A (en) * 2007-10-31 2009-05-07 한양대학교 산학협력단 How to send and receive multimedia data on demand
CN106452737A (en) * 2010-08-11 2017-02-22 安全第公司 Systems and methods for secure multi-tenant data storage
WO2017022211A1 (en) * 2015-08-03 2017-02-09 パナソニックIpマネジメント株式会社 Transmission method, reception method, transmission apparatus, and reception apparatus
CN105405325A (en) * 2015-12-22 2016-03-16 深圳市时尚德源文化传播有限公司 Network teaching method and system
CN105974836A (en) * 2016-04-29 2016-09-28 郑州宇通客车股份有限公司 Bus monitoring host data acquisition method and bus monitoring host data acquisition system
CN108052567A (en) * 2017-12-06 2018-05-18 吉旗(成都)科技有限公司 A kind of method that increment caches displaying with pulling time series data and exhaustive
CN109194451A (en) * 2018-08-17 2019-01-11 北汽福田汽车股份有限公司 The data of car-mounted terminal reissue method and car-mounted terminal
CN113934530A (en) * 2020-12-31 2022-01-14 技象科技(浙江)有限公司 Multi-core multi-queue task cross processing method, device, system and storage medium
CN114020440A (en) * 2020-12-31 2022-02-08 技象科技(浙江)有限公司 Multi-stage task classification processing method, device and system and storage medium
CN113031551A (en) * 2021-03-09 2021-06-25 科大智能物联技术有限公司 Intelligent arrangement method and system for automatic production line of workshop
CN113242165A (en) * 2021-03-31 2021-08-10 惠州市德赛西威汽车电子股份有限公司 New energy vehicle national standard data processing and uploading method
CN113114589A (en) * 2021-04-28 2021-07-13 北京中宏立达信创科技有限公司 Cross-network data secure transmission system and method
CN113992471A (en) * 2021-10-26 2022-01-28 成都亿佰特电子科技有限公司 Communication system and method for multiple hosts and multiple slaves
CN113986862A (en) * 2021-10-27 2022-01-28 上海星融汽车科技有限公司 Client log collection method

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
变电站远动装置遥信误发原因分析及优化;曾王杰;郑文棋;张东明;周立辉;;科技创新导报(19);全文 *
基于异常数据驱动的WSN簇内数据融合方法;谭德坤;付雪峰;赵嘉;涂振宇;;传感技术学报(02);全文 *

Also Published As

Publication number Publication date
CN116827883A (en) 2023-09-29

Similar Documents

Publication Publication Date Title
US6968292B2 (en) Industrial controller event logging
CN116880412B (en) A cloud-based visual production management platform
CN101895446B (en) Method and device for detecting broadcast storm
CN101136781B (en) Performance data acquisition occasion control method and device in network management system
CN117202117B (en) System and method for fusing and degrading short message channel
CN116800691B (en) Cache data segmentation processing method, system and storage medium
CN117221088A (en) Computer network intensity detection system and device
CN116827883B (en) SCADA system offline data caching method, device and storage medium
CN119519757A (en) A communication transmission system for power equipment based on power wireless private network
CN111786851A (en) Information reporting method and device
CN119520332A (en) A network status monitoring and optimization processing method, device and readable storage medium
CN118972400B (en) Intelligent exchange and integration method for heterogeneous data of digital enterprise
CN115002035A (en) Power data transmission congestion evasion method based on service level
CN110119314A (en) A kind of server calls method, apparatus, server and storage medium
US9477736B2 (en) Apparatus and method for active and passive data gathering using stochastic model in control network
CN112052226B (en) Processing method and device for vehicle-mounted service data
CN115865940A (en) Data interaction processing method based on block chain
CN114328750A (en) Method and device for synchronizing service data with ODS (oxide dispersion strengthened) layer
CN113391977A (en) Log acquisition processing method and device
CN112865312A (en) Power dispatching system and power data processing method
CN117792991B (en) Automatic switching method for router links and multi-link router
CN110838938B (en) DNC data storage server scheduling method based on industrial control network
CN220457427U (en) Self-increment ID generation system
CN118138434B (en) Hierarchical management method and hierarchical management system for distributed network management platform
CN114040442B (en) Wireless gateway transmission system, information filtering method thereof and wireless gateway

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant