CN120321267B

CN120321267B - Text data monitoring and alarming method and system for multi-protocol transmission

Info

Publication number: CN120321267B
Application number: CN202510627913.4A
Authority: CN
Inventors: 张思源; 韩小宁; 田继
Original assignee: Shanxi Hanlun Technology Co ltd
Current assignee: Shanxi Hanlun Technology Co ltd
Priority date: 2025-05-15
Filing date: 2025-05-15
Publication date: 2025-11-21
Anticipated expiration: 2045-05-15
Also published as: CN120321267A

Abstract

The invention relates to a text data monitoring and alarming method and system for multi-protocol transmission, the method comprises the steps of generating a multi-source protocol transmission example based on dynamic authorization and hardware safety verification, collecting and analyzing text data, generating a transmission state and a content abnormal event queue through multithreading real-time monitoring network connection state, data backlog quantity and sensor numerical value content parameters, learning causal relation of network congestion, equipment faults and alarming events by using a Bayesian network algorithm, calculating root probability by combining a dynamic weight distribution strategy, generating a comprehensive alarming list with priority ordering, and realizing closed-loop optimization of abnormal early warning triggering, data snapshot binding and alarming logs based on user feedback data self-adaption, so that the problems of poor safety suitability, single monitoring dimension, dependence on static rules due to root analysis and strategy updating hysteresis in a multi-protocol mixed transmission scene are solved, and the real-time property, accuracy and self-adaption of industrial Internet of things data transmission are improved.

Description

Text data monitoring and alarming method and system for multi-protocol transmission

Technical Field

The invention relates to the technical field of electric communication transmission, in particular to a text data monitoring and alarming method and system for multi-protocol transmission.

Background

With the rapid development of industrial internet of things, the data transmission requirements among sensors, devices and systems in an industrial production environment are increasingly complicated, and a scene of hybrid transmission by utilizing FTP, SAMBA, kafka and other multi-protocols is a normal state. In such a scenario, how to monitor the data states transmitted by different protocols in real time, quickly locate the cause of the abnormality and dynamically optimize the alarm strategy becomes a key technical challenge for guaranteeing the continuity and safety of industrial production. The traditional data monitoring method is generally designed aiming at a single protocol, lacks unified adaptation capability to a multi-protocol mixed transmission scene, has obvious hysteresis in links such as anomaly detection, root cause analysis and strategy iteration, and is difficult to meet the severe requirements of industrial scenes on instantaneity, accuracy and adaptivity.

In the prior art, a monitoring and alarming method aiming at multi-protocol transmission mainly has the defects that firstly, a dynamic authorization and hardware-level safety verification mechanism is lacked in a multi-protocol instantiation and data acquisition process, so that unauthorized equipment is easy to access a system and data leakage risks exist, secondly, the monitoring dimension is single, only basic parameters such as network connection state or data backlog quantity are often concerned, real-time analysis and abnormal detection of key parameters of sensor values in transmission content are ignored, so that content-level risks cannot be timely early-warned, thirdly, the alarming root depends on manual experience or a static rule base for analysis, complex causal relation among network congestion, equipment faults and protocol alarms cannot be dynamically learned, the false alarm rate and the missing alarm rate are high, fourthly, alarming strategy adjustment is lagged, a self-adaptive weight updating mechanism based on user feedback is lacked, and closed loop optimization is difficult to form between real-time early-warning and historical data.

Disclosure of Invention

Based on the above, the invention aims to provide a text data monitoring and alarming method and a system capable of realizing multi-protocol security adaptation, multi-dimensional state real-time monitoring, intelligent root cause analysis and dynamic policy iteration multi-protocol transmission.

The invention adopts the following scheme:

In a first aspect, the present invention provides a method for monitoring and alarming text data transmitted by multiple protocols, comprising the following steps:

s1, carrying out instantiation processing on FTP, SAMBA, kafka protocols based on a preset multi-protocol configuration table, and extracting a multi-source protocol transmission example;

s2, monitoring network connection state and data backlog quantity of a multi-source protocol transmission example through multithreading, and simultaneously analyzing text content key parameters of structured data to generate a monitoring event queue comprising transmission state monitoring events and content abnormal events;

The transmission state monitoring event is used for indicating network connection interruption and file backlog quantity to exceed a preset backlog threshold, and the content abnormal event is used for indicating the value of the sensor to exceed a preset content threshold;

S3, carrying out alarm root cause analysis on the transmission state monitoring event and the content abnormal event of the monitoring event queue based on a preset dynamic weight allocation strategy and a Bayesian network algorithm, and generating a comprehensive alarm list with priority;

S4, carrying out abnormal early warning triggering judgment on high-priority alarms of the comprehensive alarm list based on priority ordering of the comprehensive alarm list, generating abnormal early warning information, carrying out data association on the abnormal early warning information and the structured data, generating an alarm log and sending the alarm log to the cloud platform, and simultaneously carrying out weight updating on protocol priority parameters of the dynamic weight allocation strategy based on user feedback data fed back by the cloud platform, and generating an updated dynamic weight allocation strategy.

In a second aspect, the present invention provides a text data monitoring and alerting system for multi-protocol transmission, the system being configured with the following modules:

The multi-protocol adaptation and data preprocessing module is used for carrying out instantiation processing on FTP, SAMBA, kafka protocols based on a preset multi-protocol configuration table and extracting a multi-source protocol transmission example;

The system comprises a transmission state and content anomaly monitoring module, a transmission state monitoring module, a content anomaly monitoring module and a content anomaly monitoring module, wherein the transmission state and content anomaly monitoring module is used for monitoring the network connection state and the data backlog of a multi-source protocol transmission example through multithreading, and simultaneously analyzing text content key parameters of structured data to generate a monitoring event queue comprising transmission state monitoring events and content anomaly events;

The root cause analysis and priority ordering module is used for carrying out alarm root cause analysis on the transmission state monitoring event and the content abnormal event of the monitoring event queue based on a preset dynamic weight allocation strategy and a Bayesian network algorithm, and generating a comprehensive alarm list with priority;

The early warning feedback and strategy optimization module is used for carrying out abnormal early warning triggering judgment on high-priority alarms of the comprehensive alarm list based on priority ordering of the comprehensive alarm list, generating abnormal early warning information, carrying out data association on the abnormal early warning information and the structured data, generating an alarm log and sending the alarm log to the cloud platform, and simultaneously carrying out weight update on protocol priority parameters of the dynamic weight allocation strategy based on user feedback data fed back by the cloud platform, and generating an updated dynamic weight allocation strategy.

In a third aspect, the present application provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements any one of the above-mentioned multi-protocol transmitted text data monitoring and alerting methods when executing the computer program.

In a fourth aspect, the present application provides a computer readable storage medium having stored thereon a computer program which when executed by a processor implements any of the above-described methods of monitoring and alerting multi-protocol transmitted text data.

In summary, the text data monitoring and alarming method for multi-protocol transmission provided by the invention can realize the secure instantiation and data acquisition of FTP, SAMBA, kafka hybrid transmission protocol in the industrial Internet of things scene through the multi-protocol adaptation and dynamic security check mechanism. The method is based on hardware fingerprint encryption to generate a unique machine code of the equipment and combines with protocol factory dynamic creation examples, can achieve dual protection of unauthorized equipment interception and protocol channel encryption, effectively solve the problem of data leakage risk caused by lack of hardware-level security check in traditional multi-protocol transmission, can achieve full-dimension real-time detection of anomalies of a transmission layer and a content layer by multi-thread parallel monitoring of network connection states, data backlog amounts and sensor numerical content parameters, and can accurately generate multi-type events such as network disconnection, file backlog overrun and sensor numerical value overrun by means of a heartbeat packet detection mechanism, a dynamic backlog threshold adjustment algorithm and keyword matching rules, and remarkably improve anomaly coverage rate and detection timeliness in complex transmission scenes.

Meanwhile, based on collaborative calculation of a Bayesian network root cause analysis algorithm and a dynamic weight distribution strategy, autonomous learning and priority dynamic sequencing of alarm event cause and effect relationships can be realized, a Markov blanket algorithm is used for positioning root cause nodes and combining with conditional probability iteration of a historical alarm log, analysis limitation of a traditional static rule base on composite root causes such as network congestion, equipment faults and the like can be broken through, false alarm rate is effectively reduced, closed loop dynamic update of an alarm strategy can be realized by means of a protocol weight optimization mechanism driven by user feedback, and through binding storage of abnormal early warning information and structured data snapshot, HTTPS protocol alarm log pushing and adaptive adjustment of attenuation/enhancement rules, full link optimization from real-time alarm triggering and data tracing to strategy iteration can be realized, average fault response time of an industrial Internet of things data transmission system is greatly shortened, alarm accuracy is improved to 99%, and meanwhile, real-time processing capacity of thousands of concurrent events per second is supported, and a system solution with safety, intelligence and self-adaption is provided for a high-reliability and high-concurrency industrial data transmission scene.

For a better understanding and implementation, the present invention is described in detail below with reference to the drawings.

Drawings

Fig. 1 is a schematic flow chart of a text data monitoring and alarming method for multi-protocol transmission according to an embodiment of the present application;

FIG. 2 is a flow chart of a method for generating a monitoring event queue containing transmission status monitoring events and content anomaly events according to an embodiment of the present application;

FIG. 3 is a schematic flow chart of interactive docking with the outside according to an embodiment of the present application;

FIG. 4 is a flow chart of a text data monitoring and alerting procedure for performing multi-protocol transmission according to another embodiment of the present application;

Fig. 5 is a schematic structural diagram of a text data monitoring and alarm system for multi-protocol transmission according to another embodiment of the present application.

Detailed Description

In order that the invention may be readily understood, a more complete description of the invention will be rendered by reference to the appended drawings. Preferred embodiments of the present invention are shown in the drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and/or" as used herein includes any and all combinations of one or more of the associated listed items.

In one embodiment, as shown in fig. 1, a method for monitoring and alarming text data transmitted by multiple protocols is provided, and this embodiment is applied to a terminal for illustration, it is understood that the method may also be applied to a server, and may also be applied to a system including the terminal and the server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the steps of:

S1, carrying out instantiation processing on FTP, SAMBA, kafka protocols based on a preset multi-protocol configuration table, extracting a multi-source protocol transmission example, collecting text data stored locally, analyzing the text data into a structured field, generating structured data, and storing the structured data into a shared memory buffer.

Specifically, the preset multi-protocol configuration table details parameters such as server address, port number, user name, password, encryption mode and data transmission rate of each protocol, so that each protocol instance can be initialized normally and connection can be established according to preset configuration. In the process of instantiation, the system performs parameter verification and initialization setting on each protocol, for example, SMB signature verification is configured for SAMBA, and Topic and partition strategies of a producer and a consumer are set for Kafka, so that the protocol instance can perform data interaction according to the designated security strategy and transmission rules.

Meanwhile, the system performs acquisition operation on the locally stored text data, and identifies the text file to be processed by traversing a preset local data storage directory. The system carries out structural analysis on the collected text data according to the file type and the predefined data format specification, decomposes unstructured text content into a plurality of structured fields with definite semantics and formats, generates key information such as data collection time stamps, identifiers of sensors, measured values, state codes of equipment and the like, and generates structured data. In order to realize temporary storage and efficient reading and writing of data, the system stores structured data into a shared memory buffer area. The buffer area is based on a memory management algorithm, adopts a ring buffer area structure, ensures the quick access and thread safety of data, ensures the consistency of the data through a memory barrier technology, and provides real-time data support for subsequent data monitoring and processing. In the process of storing data into the buffer area, the system performs integrity check on the data, ensures the accuracy and consistency of the data in the process of acquisition and transmission, and records meta information of the data, such as file names, paths, sizes, hash values and the like, so as to facilitate subsequent data tracking and auditing.

The transmission state monitoring event is used for indicating network connection interruption and file backlog quantity to exceed a preset backlog threshold, and the content abnormal event is used for indicating the sensor value to exceed a preset content threshold.

Specifically, in the aspect of network connection state monitoring, the system detects whether network connection of each protocol is interrupted or not by periodically sending heartbeat packets, and simultaneously monitors network indexes such as connection delay, data packet loss rate and the like by adopting a TCP window scaling algorithm. For monitoring the data backlog quantity, the system deploys data backlog monitoring modules at the local and target server ends respectively, and judges whether the data backlog exceeds a normal range by counting the number and the size of files in a file system and combining a preset backlog threshold. The preset backlog threshold is dynamically adjusted according to the historical data flow and the system processing capacity so as to adapt to different workload scenes.

Meanwhile, the system carries out deep analysis on the structured data in the shared memory buffer area, and extracts key parameters in text content, such as physical quantity values measured by a sensor, equipment running state codes and the like. The system judges whether the parameters are abnormal or not according to a preset content threshold, the content threshold is set according to the measuring range of the sensor, the normal operation range of the equipment and the process requirement, and dynamic optimization is carried out through a machine learning algorithm. The system integrates various events generated in the monitoring process to generate a monitoring event queue, wherein the transmission state monitoring event is used for indicating abnormal interruption of network connection and the condition that the number of backlog files exceeds a preset backlog threshold, and the content abnormal event is used for indicating that key parameters such as sensor values exceed a preset content threshold range. The event queue adopts a priority queue structure, and is ordered according to the emergency degree and the severity of the event, so that the high-priority event can be timely processed. The system updates and maintains the event queue in real time, avoids the interference of repeated events through an event filtering and deduplication mechanism, and improves the monitoring efficiency and accuracy.

And S3, carrying out alarm root cause analysis on the transmission state monitoring event and the content abnormal event of the monitoring event queue based on a preset dynamic weight distribution strategy and a Bayesian network algorithm, and generating a comprehensive alarm list with priority.

Specifically, the dynamic weight distribution strategy comprehensively considers various factors such as event type, occurrence frequency, history influence degree, user feedback and the like, and different weight values are respectively given to a transmission state monitoring event and a content abnormal event through a weight calculation model. The weight calculation model adopts a multi-factor weighted summation mode, wherein the event type weight is preset according to the influence degree of the event on the system operation, the occurrence frequency weight is dynamically adjusted through statistical analysis of historical event data, the historical influence degree weight is determined according to the number and severity of system faults caused by the event in the past, and the user feedback weight is updated according to the satisfaction evaluation of the user on the event processing result.

The Bayesian network algorithm builds a causal relationship network between events based on the probability graph model. The network nodes represent different event types and system state variables, and the directed edges represent causal relationships and conditional probabilities between events. The system discovers the root cause of abnormal alarm by analyzing various complex causal relations such as data backlog possibly caused by network connection interruption, abnormal sensor values possibly caused by equipment faults and the like. And combining the dynamic weight and the causal relationship analysis result, and generating a comprehensive alarm list with priority by the system. The priority calculation is based on the comprehensive evaluation of the weight value and the causal relationship probability, and the priority of each alarm is determined by adopting a weighted average method. The alarm list is ordered from high to low according to the priority, and clear priority guidance is provided for subsequent abnormal early warning processing. The system updates and maintains the comprehensive alarm list in real time, ensures the accuracy and the integrity of alarm information through an alarm merging and decomposing mechanism, and simultaneously avoids the generation of alarm storm.

Specifically, the early warning rule and the threshold are set according to historical data of system operation and a user-defined strategy, and the early warning rule and the threshold comprise parameters such as a triggering condition, an early warning level, a notification mode and the like of the alarm. When the alarm meets the triggering condition, the system generates abnormal early warning information and accurately associates the early warning information with corresponding structured data. The correlation process is matched through key fields such as a time stamp, a device identifier and a sensor number of data, and consistency and accuracy of early warning information and the data are ensured. The system generates detailed alarm logs, and the content covers the detailed information such as the time, type, position, level, related equipment and sensor information, abnormal parameter values, possible reason analysis and the like of the occurrence of the alarms, so that the subsequent analysis and tracing are convenient.

And then, the system sends the alarm log to the cloud platform for centralized storage and management. The cloud platform is based on a distributed storage architecture, a multi-copy mechanism is adopted to ensure the reliability and availability of data, and meanwhile, the storage efficiency and the query performance are improved through a data compression and indexing technology. And after the cloud platform receives alarm logs from a plurality of systems, data aggregation and analysis are carried out, and potential common problems and trends are mined. Meanwhile, the cloud platform updates the weight of the protocol priority parameters in the dynamic weight allocation strategy based on a large amount of collected user feedback data. The user feedback data comprises information such as evaluation of the alarm processing result, opinion of early warning accuracy, suggestion of system performance and the like by the user. The system excavates modes and trends in user feedback data through a machine learning algorithm, such as a gradient descent method and a random forest algorithm, and adjusts parameter values of each protocol in dynamic weight distribution. The updated dynamic weight distribution strategy is pushed to each system node through the cloud, so that the self-adaptive optimization of the alarm strategy is realized, and the alarm accuracy and timeliness of the system in complex and changeable industrial environments are improved. The system carries out version management and compatibility test on the updated strategy, ensures that the new strategy can be stably transited and effectively improves the system performance.

In one embodiment, the S1 method for monitoring and alarming text data transmitted by multiple protocols provided by the present invention specifically includes the following steps:

S11, acquiring a serial number of a main board, a CPU chip identifier and a network card physical address through a hardware fingerprint encryption module, generating a unique machine code of the equipment, completing equipment authorization verification based on the unique machine code of the equipment, and activating a multi-protocol configuration table on the authorized equipment.

Specifically, the system firstly calls the bottom hardware access interface to acquire the serial number of the main board, the chip identifier of the CPU and the physical address of the network card, and the information is taken as the unique identifier of the equipment and is acquired and transmitted to the hardware fingerprint encryption module. The module adopts SHA-256 algorithm to carry out hash operation on the acquired hardware information, and generates a device unique machine code with fixed length. The generated machine code is then sent to an authorization server for verification, and the authorization server verifies whether the machine code has legal authorization according to pre-stored device authorization information. If the verification is passed, the authorization server returns an authorization success response, and sends a multi-protocol configuration table to the device, and after the system receives the response, the configuration table is locally activated so that the configuration table can be used for subsequent protocol instantiation operation.

S12, based on the server address, the port number and the key information recorded in the multi-protocol configuration table, calling the protocol factory class to create an FTP protocol instance, a SAMBA protocol instance and a Kafka protocol instance, respectively operating the FTP protocol instance, the SAMBA protocol instance and the Kafka protocol instance on independent threads, and extracting a final multi-source protocol transmission instance.

Specifically, the system reads various parameters in the multiprotocol configuration table, including the IP address, port number, user name, password of the FTP server, hostname, port number, SMB signature verification configuration of the SAMBA server, and Bootstrap server address, topic name, configuration parameters of the producer and consumer of the Kafka cluster, etc. Then, the system calls the protocol factory class, and corresponding protocol instances are respectively created according to the parameters. The protocol factory class initializes the corresponding protocol client object according to different protocol types, and sets the connection attribute and the security parameter of the protocol client object, such as setting a passive mode for FTP, loading SMB signature verification configuration for SAMBA, configuring a data serialization mode for Kafka, and the like. After the creation is completed, the system starts an independent thread for each protocol instance, ensures that each protocol operates in the independent thread, avoids mutual interference, and improves the concurrent processing capacity of the system. Through a thread management mechanism, the system monitors the state of each thread, ensures the stable operation of the protocol instance, and once the abnormality of the thread is found, the restarting or recovery operation can be timely carried out, thereby ensuring the normal operation of the multi-source protocol transmission instance and providing basic support for the subsequent data acquisition and transmission.

And S13, performing extension matching on text files in the local path, filtering non-text files, screening and extracting contents of text files with extension of TXT, XML and DOC, and analyzing sensor numbers, values and timestamp fields in the screened text files through a regular expression to generate structured data.

Specifically, the system starts a file scanning process, and traverses files under a designated directory and sub-directories thereof according to a preset local path specification. In the scanning process, the system checks the extension name of each file one by one, filters non-text files which do not meet the requirements, such as files in formats of pictures, audios, videos and the like, according to a predefined text file type list, such as TXT, XML, DOC text, and ensures that only the text files are subjected to subsequent processing. For the screened text files, the system adopts a multithreaded file reading mechanism to efficiently load file contents. For text files with different formats, the system calls corresponding resolvers, for example, for XML files, a document object model is constructed by using a DOM resolvers, and for TXT and DOC files, content extraction is carried out through regular expression matching and character string operation. In the analysis process, key fields such as a sensor number, numerical data acquired by the sensor, a time stamp for data acquisition and the like are identified and extracted by a system in a key mode. The regular expression pattern is predefined according to a format specification of sensor data, for example, the sensor number may follow a pattern of "sn\d {6}, the numerical data may be in a floating point number format of" \d+, and the timestamp may conform to a date-time format of "\d {4} - { d {2} -, d {2} -, s\d {2}: \d {2 }. Through the accurate matching of regular expressions, the system can accurately extract required field information from text contents and organize the field information into a structured data format such as JSON or XML so as to facilitate subsequent data processing and analysis.

And S14, storing the structured data into a shared memory buffer area, marking the structured data as an unprocessed state, and generating the structured data marked as unprocessed.

Specifically, after the generation of the structured data is completed, the system calls the shared memory management interface to apply for a section of shared memory space. The shared memory buffer zone adopts a ring queue structure to realize efficient data storage and reading operation. The system performs serialization processing on the generated structured data according to a predefined data format specification, so that the data is ensured to be stored compactly and is easy to analyze in the shared memory. The serialized data is written into the tail of the shared memory buffer, and the write pointer position of the buffer is updated at the same time, so as to maintain the correct sequence of the data. To identify the processing status of the data, the system adds a status field in each structured data record, sets its initial value to "unprocessed," and explicitly indicates that the data has not undergone subsequent monitoring and analysis procedures. In addition, the system records meta-information of the data, including the time stamp of the data generation, the source file path, the file size, etc., which aids in tracking and auditing during data processing. The shared memory buffer area adopts a concurrency control mechanism to ensure the consistency and the integrity of data in a multithreading environment. When the data is successfully written into the buffer zone, the system triggers the data available event notification to inform the monitoring thread that new data is to be processed, so that efficient cooperation between the data acquisition and the data monitoring module is realized, and the system is ensured to respond and process the newly generated structured data in time.

In one embodiment, the S2 method for monitoring and alarming text data transmitted by multiple protocols provided by the present invention specifically includes the following steps:

And S21, sending a heartbeat packet to the multi-source protocol transmission example based on a heartbeat packet detection mechanism, counting the failure times of receiving the response signal, and if the response signal is not received for three times continuously, generating a network disconnection event, wherein the network disconnection event is the connection interruption state of the continuous unresponsive heartbeat packet of the protocol example.

Specifically, heartbeat packets are continuously sent to each protocol instance at preset time intervals, for example, once every 30 seconds, to ensure that the system is able to quickly capture abnormal changes in the network connection. After each heartbeat packet is sent out, the system starts a timer, and synchronously records the time waiting for response. If a response signal is not received within the timeout period, the system marks the attempt as failed and increments a failure count counter. Once no response signal is received three consecutive times, the system determines that the network connection of the protocol instance has been broken, and then triggers a network disconnection event. This event not only details the protocol type, unique identifier of the instance, but also marks precisely the time stamp of the disconnection, and the specific time of the last successful communication, thus providing detailed data support for subsequent failure diagnosis and network recovery. At the same time, the system automatically activates exception handling procedures including, but not limited to, attempting to reestablish a connection, logging a disconnection event to a log file, and sending a notification to a system administrator, ensuring that network exceptions can be handled timely and efficiently.

S22, counting the number of the local files to be transmitted and the number of backlog files at the server side of each protocol instance in the multi-source protocol transmission instance, judging whether the number of the local files to be transmitted exceeds a preset backlog threshold, if so, generating a backlog alarm event, wherein the preset backlog threshold is the maximum file accumulation number dynamically adjusted according to the storage capacity of the server.

Specifically, the system performs file quantity statistics on each protocol instance in the multi-source protocol transmission instance so as to monitor the situation of local files to be transmitted and backlog files at the server side, thereby ensuring the smoothness of data transmission. In the aspect of local file statistics, the system periodically scans a local storage catalog, comprehensively counts the number of files to be transmitted, and compares the number of files with a preset backlog threshold. The threshold is not fixed, but dynamically adjusted according to the storage capacity of the server, the current processing load, the historical data flow and other factors so as to adapt to different working load conditions and effectively prevent the occurrence of data blocking. If the number of the files to be transmitted locally exceeds a preset backlog threshold, the system immediately generates a backlog alarm event and records the information of the related protocol instance, the total number of the files currently backlogged, the specific time exceeding the threshold and other key contents in detail. After the backlog alarm event is triggered, the system rapidly takes a series of countermeasures, such as suspending new data acquisition tasks, dynamically increasing the number of data transmission threads, optimizing transmission scheduling strategies and the like, so as to relieve the problem of file backlog and ensure the high efficiency and stability of data transmission.

S23, extracting a current sensing value from a sensor value field of the structured data, judging whether the current sensing value exceeds a preset content threshold, and if so, generating a content alarm event comprising a sensor number, the current value and the preset threshold.

Specifically, in the extraction process, the system carries out fine analysis on key fields such as sensor numbers, values and time stamps in the structured data, so as to ensure that the obtained sensing values are accurate. The system strictly compares the extracted current sensing value with a preset content threshold value, and the threshold value is set after fine calibration according to parameters such as the measuring range of the sensor, the measurement precision requirement, specific process operation conditions and the like, so that the value fluctuation range of the sensor in a normal working state can be fully covered. If the current sensing value exceeds the preset content threshold, the system will quickly generate a content alert event comprising the sensor number, the current value and the preset threshold. The event not only records the basic information and abnormal constant value of the sensor in detail, but also performs preliminary analysis on possible reasons, such as sensor hardware faults, environmental factor interference and the like, by combining the time stamp of the occurrence of the abnormality, and provides corresponding processing suggestions. By generating the content alarm event in time, the system can quickly locate the source of sensor data abnormality and provide powerful data support and decision basis for the maintenance of equipment and the optimization adjustment of the process flow. Specifically, the process comprises the steps of:

s231, extracting the encrypted text data transmitted by the SAMBA protocol from the structured data, and executing a stream decryption algorithm by the hardware acceleration module to generate a clear text data stream.

Specifically, upon receiving the encrypted text data, the system quickly initializes a streaming decryption algorithm, such as an AES-CTR mode, to ensure the efficiency and security of data decryption. The hardware acceleration module obviously improves the data decryption speed and reduces the load of the CPU by calling a special encryption chip instruction set. In the decryption process, the system performs real-time verification on the encrypted data, and ensures the integrity and consistency of the data. The decrypted plaintext data stream is buffered in a memory buffer for subsequent key field parsing and content matching operations. The system adopts an advanced memory pool technology on the management of the memory buffer area, and frequent memory allocation and release operations are avoided, so that the data processing efficiency and the system stability are further improved.

S232, analyzing key fields of the plaintext data stream, extracting sensor numbers and associated numerical value fields, and matching data contents based on a preset keyword set.

Specifically, in the analysis process, the system adopts a regular expression engine, and combines a predefined sensor data format specification to accurately identify and extract the sensor number and the associated numerical value. The preset keyword set contains specific words or phrases related to the system monitoring target, such as equipment fault codes, abnormal state identifiers and the like. The system compares the extracted numerical value field with a preset keyword set through a pattern matching algorithm, and judges whether the data content contains key information or not. And meanwhile, the system performs format verification and data type conversion on the analyzed key fields, so that the extracted data is ensured to be accurate. In order to improve the analysis efficiency, the system adopts a multithreading parallel processing mechanism to analyze a plurality of sensor data simultaneously, so that the data accuracy is ensured, and the real-time performance and the high efficiency of the data processing are ensured.

S233, judging whether the associated numerical value field exceeds a preset content threshold value, and if so, generating an initial alarm event, wherein the initial alarm event comprises a sensor number, a current numerical value and a matching keyword.

Specifically, the system determines whether the associated value field exceeds a preset content threshold, and if so, generates an initial alert event comprising a sensor number, a current value, and a matching key. The preset content threshold is set according to the measuring range, the measuring precision and the technological operation requirement of the sensor, and is optimized and adjusted through historical data analysis and expert experience. In the judging process, the system adopts a multithread parallel processing mechanism to judge a plurality of sensor data simultaneously so as to improve the processing efficiency. Upon finding that the association value exceeds the threshold and hits the keyword, the system immediately generates an initial alert event and assigns it a unique event identifier. And the information such as the sensor number, the current value, the matching keyword, the time stamp of the alarm occurrence and the like is recorded in detail in the event, so that the integrity and traceability of the alarm event are ensured. The system also evaluates the priority of the initial alarm event, and determines the processing sequence according to the emergency degree and the potential influence of the alarm, thereby ensuring that the key alarm can be responded in time.

S234, the plaintext data stream is subjected to block processing, hash values of all data blocks are calculated based on an SHA-256 algorithm to generate a data integrity check code, the data integrity check code is bound with an initial alarm event, and a stored content alarm event is generated and stored in an alarm log database.

Specifically, in the process of block processing, the system divides the plaintext data stream into a plurality of data blocks with fixed sizes according to the preset data block sizes, so as to ensure the high efficiency and the parallelism of data processing. For each data block, the system calls the SHA-256 algorithm to perform hash operation, and generates a unique data integrity check code. The check code is used for verifying the integrity of the data in the transmission and storage processes and preventing the data from being tampered or damaged. The system binds the generated data integrity check code with the initial alarm event to form a stored content alarm event, and stores the stored content alarm event in an alarm log database. The alarm log database adopts a high-performance storage engine, supports the rapid storage and query of mass data, and ensures the long-term availability and the analyzability of alarm events. By binding the data integrity check code with the alarm event, the system can provide data tracing and verification functions, and the reliability and the credibility of the system are enhanced. The system also periodically maintains and optimizes the alarm log database, such as index rebuilding, data compression, etc., to ensure efficient operation of the database and rapid retrieval of data.

And S24, merging the network disconnection event, the backlog alarm event and the content alarm event to generate a monitoring event queue containing a transmission state monitoring event and a content abnormal event, wherein the transmission state monitoring event comprises the network disconnection event and the backlog alarm event.

Specifically, the shared memory buffer area has high concurrent read-write capability, and can rapidly respond to read-write requests of a plurality of processes or threads. When data is stored, the system adds a status flag to each piece of structured data, and marks the structured data as unprocessed, so that the subsequent processing modules can accurately identify and preferentially process the unprocessed data. The marking mechanism not only improves the efficiency of data management, but also provides clear state identification for the data processing flow of the system, and ensures orderly circulation of data in the whole processing chain. Specifically, the process comprises the steps of:

s241, merging the network disconnection event, the backlog alarm event and the content exception event to generate an initial event queue containing the transmission state monitoring event and the content exception event.

Specifically, the system accurately classifies various events through the event recognition module, marks the interruption condition of network connection, the data backlog condition and the content abnormality details respectively, integrates the interruption condition, the data backlog condition and the content abnormality details into an initial event queue, and lays a foundation for subsequent time stamp standardization processing. During the integration process, the system performs detailed attribute extraction and recording on each type of event, so that each event is ensured to carry enough context information, and a subsequent processing module can accurately understand and respond to the events. For example, a network disconnect event may record the time of the disconnect, the affected protocol instance, and the possible reason for the disconnect, and a backlog alert event may record the start time of the backlog of data, the amount of backlog, and related transmission paths.

And S242, performing standardized processing on the time stamp in the initial event queue, eliminating clock deviation among different protocol instances, and generating a standard event queue of the standardized time stamp.

Specifically, the system adopts a time synchronization algorithm to correct the time stamp of each event and generate a standard event queue of a standardized time stamp. The algorithm considers various factors such as network delay, equipment clock drift and the like, ensures that the corrected time stamp can accurately reflect the actual occurrence time of the event, and provides a reliable basis for event analysis and processing based on time sequence. During the normalization process, the system will convert all time stamps into a uniform time coordinate system, such as coordinated Universal Time (UTC), and accurate to the millisecond level to ensure high accuracy and consistency of the time stamps. The event queue processed in this way can reflect the actual occurrence sequence and time interval of the event more accurately, and provides a solid foundation for subsequent event association analysis and alarm processing.

S243, counting the alarm triggering times of the same protocol instance in the standard event queue based on a preset time window threshold, and if the alarm triggering times exceed a preset high-frequency threshold, extracting a corresponding alarm event from the standard event queue, and marking the corresponding alarm event as a high-frequency abnormal event.

Specifically, the system monitors the alarm frequency of each protocol instance in real time through a sliding time window mechanism. When the alarm triggering times of a certain protocol instance exceed a preset high-frequency threshold, the system immediately extracts a corresponding alarm event from the standard event queue and marks the corresponding alarm event as a high-frequency abnormal event. This tagging process not only records the high frequency nature of the event, but also retains all key information of the original event for subsequent in-depth analysis and processing. In the statistical process, the system dynamically adjusts the size of the time window to adapt to different monitoring scenes and data flow changes. Meanwhile, the system can evaluate the priority of the high-frequency abnormal event, and determine the priority of the high-frequency abnormal event in the subsequent processing flow according to the type, frequency, influence range and other factors of the alarm event, so that the key problem can be timely responded and processed.

S244, merging the high-frequency abnormal event into a monitoring event queue of a standardized timestamp, and attaching a current network bandwidth occupancy rate tag to generate a final monitoring event queue.

Specifically, the system merges the high-frequency abnormal event into a monitoring event queue with a standardized time stamp, and attaches a current network bandwidth occupancy rate tag to generate a final monitoring event queue. This merging process ensures that all critical event information is centrally managed and processed, while the additional information of network bandwidth occupancy provides additional context for event analysis. By the method, the system can evaluate the influence of the event more comprehensively, particularly under the condition of limited network resources, help operation and maintenance personnel to quickly identify and respond to potential system problems, and ensure the stable operation of the system and the reliability of data transmission. During the merging process, the system reorders and integrates the events, ensuring that the final monitored event queue is arranged in time order and each event carries complete context information. In addition, the system can also carry out redundancy check and deduplication processing on the final monitoring event queue, so that unnecessary interference of repeated events on subsequent processing flows is avoided. The finally generated monitoring event queue not only contains the key information of the original event, but also fuses the analysis and evaluation results of the system on the event, and provides comprehensive and accurate data support for subsequent alarm processing and system optimization.

According to the text data monitoring and alarming method for multi-protocol transmission, through the collaborative monitoring mechanism of multi-dimensional transmission state and content abnormality, accurate identification and real-time early warning of multi-protocol hybrid transmission risks in an industrial Internet of things scene can be achieved. The survival state of the protocol instance is periodically detected based on a heartbeat packet detection mechanism, the millisecond level sensing capability of network connection interruption can be achieved, the misjudgment problem caused by instantaneous network jitter can be effectively avoided through dynamic statistics and threshold judgment of continuous response failure times, and the generation of a network disconnection event is ensured to accurately reflect the real connection state of the protocol instance; the method is characterized in that a backlog threshold algorithm for dynamically adjusting the storage capacity of a server is combined, real-time quantity statistics and overrun early warning of a local file to be transmitted and the backlog file of the server can be realized, resource allocation differences of different protocol examples are matched through self-adaptive thresholds, file backlog risks caused by insufficient transmission bandwidth or storage overflow can be accurately identified, key parameter anomaly detection of a transmission content layer can be realized by means of real-time analysis and content threshold comparison of a sensor value field in structured data, specific equipment nodes and anomaly degree of numerical value out-of-range warning can be accurately positioned through multidimensional information binding of a sensor number, a current value and a preset threshold, finally, a full-dimension monitoring event queue covering network connection, transmission efficiency and data quality can be constructed through intelligent combination of a transmission layer state event and a content layer anomaly event, structured input is provided for subsequent root cause analysis, state sensing precision and anomaly handling efficiency of a multi-protocol transmission system under a complex industrial environment can be remarkably improved through event classification storage and time stamp calibration, single-covered monitoring dimension and full-delay safety response of the multi-protocol transmission system can be effectively guaranteed, and multi-layer internet-of things safety and the like can be solved.

In one embodiment, the S3 method for monitoring and alarming text data transmitted by multiple protocols provided by the present invention specifically includes the following steps:

s31, acquiring an initial weight value from a preset weight configuration table according to the protocol type of the multi-source protocol transmission example.

Specifically, the weight configuration table details the relative importance of different protocols in data transmission, and the weight values are preset based on factors such as historical data, stability requirements of transmission, priority of service and the like. The system rapidly locates the corresponding initial weight value through the protocol type identifier, and the process involves the analysis of the protocol type code and the accurate matching of the weight value. For example, the FTP protocol may be given a higher initial weight due to its higher real-time requirements, while the Kafka protocol is given a different weight due to its high reliability in large data transmissions. After the system acquires the initial weight values, the values are used as the basis of the subsequent dynamic weight calculation, so that the priority of each protocol instance in the system is reasonably embodied. The system also updates the weight configuration table periodically to adapt to the change of network environment and service requirements, and ensures that the weight value can reflect the latest priority requirements. In the updating process, the system adopts a smooth transition strategy, so that unstable system caused by abrupt change of weight values is avoided. Meanwhile, the system performs encryption storage and transmission on the weight configuration table, so that the safety and the integrity of weight information are ensured, and unauthorized tampering and leakage are prevented.

S32, based on Bayesian network algorithm, carrying out conditional probability calculation on protocol type, time interval and network state field in the history alarm log, learning causal relationship between network congestion, equipment fault and protocol alarm event, and constructing an alarm dependency graph.

Specifically, the Bayesian network algorithm computes conditional probabilities between different events by analyzing historical alert data, thereby revealing causal links between events. The system firstly carries out structural processing on the history alarm log, and extracts key fields such as protocol type, time interval, network state and the like. These data are then used to construct a bayesian network model in which nodes represent different alarm events and network states and edges represent causal relationships and conditional probabilities between them. Through iterative learning and parameter estimation, the system continuously optimizes the structure and parameters of the Bayesian network, so that the model can accurately reflect the complex causal relationship among network congestion, equipment faults and protocol alarms. The alarm dependency graph intuitively displays the relations and provides basis for subsequent root cause analysis. When the system builds the Bayesian network model, a new alarm data is continuously incorporated by adopting an incremental learning method, and model parameters are dynamically adjusted so as to improve the accuracy and adaptability of the model. Meanwhile, the system verifies and evaluates the Bayesian network model, and ensures the reliability and effectiveness of the model through methods such as cross verification, likelihood evaluation and the like. In order to improve the calculation efficiency, the system adopts a distributed calculation framework to carry out parallel processing on large-scale historical alarm data and accelerate the calculation of conditional probability and the training process of a model.

And S33, carrying out root cause probability calculation on the transmission state monitoring event and the content abnormal event in the monitoring event queue based on the alarm dependency graph, positioning root cause nodes through a Markov blanket algorithm, and generating alarm items with probability values.

Specifically, root probability calculation utilizes the conditional probability of each node in the bayesian network, and calculates the occurrence probability of each possible root in combination with event data in the current monitoring event queue. The Markov blanket algorithm is then used to efficiently locate the most likely causative nodes in the complex alarm dependency graph, i.e., those nodes that have the greatest impact on the current alarm event. Through these calculations, the system associates a probability value for each alarm entry that represents the likelihood that the alarm event was caused by a particular root cause. The alarm entries with probability values not only provide detailed information of alarms, but also provide quantitative basis for subsequent priority adjustment. When the system performs root cause probability calculation, approximate reasoning algorithms such as variation inference and Markov chain Monte Carlo methods are adopted to improve calculation efficiency and accuracy. And meanwhile, the system carries out sensitivity analysis on root cause probability calculation results, evaluates the influence of different factors on the root cause probability and ensures the stability and reliability of the calculation results. In order to verify the accuracy of root cause analysis, the system periodically performs simulation tests, and by introducing alarm events of known root causes, the capability and accuracy of the system for positioning the root causes are evaluated, and the root cause analysis algorithm is continuously optimized.

And S34, adjusting the alarm priority of the alarm items according to the dynamic weight allocation strategy, and if the probability value of the alarm items exceeds the preset root cause threshold value, increasing the priority of the corresponding alarm items according to the preset proportion to generate a comprehensive alarm list with priority.

Specifically, the dynamic weight allocation strategy comprehensively considers various factors such as an initial weight value, root cause probability, current network state and the like. The system first evaluates the probability value of the alert entry, and when the probability value exceeds a preset root cause threshold, it indicates that the alert event is likely to be caused by an important root cause, and the system will raise its priority according to a preset ratio. This process involves the recalculation and adjustment of the weight values, ensuring that high probability alert events have higher priority in the integrated alert list. The finally generated comprehensive alarm list is ordered according to the priority, so that a clear processing sequence is provided for a system administrator, and the key alarms can be timely responded and processed. When the system adjusts the alarm priority, a multi-factor weighted summation method is adopted, and a plurality of factors such as initial weight, root probability, network state and the like are comprehensively considered to calculate the final priority of each alarm item. Meanwhile, the system monitors and optimizes the dynamic weight distribution strategy in real time, and dynamically adjusts the weight distribution parameters according to the change of network environment and service requirements so as to improve the adaptability and effectiveness of the system. In order to ensure the rationality and accuracy of alarm priority adjustment, the system periodically evaluates the alarm processing effect, and continuously optimizes the dynamic weight allocation strategy by analyzing the timeliness and accuracy indexes of alarm processing, thereby improving the overall performance and reliability of the system.

In one embodiment, the S4 method for monitoring and alarming text data transmitted by multiple protocols specifically includes the following steps:

s41, sorting the comprehensive alarm list from high to low according to priority, screening alarms with priority exceeding a preset alarm triggering threshold, and generating abnormal early warning information and normal early warning information, wherein the abnormal early warning information comprises alarm types, file paths and triggering time of alarms exceeding the preset alarm triggering threshold.

Specifically, the interactive interfacing with the outside world in the process of text data monitoring and alarming of multi-protocol transmission is shown in fig. 3. The external data source generates networking data through the mine monitoring subsystem, which is transmitted to the multi-protocol transmission monitoring program via the mine monitoring subsystem host. The multiprotocol transmission monitor is responsible for collecting and analyzing text data from various protocols such as FTP, SAMBA, kafka and converting it into a structured format for storage in the shared memory buffer. The system then performs multidimensional monitoring and analysis on the structured data, including network connection status, data backlog, content anomaly detection, and the like. Various events generated in the monitoring process are integrated into an initial event queue, and a comprehensive alarm list with priority is finally formed through standardized processing and causal relation analysis. The system interacts with the outside through the cloud service platform, ensures the security of data transmission by utilizing the internet/VNP technology, and protects the internal network from external threat through a firewall. The cloud service platform not only receives normal early warning information pushed by the system, but also supports a user to receive abnormal early warning reminding through various modes such as small programs, cloud phones, APP and the like. In addition, the system exchanges data with the national bureau data acquisition front-end system and the provincial bureau data transmission front-end system, and data reporting and synchronization are realized. In the improved flow, the system acquires the unique machine code of the equipment through the hardware fingerprint encryption module and completes the equipment authorization verification, so that only authorized equipment can be ensured to be accessed into the system, and the data security and the system reliability are enhanced. Meanwhile, the system performs extension matching and content extraction on the text file in the local path in the data acquisition stage, analyzes key fields in the screened text file through the regular expression, generates structured data, and improves the accuracy and efficiency of data acquisition. Furthermore, the system optimizes the event processing and the alarm priority determination through a dynamic weight distribution strategy and a Bayesian network algorithm, so that the alarm information can be more timely and accurately transmitted to the user, and the overall performance and the user experience of the system are improved.

Specifically, the system strictly sorts the comprehensive alarm list, and the comprehensive alarm list is orderly arranged from high to low according to the alarm priority. The process adopts a multi-field comparison mechanism, and when the priorities are the same, key factors such as the time stamp, the influence range and the like of the alarm event are comprehensively considered so as to ensure the rationality of the sequencing result. The system compares the ordered alarm lists item by item through a preset alarm triggering threshold screening mechanism. For alarms with priority exceeding the preset alarm triggering threshold, the system classifies the alarms into abnormal early warning information and normal early warning information in detail. The abnormal early warning information precisely comprises key elements such as warning types, specific file paths, triggering time accurate to millisecond level and the like, while the normal early warning information is relatively simple, and the warning types and the triggering time are mainly recorded. When the system generates the information, a special data packaging interface is called, so that the uniformity and standardization of the information format are ensured, and the subsequent processing module can quickly identify and analyze.

S42, processing the abnormal early warning information, generating an alarm triggering signal, pushing the normal early warning information to the cloud platform through the HTTPS protocol interface, and controlling the mobile terminal to execute popup prompt and voice broadcasting by the alarm triggering signal.

Specifically, the popup window prompt adopts a multi-level priority display strategy, so that high-priority alarms can break through the conventional information display limit, and the user is attracted in the first time. And selecting a proper voice synthesis template according to the alarm type and the priority level to clearly and concisely transmit key alarm information to a user by voice broadcasting. And meanwhile, the system pushes the normal early warning information to the cloud platform through a safe HTTPS protocol interface. In the pushing process, the system can encrypt the data and add necessary identity verification information to ensure the safety and the integrity of data transmission. The application of the HTTPS protocol not only ensures the confidentiality of information in the transmission process, but also effectively prevents security threats such as man-in-the-middle attacks and the like by means of digital certificate verification and the like, and constructs a solid security barrier for the alarm information transmission of the system.

S43, extracting corresponding structured data from the shared memory buffer area according to the file path field in the abnormal early warning information, generating a data snapshot, binding the data snapshot with the abnormal early warning information, and generating an alarm log.

Specifically, the shared memory buffer area adopts a high-efficiency memory management algorithm, supports high-concurrency read-write operation, and can quickly respond to a data extraction request on the premise of ensuring data consistency. After the system extracts data from the buffer, the data state at the current moment is completely captured by utilizing a data snapshot generation technology, and a detailed data snapshot is generated. The snapshot not only contains basic data content, but also records metadata such as generation time of data, associated alarm information and the like. And then, the system carries out deep binding on the data snapshot and the abnormal early warning information to generate a complete warning log. The process is completed through a special log management interface, and the log record adopts standard format specification, so that the uniformity and the readability of the log data are ensured. After the alarm log is generated, the alarm log is stored in a high-reliability storage system, and a log index updating mechanism is triggered at the same time, so that historical alarm data can be quickly queried and analyzed later.

S44, receiving user feedback data submitted by a user through an interactive interface of the cloud platform, updating a protocol weight value in the dynamic weight distribution strategy according to a preset attenuation rule and an enhancement rule, and generating an updated dynamic weight distribution strategy.

Specifically, the system first pre-processes the feedback data, including data cleansing, format conversion, etc., to ensure the quality and usability of the data. And then, the system accurately adjusts the protocol weight value in the dynamic weight distribution strategy according to a preset attenuation rule and an enhancement rule. The attenuation rules are mainly used for reducing protocol weights corresponding to frequent false alarms or low-value alarms, while the enhancement rules are mainly used for improving the alarm protocol weights capable of accurately reflecting key problems. The process is realized through a machine learning algorithm, and the system trains a model according to historical feedback data and dynamically optimizes a weight adjustment strategy. The updated dynamic weight allocation strategy can take effect in real time, and influence the priority evaluation and processing flow of the subsequent alarm event, so that the self-adaptive optimization of the system is realized, the accuracy and efficiency of alarm processing are continuously improved, and the system can better meet the actual demands of users.

Preferably, the interaction and data flow of the components of the system presents a high degree of coordination and complexity throughout the text data monitoring and alert process of the multi-protocol transmission. As shown in fig. 4, the system first receives text data from various protocols such as FTP, SAMBA, kafka through a multi-source protocol transport instance. After collection, analysis and preliminary processing, the data are converted into a structured format and stored in a shared memory buffer area. The system then performs multidimensional monitoring and analysis on the structured data, including network connection status, data backlog, content anomaly detection, and the like. Various events generated in the monitoring process are integrated into an initial event queue, and a comprehensive alarm list with priority is finally formed through standardized processing and causal relation analysis. The series of processes are efficiently and orderly carried out in the system, and each link from data access to alarm generation can be tightly connected, so that the accurate monitoring and management of the whole data transmission process are realized. In the improved flow, the system optimizes event handling and alarm priority determination through dynamic weight distribution policies and bayesian network algorithms. Specifically, the system acquires an initial weight value from a preset weight configuration table according to the protocol type of the multi-source protocol transmission example, performs conditional probability calculation on the protocol type, the time interval and the network state field in the historical alarm log based on a Bayesian network algorithm, learns the causal relationship between network congestion, equipment failure and protocol alarm events, and builds an alarm dependency graph. Further, the system calculates root cause probability of the transmission state monitoring event and the content abnormal event in the monitoring event queue based on the alarm dependency graph, positions root cause nodes through a Markov blanket algorithm, and generates alarm items with probability values. Finally, the system adjusts the alarm priority of the alarm items according to the dynamic weight allocation strategy, if the probability value of the alarm items exceeds the preset root cause threshold value, the priority of the corresponding alarm items is increased according to the preset proportion, and a comprehensive alarm list with priority is generated. The improved flow obviously improves the accuracy and timeliness of the alarm, so that the system can more effectively cope with complex network environments and variable data transmission requirements. "

Preferably, as shown in fig. 5, the present invention provides a multi-protocol transmitted text data monitoring and alerting system 600 configured with the following modules:

the multi-protocol adaptation and data preprocessing module 610 is configured to perform instantiation processing on FTP, SAMBA, kafka protocols based on a preset multi-protocol configuration table, extract a multi-source protocol transmission example;

The transmission state and content anomaly monitoring module 620 is configured to monitor a network connection state and a data backlog amount of a multi-source protocol transmission instance through multiple threads, and simultaneously analyze text content key parameters of structured data to generate a monitoring event queue including a transmission state monitoring event and a content anomaly event, where the transmission state monitoring event is used to indicate that the network connection is interrupted and the number of file backlogs exceeds a preset threshold, and the content anomaly event is used to indicate that a sensor value exceeds the preset content threshold;

The root cause analysis and prioritization module 630 is configured to perform alarm root cause analysis on the transmission state monitoring event and the content abnormal event of the monitoring event queue based on a preset dynamic weight allocation policy and a bayesian network algorithm, and generate a comprehensive alarm list with priority;

The early warning feedback and policy optimization module 640 is configured to perform abnormal early warning triggering judgment on the high priority warning of the comprehensive warning list based on the priority ranking of the comprehensive warning list, generate abnormal early warning information, perform data association on the abnormal early warning information and the structured data, generate a warning log, send the warning log to the cloud platform, and update the protocol priority parameters of the dynamic weight allocation policy based on the user feedback data fed back by the cloud platform, so as to generate an updated dynamic weight allocation policy.

In summary, the text data monitoring and alarming system for multi-protocol transmission provided by the invention can realize the secure instantiation and data acquisition of FTP, SAMBA, kafka hybrid transmission protocol in the industrial Internet of things scene through the multi-protocol adaptation and dynamic security check mechanism. The method is based on hardware fingerprint encryption to generate a unique machine code of the equipment and combines with protocol factory dynamic creation examples, can achieve dual protection of unauthorized equipment interception and protocol channel encryption, effectively solve the problem of data leakage risk caused by lack of hardware-level security check in traditional multi-protocol transmission, can achieve full-dimension real-time detection of anomalies of a transmission layer and a content layer by multi-thread parallel monitoring of network connection states, data backlog amounts and sensor numerical content parameters, and can accurately generate multi-type events such as network disconnection, file backlog overrun and sensor numerical value overrun by means of a heartbeat packet detection mechanism, a dynamic backlog threshold adjustment algorithm and keyword matching rules, and remarkably improve anomaly coverage rate and detection timeliness in complex transmission scenes.

Preferably, the multi-protocol adaptation and data preprocessing module 610 is configured with the following elements:

The device authorization and configuration activation unit is used for acquiring the serial number of the main board, the identification of the CPU chip and the physical address of the network card through the hardware fingerprint encryption module, generating a unique machine code of the device, completing the device authorization verification based on the machine code, and activating a multi-protocol configuration table on the authorization device;

The multi-source protocol transmission example extracting unit is used for calling the protocol factory class to create an FTP protocol example, a SAMBA protocol example and a Kafka protocol example based on the server address, the port number and the key information recorded in the multi-protocol configuration table, respectively operating the FTP protocol example, the SAMBA protocol example and the Kafka protocol example on independent threads, and extracting a final multi-source protocol transmission example;

The structured data generation unit is used for carrying out extension matching on text files in the local path, filtering non-text files, screening and extracting contents of text files with extension names of TXT, XML and DOC, and analyzing sensor numbers, values and timestamp fields in the screened text files through a regular expression to generate structured data;

The marking data generating unit is used for storing the structured data into the shared memory buffer area, marking the structured data as an unprocessed state and generating the structured data marked as unprocessed.

Preferably, the transmission state and content abnormality monitoring module 620 is configured with the following units:

The network disconnection event generating unit is used for sending a heartbeat packet to the multi-source protocol transmission example based on a heartbeat packet detection mechanism, counting the failure times of receiving the response signal, and generating a network disconnection event which is a connection interruption state of the continuous unresponsive heartbeat packet of the protocol example if the response signal is not received for three times continuously;

The backlog alarm event generation unit is used for counting the number of the local files to be transmitted and the number of backlog files at the server side of each protocol instance in the multi-source protocol transmission instance, judging whether the number of the local files to be transmitted exceeds a preset backlog threshold, if so, generating a backlog alarm event, wherein the preset backlog threshold is the maximum file accumulation number dynamically adjusted according to the storage capacity of the server;

The content alarm event generation unit is used for extracting a current sensing value from a sensor value field of the structured data, judging whether the current sensing value exceeds a preset content threshold value, and if so, generating a content alarm event comprising a sensor number, the current value and the preset threshold value;

and the monitoring event queue generating unit is used for combining the network disconnection event, the backlog alarm event and the content alarm event to generate a monitoring event queue containing a transmission state monitoring event and the content abnormal event, wherein the transmission state monitoring event comprises the network disconnection event and the backlog alarm event.

Preferably, the content alarm event generating unit comprises a plaintext data stream generating subunit, a key field parsing and matching subunit, an initial alarm event generating subunit and an alarm event generating storage subunit. The system comprises a plaintext data stream generation subunit, a key field analysis and matching subunit, an initial alarm event generation subunit, an alarm event generation storage subunit, an alarm event generation subunit, a data integrity check code generation subunit and an alarm database, wherein the plaintext data stream generation subunit is used for extracting encrypted text data transmitted by a SAMBA protocol from structured data, executing a stream decryption algorithm through a hardware acceleration module to generate a plaintext data stream, the key field analysis and matching subunit is used for carrying out key field analysis on the plaintext data stream, extracting a sensor number and an associated numerical value field and matching data content based on a preset keyword set, the initial alarm event generation subunit is used for judging whether the associated numerical value field exceeds a preset content threshold, if so, generating an initial alarm event, the initial alarm event comprises a sensor number, a current numerical value and the matching keyword, the alarm event generation storage subunit is used for carrying out block processing on the plaintext data stream, calculating hash values of data blocks based on an SHA-256 algorithm to generate the data integrity check code, binding the data integrity check code with the initial alarm event, generating a stored content log alarm event and storing the stored content alarm event in the alarm database.

Preferably, the monitor event queue generating unit includes an initial event queue generating subunit, a standard event queue generating subunit, a high-frequency abnormal event extracting subunit, and a final monitor event queue generating subunit. The system comprises an initial event queue generating subunit, a standard event queue generating subunit, a high-frequency abnormal event extracting subunit, a final monitoring event queue generating subunit and a final monitoring event queue generating subunit, wherein the initial event queue generating subunit is used for merging a network disconnection event, a backlog alarm event and a content abnormal event to generate an initial event queue containing a transmission state monitoring event and a content abnormal event, the standard event queue generating subunit is used for carrying out standardized processing on timestamps in the initial event queue, eliminating clock deviation among different protocol instances to generate a standard event queue with standardized timestamps, the high-frequency abnormal event extracting subunit is used for counting the alarm triggering times of the same protocol instance in the standard event queue based on a preset time window threshold, and extracting corresponding alarm events from the standard event queue and marking the corresponding alarm events as high-frequency abnormal events if the alarm triggering times exceed the preset high-frequency threshold.

Preferably, root cause analysis and prioritization module 630 is configured with the following elements:

The initial weight value acquisition unit is used for acquiring an initial weight value from a preset weight configuration table according to the protocol type of the multi-source protocol transmission example;

The alarm dependency graph construction unit is used for carrying out conditional probability calculation on the protocol type, the time interval and the network state field in the historical alarm log based on a Bayesian network algorithm, learning the causal relationship between network congestion, equipment faults and protocol alarm events and constructing an alarm dependency graph;

The alarm item generating unit with probability value is used for carrying out root cause probability calculation on the transmission state monitoring event and the content abnormal event in the monitoring event queue based on the alarm dependency graph, positioning root cause nodes through a Markov blanket algorithm and generating an alarm item with probability value;

And the comprehensive alarm list generation unit is used for adjusting the alarm priority of the alarm items according to the dynamic weight allocation strategy, and if the probability value of the alarm items exceeds the preset root cause threshold value, the priority of the corresponding alarm items is increased according to the preset proportion, and the comprehensive alarm list with the priority is generated.

Preferably, the early warning feedback and strategy optimization module 640 is configured with the following elements:

the early warning information generation unit is used for sorting the comprehensive warning list from high to low according to priority, screening the warning with the priority exceeding the preset warning trigger threshold value, and generating abnormal early warning information and normal early warning information, wherein the abnormal early warning information comprises the warning type, the file path and the triggering time of the warning exceeding the preset warning trigger threshold value;

The signal pushing and prompting unit is used for processing the abnormal early warning information, generating an alarm triggering signal, pushing the normal early warning information to the cloud platform through the HTTPS protocol interface, and controlling the mobile terminal to execute popup prompt and voice broadcast;

the alarm log generation unit is used for extracting corresponding structured data from the shared memory buffer area according to the file path field in the abnormal early warning information, generating a data snapshot and binding the data snapshot with the abnormal early warning information to generate an alarm log;

The weight strategy updating unit is used for receiving user feedback data submitted by a user through an interactive interface of the cloud platform, updating a protocol weight value in the dynamic weight allocation strategy according to a preset attenuation rule and an enhancement rule, and generating an updated dynamic weight allocation strategy.

In one embodiment, the present application further provides a computer device, including a memory and a processor, where the memory stores a computer program, and the processor implements the above-mentioned text data monitoring and alerting method of multiprotocol transmission when executing the computer program.

In one embodiment, the present application further provides a computer readable storage medium having a computer program stored thereon, which when executed by a processor implements the above-described method for monitoring and alerting text data of multiprotocol transmission.

In the description of the present specification, a description referring to terms "one embodiment," "some embodiments," "examples," "specific examples," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the present application. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, the different embodiments or examples described in this specification and the features of the different embodiments or examples may be combined and combined by those skilled in the art without contradiction.

For the device embodiments, reference is made to the description of the method embodiments for the relevant points, since they essentially correspond to the method embodiments. The above-described apparatus embodiments are merely illustrative, wherein the components illustrated as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, may be located in one place, or may be distributed over a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the objectives of the disclosed solution. Those of ordinary skill in the art will understand and implement the present invention without undue burden.

The foregoing is merely illustrative of the present application, and the present application is not limited thereto, and any person skilled in the art will readily recognize that various changes and substitutions are possible within the scope of the present application. Therefore, the protection scope of the application is subject to the protection scope of the claims.

Claims

1. The text data monitoring and alarming method for multi-protocol transmission is characterized by comprising the following steps:

S2, monitoring the network connection state and the data backlog quantity of the multi-source protocol transmission example through multithreading, and simultaneously analyzing text content key parameters of the structured data to generate a monitoring event queue comprising transmission state monitoring events and content abnormal events;

The transmission state monitoring event is used for indicating network connection interruption and file backlog quantity to exceed a preset backlog threshold, and the content abnormal event is used for indicating the value of the sensor to exceed the preset content threshold;

S4, carrying out abnormal early warning triggering judgment on the high-priority warning of the comprehensive warning list based on the priority ordering of the comprehensive warning list, generating abnormal early warning information, carrying out data association on the abnormal early warning information and the structured data, generating a warning log and sending the warning log to a cloud platform, and simultaneously carrying out weight updating on the protocol priority parameters of the dynamic weight allocation strategy based on user feedback data fed back by the cloud platform, and generating an updated dynamic weight allocation strategy.

2. The method according to claim 1, wherein S1 comprises:

S11, acquiring a serial number of a main board, a CPU chip identifier and a network card physical address through a hardware fingerprint encryption module, generating a unique machine code of equipment, completing equipment authorization verification based on the unique machine code of the equipment, and activating a multi-protocol configuration table on the authorization equipment;

S12, based on the server address, port number and key information recorded in the multi-protocol configuration table, calling a protocol factory to create an FTP protocol instance, a SAMBA protocol instance and a Kafka protocol instance, respectively operating the FTP protocol instance, the SAMBA protocol instance and the Kafka protocol instance on independent threads, and extracting a final multi-source protocol transmission instance;

s13, performing extension matching on text files in a local path, filtering non-text files, screening and extracting contents of text files with extension of TXT, XML and DOC, and analyzing sensor numbers, values and timestamp fields in the screened text files through a regular expression to generate structured data;

3. The method according to claim 1, wherein S2 comprises:

S21, sending a heartbeat packet to the multi-source protocol transmission example based on a heartbeat packet detection mechanism, counting the failure times of receiving a response signal, and if the response signal is not received for three times continuously, generating a network disconnection event, wherein the network disconnection event is a connection interruption state of the continuous unresponsive heartbeat packet of the protocol example;

S22, counting the number of local files to be transmitted and the number of backlog files at a server side of each protocol instance in the multi-source protocol transmission instance, judging whether the number of the local files to be transmitted exceeds a preset backlog threshold, and if so, generating a backlog alarm event, wherein the preset backlog threshold is the maximum file accumulation number dynamically adjusted according to the storage capacity of the server;

S23, extracting a current sensing value from a sensor value field of the structured data, judging whether the current sensing value exceeds a preset content threshold, and if so, generating a content alarm event comprising a sensor number, the current value and the preset threshold;

4. A method according to claim 3, wherein S23 comprises:

S231, extracting encrypted text data transmitted by the SAMBA protocol from the structured data, and executing a stream decryption algorithm through a hardware acceleration module to generate a plaintext data stream;

s232, carrying out key field analysis on the plaintext data stream, extracting a sensor number and an associated numerical value field, and matching data content based on a preset key word set;

S233, judging whether the associated numerical value field exceeds a preset content threshold value, and if so, generating an initial alarm event, wherein the initial alarm event comprises a sensor number, a current numerical value and a matching keyword;

S234, the plaintext data stream is subjected to block processing, hash values of all data blocks are calculated based on an SHA-256 algorithm to generate a data integrity check code, the data integrity check code is bound with the initial alarm event, and a stored content alarm event is generated and stored in an alarm log database.

5. A method according to claim 3, wherein S24 comprises:

S241, merging the network disconnection event, the backlog alarm event and the content exception event to generate an initial event queue containing a transmission state monitoring event and the content exception event;

s242, performing standardization processing on the time stamp in the initial event queue, eliminating clock deviation among different protocol instances, and generating a standard event queue of the standardized time stamp;

S243, counting the alarm triggering times of the same protocol instance in the standard event queue based on a preset time window threshold, and if the alarm triggering times exceed a preset high-frequency threshold, extracting a corresponding alarm event from the standard event queue and marking the corresponding alarm event as a high-frequency abnormal event;

S244, merging the high-frequency abnormal event into the monitoring event queue of the standardized timestamp, and attaching a current network bandwidth occupancy rate tag to generate a final monitoring event queue.

6. The method according to claim 1, wherein S3 comprises:

s31, acquiring an initial weight value from a preset weight configuration table according to the protocol type of the multi-source protocol transmission example;

S32, carrying out conditional probability calculation on protocol types, time intervals and network state fields in a historical alarm log based on a Bayesian network algorithm, learning causal relations between network congestion, equipment faults and protocol alarm events, and constructing an alarm dependency graph;

S33, carrying out root cause probability calculation on the transmission state monitoring event and the content abnormal event in the monitoring event queue based on the alarm dependency graph, positioning root cause nodes through a Markov blanket algorithm, and generating alarm items with probability values;

And S34, adjusting the alarm priority of the alarm items according to the dynamic weight allocation strategy, and if the probability value of the alarm items exceeds a preset root cause threshold value, increasing the priority of the corresponding alarm items according to a preset proportion to generate a comprehensive alarm list with priority.

7. The method according to claim 1, wherein S4 comprises:

S41, sorting the comprehensive alarm list from high to low according to priority, screening alarms with priority exceeding a preset alarm triggering threshold, and generating abnormal early warning information and normal early warning information, wherein the abnormal early warning information comprises alarm types, file paths and triggering time of alarms exceeding the preset alarm triggering threshold;

S42, processing the abnormal early warning information, generating an alarm trigger signal, pushing the normal early warning information to a cloud platform through an HTTPS protocol interface, wherein the alarm trigger signal is used for controlling a mobile terminal to execute popup prompt and voice broadcasting;

s43, extracting the corresponding structured data from the shared memory buffer area according to the file path field in the abnormal early warning information, generating a data snapshot and binding the data snapshot with the abnormal early warning information to generate an alarm log;

8. A multi-protocol transmitted text data monitoring and alerting system, the system comprising:

The system comprises a transmission state and content anomaly monitoring module, a transmission state monitoring module and a content anomaly monitoring module, wherein the transmission state and content anomaly monitoring module is used for monitoring the network connection state and the data backlog of the multi-source protocol transmission example through multithreading, and simultaneously analyzing text content key parameters of the structured data to generate a monitoring event queue comprising transmission state monitoring events and content anomaly events;

the root cause analysis and priority ordering module is used for carrying out alarm root cause analysis on the transmission state monitoring event and the content abnormal event of the monitoring event queue based on a preset dynamic weight distribution strategy and a Bayesian network algorithm, and generating a comprehensive alarm list with priority;

The early warning feedback and strategy optimization module is used for carrying out abnormal early warning triggering judgment on the high-priority warning of the comprehensive warning list based on the priority ordering of the comprehensive warning list, generating abnormal early warning information, carrying out data association on the abnormal early warning information and the structured data, generating a warning log and sending the warning log to a cloud platform, and simultaneously carrying out weight update on the protocol priority parameters of the dynamic weight distribution strategy based on user feedback data fed back by the cloud platform, and generating an updated dynamic weight distribution strategy.

9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the method of any one of claims 1 to 7 when executing the computer program.

10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the method of any of claims 1 to 7.