CN114238402A - Alarm data processing method and device, storage medium and electronic equipment - Google Patents
Alarm data processing method and device, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN114238402A CN114238402A CN202111447628.2A CN202111447628A CN114238402A CN 114238402 A CN114238402 A CN 114238402A CN 202111447628 A CN202111447628 A CN 202111447628A CN 114238402 A CN114238402 A CN 114238402A
- Authority
- CN
- China
- Prior art keywords
- information
- target
- historical
- pieces
- screening
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/245—Query processing
- G06F16/2455—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/24—Querying
- G06F16/242—Query formulation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Mathematical Physics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- Testing And Monitoring For Control Systems (AREA)
Abstract
The application discloses an alarm data processing method, an alarm data processing device, a storage medium and electronic equipment, and relates to the field of financial science and technology. The method comprises the following steps: acquiring alarm data to be processed; screening the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target screening conditions to obtain target attribute information and target feature information; and inputting the target attribute information and the target characteristic information into the first model, and processing to obtain an alarm handling scheme corresponding to the alarm data to be processed. By the method and the device, the problem that an alarm processing method corresponding to the alarm data cannot be accurately obtained through a machine learning algorithm in the related technology is solved.
Description
Technical Field
The application relates to the field of financial science and technology, in particular to an alarm data processing method, an alarm data processing device, a storage medium and electronic equipment.
Background
In many scenarios, in order to process alarm data generated by each production unit in time to obtain a disposal plan corresponding to the alarm data, a machine learning algorithm is required to process the alarm data, so that an alarm disposal scheme is obtained through a machine learning model.
When the machine learning algorithm is used for processing the alarm data in the related art, the number and the types of the alarm data and the treatment plans are increased due to the increasing large types and scales of the devices. In the current method for processing alarm data by using a machine learning algorithm, an alarm disposal scheme cannot be accurately obtained for part of alarm data under part of conditions, and at the moment, the alarm disposal scheme needs to be determined manually, so that the work of operation and maintenance personnel is difficult, and hidden dangers are brought to safe and stable operation of production.
Aiming at the problem that an alarm processing method corresponding to alarm data cannot be accurately obtained through a machine learning algorithm in the related art, an effective solution is not provided at present.
Disclosure of Invention
The application provides an alarm data processing method, an alarm data processing device, a storage medium and electronic equipment, and aims to solve the problem that an alarm processing method corresponding to alarm data cannot be accurately obtained through a machine learning algorithm in the related art.
According to one aspect of the application, an alarm data processing method is provided. The method comprises the following steps: acquiring alarm data to be processed, wherein the alarm data to be processed comprises a plurality of pieces of attribute information and a plurality of pieces of characteristic information; screening the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target screening conditions to obtain target attribute information and target feature information; and inputting the target attribute information and the target characteristic information into a first model, and processing to obtain an alarm handling scheme corresponding to alarm data to be processed, wherein the first model is obtained by training a plurality of groups of first sample data, and each group of first sample data comprises all attribute information, all characteristic information and alarm results corresponding to the sample alarm data of the sample alarm data.
Optionally, the screening the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target screening condition, and the obtaining the target attribute information and the target feature information includes: screening the plurality of pieces of attribute information according to a first screening condition to obtain target attribute information; screening the plurality of pieces of feature information according to a second screening condition to obtain initial feature information; segmenting each piece of initial characteristic information according to fields to obtain a plurality of pieces of field information corresponding to the initial characteristic information; and screening the plurality of field information according to a third screening condition to obtain a plurality of target field information, and determining the plurality of target field information as target characteristic information.
Optionally, before the multiple pieces of attribute information and the multiple pieces of feature information are filtered according to the target filtering condition to obtain the target attribute information and the target feature information, the method further includes: acquiring a plurality of historical alarm data and a historical alarm result corresponding to each historical alarm data, wherein each historical alarm data comprises a plurality of pieces of historical attribute information and a plurality of pieces of historical characteristic information; screening a plurality of pieces of historical attribute information and a plurality of pieces of historical characteristic information in each piece of historical alarm data according to preset screening conditions to obtain historical target attribute information and historical target characteristic information; determining the prediction accuracy of the first model according to the historical target attribute information and the historical target characteristic information of the plurality of historical alarm data; under the condition that the prediction accuracy of the first model is smaller than the accuracy condition, adjusting the preset screening condition, re-executing the step of screening the plurality of pieces of historical attribute information and the plurality of pieces of historical characteristic information in each piece of historical alarm data according to the preset screening condition to obtain the historical target attribute information and the historical target characteristic information, and the step of determining the prediction accuracy of the first model according to the historical target attribute information and the historical target characteristic information of the plurality of pieces of historical alarm data until the prediction accuracy of the first model is larger than or equal to the accuracy condition; and determining the adjusted preset screening condition as a target screening condition.
Optionally, the determining the prediction accuracy of the first model according to the historical target attribute information and the historical target feature information of the plurality of historical alarm data includes: respectively inputting the historical target attribute information and the historical target characteristic information of each historical alarm data into a first model, and processing to obtain a predicted alarm result corresponding to each historical alarm data; respectively comparing the historical alarm result corresponding to each historical alarm data with the corresponding predicted alarm result to obtain a plurality of comparison results; and determining the prediction accuracy of the first model according to the comparison results.
Optionally, after the multiple pieces of attribute information and the multiple pieces of feature information are filtered according to the target filtering condition to obtain the target attribute information and the target feature information, the method further includes: converting each character in each target characteristic information according to a preset conversion rule to obtain character string information; and inputting the character string information into a second model for processing to obtain standard character string information, and determining the standard character string information as updated target characteristic information, wherein the number in the standard character string information is positioned in a preset interval.
Optionally, before the character string information is input into the second model for processing, so as to obtain standard character string information, the method further includes: obtaining a plurality of sample character string information, converting each sample character string according to a preset interval conversion rule to obtain converted digital information, and combining the sample character strings and the converted digital information to determine second sample data to obtain a plurality of second sample data; and training the preset model through a plurality of second sample data to obtain a second model.
According to another aspect of the present application, an alert data processing apparatus is provided. The device includes: the device comprises a first acquisition unit, a second acquisition unit and a processing unit, wherein the first acquisition unit is used for acquiring alarm data to be processed, and the alarm data to be processed comprises a plurality of pieces of attribute information and a plurality of pieces of characteristic information; the first screening unit is used for screening the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target screening conditions to obtain target attribute information and target feature information; the first input unit is used for inputting the target attribute information and the target characteristic information into a first model and processing the target attribute information and the target characteristic information to obtain an alarm handling scheme corresponding to alarm data to be processed, wherein the first model is obtained by training a plurality of groups of first sample data, and each group of first sample data comprises all attribute information, all characteristic information and alarm results corresponding to the sample alarm data.
Optionally, the first screening unit comprises: the first screening module is used for screening the plurality of pieces of attribute information according to a first screening condition to obtain target attribute information; the second screening module is used for screening the plurality of pieces of feature information according to a second screening condition to obtain initial feature information; the first segmentation module is used for segmenting each piece of initial characteristic information according to fields to obtain a plurality of pieces of field information corresponding to the initial characteristic information; and the third screening module is used for screening the plurality of field information according to a third screening condition to obtain a plurality of target field information, and determining the plurality of target field information as target characteristic information.
According to another aspect of the embodiments of the present invention, a non-volatile storage medium is further provided, where the non-volatile storage medium includes a stored program, and the program controls, when running, a device in which the non-volatile storage medium is located to execute an alarm data processing method.
According to another aspect of the embodiments of the present invention, there is also provided an electronic device, including one or more processors and a memory; the memory stores computer readable instructions, and the processor is used for executing the computer readable instructions, wherein the computer readable instructions execute an alarm data processing method.
Through the application, the following steps are adopted: acquiring alarm data to be processed, wherein the alarm data to be processed comprises a plurality of pieces of attribute information and a plurality of pieces of characteristic information; screening the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target screening conditions to obtain target attribute information and target feature information; and inputting the target attribute information and the target characteristic information into a first model, and processing to obtain an alarm handling scheme corresponding to alarm data to be processed, wherein the first model is obtained by training a plurality of groups of first sample data, and each group of first sample data comprises all attribute information, all characteristic information and alarm results corresponding to the sample alarm data of the sample alarm data. The problem that an alarm processing method corresponding to alarm data cannot be accurately obtained through a machine learning algorithm in the related art is solved. By splitting the alarm data, target characteristic information and target attribute information which can be used for determining an alarm result are selected, and the alarm data are preprocessed to obtain the alarm data which can be more easily identified and judged by a machine learning model, so that the effect of improving the accuracy of the alarm result obtained by machine learning is achieved.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:
FIG. 1 is a flow chart of an alarm data processing method provided according to an embodiment of the present application;
FIG. 2 is a schematic diagram of an alarm data processing apparatus provided in accordance with an embodiment of the present application;
fig. 3 is a schematic view of an electronic device according to an embodiment of the present application.
Detailed Description
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It should be noted that, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for presentation, analyzed data, etc.) referred to in the present disclosure are information and data authorized by the user or sufficiently authorized by each party.
It should be noted that the alarm data processing method, the alarm data processing apparatus, the storage medium, and the electronic device determined in the present disclosure may be used in the field of financial technology, and may also be used in any field other than the field of financial technology.
According to an embodiment of the application, an alarm data processing method is provided.
Fig. 1 is a flowchart of an alarm data processing method according to an embodiment of the present application. As shown in fig. 1, the method comprises the steps of:
step S101, alarm data to be processed is obtained, wherein the alarm data to be processed comprises a plurality of pieces of attribute information and a plurality of pieces of characteristic information.
Specifically, the attribute information may be inherent information corresponding to the alarm data, such as device ID, device IP, alarm occurrence time, and the like corresponding to the alarm data, and the characteristic information may be information representing alarm content of the alarm data, such as alarm type, alarm content, and the like. After the system acquires the alarm data to be processed, an alarm result corresponding to the alarm data to be processed, namely a treatment plan corresponding to the alarm data to be processed, is determined by identifying attribute information and characteristic information in the alarm data.
For example, a Kafka system acquires real-time alarm data to be processed, sends the alarm data to be processed to a Flink system for acquiring and processing attribute information and feature information of the alarm data, and sends the processed alarm data to a machine learning model for processing to obtain a disposal plan, wherein Kafka is a high-throughput distributed publish-subscribe message system capable of processing action stream data, and Flink is a distributed processing engine system for stream data and batch data.
And S102, screening the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target screening conditions to obtain target attribute information and target feature information.
Specifically, the target screening condition may be set when the machine learning model is trained, and the target screening condition is used to screen out, from the plurality of pieces of attribute feature information and the plurality of pieces of feature information, target attribute information and target feature information that enable the treatment plan output by the machine learning model to have higher accuracy.
For example, the target screening conditions obtained after the machine learning model training are information such as equipment, platform, middleware, network and the like generating alarms and equipment IP and the like, the characteristic information is screened according to the target screening conditions, the rest information is deleted, the information such as the platform, the middleware, the network and the like is set as target characteristic information, the information such as the equipment, the equipment IP and the like generating alarms is set as target attribute data, and the effects of removing unnecessary information and improving the data processing efficiency are achieved.
Step S103, inputting the target attribute information and the target characteristic information into a first model, and processing to obtain an alarm handling scheme corresponding to alarm data to be processed, wherein the first model is obtained by training a plurality of groups of first sample data, and each group of first sample data comprises all attribute information, all characteristic information and alarm results corresponding to the sample alarm data.
Specifically, the first model may be a machine learning model, and the target attribute information and the target feature information are input to the machine learning model to generate a disposal plan, so as to obtain the disposal plan corresponding to the to-be-processed alarm data. Wherein, the machine learning model is a trained machine learning model.
It should be noted that, the first sample data is the historical alarm data and the historical disposal plan corresponding to the historical alarm data, and after the historical alarm data is input into the machine learning model, because the process of further optimizing the machine learning model is complex and the steps are complex, the data can be screened and processed by setting the target screening conditions, and the processed alarm data is input into the machine learning model, thereby achieving the effect of improving the success rate of predicting the disposal plan.
According to the alarm data processing method provided by the embodiment of the application, alarm data to be processed is obtained, wherein the alarm data to be processed comprises a plurality of pieces of attribute information and a plurality of pieces of characteristic information; screening the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target screening conditions to obtain target attribute information and target feature information; and inputting the target attribute information and the target characteristic information into a first model, and processing to obtain an alarm handling scheme corresponding to alarm data to be processed, wherein the first model is obtained by training a plurality of groups of first sample data, and each group of first sample data comprises all attribute information, all characteristic information and alarm results corresponding to the sample alarm data of the sample alarm data. The problem that an alarm processing method corresponding to alarm data cannot be accurately obtained through a machine learning algorithm in the related art is solved. By splitting the alarm data, target characteristic information and target attribute information which can be used for determining an alarm result are selected, and the alarm data are preprocessed to obtain the alarm data which can be more easily identified and judged by a machine learning model, so that the effect of improving the accuracy of the alarm result obtained by machine learning is achieved.
Optionally, in the alarm data processing method provided in the embodiment of the present application, the screening the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target screening condition, and obtaining the target attribute information and the target feature information includes: screening the plurality of pieces of attribute information according to a first screening condition to obtain target attribute information; screening the plurality of pieces of feature information according to a second screening condition to obtain initial feature information; segmenting each piece of initial characteristic information according to fields to obtain a plurality of pieces of field information corresponding to the initial characteristic information; and screening the plurality of field information according to a third screening condition to obtain a plurality of target field information, and determining the plurality of target field information as target characteristic information.
Specifically, the target screening conditions include a first screening condition, a second screening condition, and a third screening condition, where the first screening condition may be used to screen target attribute information from alarm data to be processed, the second screening condition may be used to screen initial characteristic information from the alarm data to be processed, for example, to screen target attribute information of an alarm device, a platform, and the like from attribute information of an alarm server IP, a campus, an alarm device, a platform, middleware, a network, and the like, and to screen initial characteristic information of an alarm type, a resource domain, a CPU usage rate, a memory usage rate, a network bandwidth usage rate, and the like from the alarm server IP, the campus, the alarm device, the platform, the middleware, the network, and the like.
For example, the initial feature information "alarm type" is segmented according to fields to obtain field information of "alarm type name is overtime", "alarm reason is time exceeding threshold", and "threshold is 2345 ms", and field information required by the machine learning model is screened out according to a third screening condition, and the field information obtained after screening is set as target feature information. By screening the attribute information and the characteristic information in the alarm data through the embodiment, the effects of reducing the data volume and improving the prediction accuracy of the machine learning model are achieved.
Optionally, in the alarm data processing method provided in the embodiment of the present application, before the multiple pieces of attribute information and the multiple pieces of feature information are filtered according to the target filtering condition to obtain the target attribute information and the target feature information, the method further includes: acquiring a plurality of historical alarm data and a historical alarm result corresponding to each historical alarm data, wherein each historical alarm data comprises a plurality of pieces of historical attribute information and a plurality of pieces of historical characteristic information; screening a plurality of pieces of historical attribute information and a plurality of pieces of historical characteristic information in each piece of historical alarm data according to preset screening conditions to obtain historical target attribute information and historical target characteristic information; determining the prediction accuracy of the first model according to the historical target attribute information and the historical target characteristic information of the plurality of historical alarm data; under the condition that the prediction accuracy of the first model is smaller than the accuracy condition, adjusting the preset screening condition, re-executing the step of screening the plurality of pieces of historical attribute information and the plurality of pieces of historical characteristic information in each piece of historical alarm data according to the preset screening condition to obtain the historical target attribute information and the historical target characteristic information, and the step of determining the prediction accuracy of the first model according to the historical target attribute information and the historical target characteristic information of the plurality of pieces of historical alarm data until the prediction accuracy of the first model is larger than or equal to the accuracy condition; and determining the adjusted preset screening condition as a target screening condition.
Specifically, before using the machine learning model, training the machine learning model is required, first, historical alarm data and a corresponding historical disposal plan may be obtained from a database, for example, historical data is obtained from an elasticsearch database, the elasticsearch is a Lucene-based search server, data in the elasticsearch may be searched and used in real time in cloud computing, preset screening conditions are manually set, a plurality of pieces of historical attribute information and a plurality of pieces of historical feature information in the historical alarm data are screened through the preset screening conditions, the historical target attribute information and the historical target feature information are standardized after screening, the processed historical target attribute information and the historical target feature information are sent to the machine learning model to obtain the plan, and the disposal plan is compared with the historical disposal plan corresponding to the historical alarm data, and judging whether the two are consistent or not to obtain a result of judging whether the two are correct or not.
It should be noted that after the machine learning model processes the plurality of pieces of historical alarm data and the corresponding historical disposal plans, a plurality of comparison results are obtained, the prediction accuracy of the machine learning model under the preset screening condition is calculated according to the plurality of comparison results, the prediction accuracy is compared with the preset accuracy condition, the preset screening condition is modified under the condition that the preset accuracy condition is not met, the machine learning model training process is repeated until the prediction accuracy is greater than or equal to the preset accuracy condition, and the preset screening condition at the moment is set as the target screening condition.
For example, the preset accuracy condition is that the accuracy reaches 90%, and after training is performed using sample data, the obtained prediction accuracy is 70%, then training is performed again, and when the obtained prediction accuracy is 90%, the preset screening condition at this time may be set as the target screening condition, and meanwhile, training may also be continued to obtain the optimal target screening condition. According to the method and the device, the target screening conditions are obtained, so that the important information can be screened more accurately, and the prediction accuracy of the machine learning model is improved.
Optionally, in the alarm data processing method provided in the embodiment of the present application, determining the prediction accuracy of the first model according to the historical target attribute information and the historical target feature information of the plurality of historical alarm data includes: respectively inputting the historical target attribute information and the historical target characteristic information of each historical alarm data into a first model, and processing to obtain a predicted alarm result corresponding to each historical alarm data; respectively comparing the historical alarm result corresponding to each historical alarm data with the corresponding predicted alarm result to obtain a plurality of comparison results; and determining the prediction accuracy of the first model according to the comparison results.
Specifically, a plurality of groups of historical alarm data and treatment plans corresponding to the historical alarm data are input into a machine learning model, a plurality of comparison results are obtained after the machine learning model processes a plurality of pieces of historical alarm data and corresponding historical treatment plans, the prediction accuracy of the machine learning model under the preset screening condition is calculated according to the plurality of comparison results, and the prediction accuracy is compared with the preset accuracy condition. For example, if 100 sets of sample data are input into the machine learning model and 70 sets of sample data successfully obtain the correct treatment plan, the prediction accuracy is 70%. According to the method and the device, the prediction result of the machine learning model is obtained, and the effect of obtaining the optimal target screening information after the prediction result is obtained for multiple times on the basis is achieved.
Optionally, in the alarm data processing method provided in the embodiment of the present application, after the multiple pieces of attribute information and the multiple pieces of feature information are screened according to the target screening condition to obtain the target attribute information and the target feature information, the method further includes: converting each character in each target characteristic information according to a preset conversion rule to obtain character string information; and inputting the character string information into a second model for processing to obtain standard character string information, and determining the standard character string information as updated target characteristic information, wherein the number in the standard character string information is positioned in a preset interval.
Specifically, after obtaining the target characteristic information, the Flink classifies each field in the target characteristic information, can classify according to numbers, Chinese and English, and changes the Chinese and English fields through a conversion program to obtain preset character string information which enables a machine learning model to identify more accurately, wherein the conversion program can be an OneHot Encoding program.
It should be noted that after the character string information is obtained, the field information and the character string information of the number type need to be input into the second model for digital interval processing, so as to obtain the character string information and the field information in the preset interval, for example, the second model may be a minmaxscale interval scaling model in a Flink system, an error value statement of "2345 ms" is input into the minmaxscale for interval scaling, and a large value scaling is put into an interval of 0 to 1, so that it is avoided that a large value causes an excessive data weight to affect a machine learning model for judgment, and after the interval scaling is performed, the output may be obtained: results of "error value": 0.002345 "and input the results into the machine learning model.
For example, the obtained target feature information and target attribute information are in the following styles:
“ip”:“76.48.65.2”,
“id”:“xxx”,
“type”:“network”;
“error”:“timeout”;
"error type": "super-threshold";
“error value”:“2345ms”;
“ok value”:“200ms”;
and screening the target characteristic information in the alarm data to obtain:
“type”:“network”;
“error”:“timeout”;
"error type": "super-threshold";
“error value”:“2345ms”;
“ok value”:“200ms”;
standardizing the initial characteristic data to obtain target characteristic data:
“type”:“00001”;
“error”:“00000001000”;
“error type”:“1000”;
“error value”:“0.002345”;
“ok value”:“0.0002”;
the target characteristic information and the target attribute information can be sent to a machine learning model through a Flink system to generate a treatment plan. In the embodiment, the effect of better identifying the alarm data by the machine learning model is achieved by changing and zooming the fields in the target characteristic information.
Optionally, in the alarm data processing method provided in the embodiment of the present application, before the character string information is input into the second model for processing to obtain the standard character string information, the method further includes: obtaining a plurality of sample character string information, converting each sample character string according to a preset interval conversion rule to obtain converted digital information, and combining the sample character strings and the converted digital information to determine second sample data to obtain a plurality of second sample data; and training the preset model through a plurality of second sample data to obtain a second model.
Specifically, the second model is used for carrying out interval scaling on the data, and the situation that the machine learning model is influenced to judge due to the fact that the data weight is too large due to the fact that the numerical value is large is prevented. Before the second model is used, the model needs to be trained, digital information in a plurality of sample character strings is extracted, a preset interval conversion rule is manually set, the digital information is converted, all the digital information is in the interval range, the digital information after each sample character string information and the corresponding interval conversion are combined to obtain sample data, the sample data is input into the second model for model training, and the trained second model is obtained through multiple times of model parameter adjustment. This embodiment has reached training second model to through the more accurate effect of carrying out interval classification of second model.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.
The embodiment of the present application further provides an alarm data processing apparatus, and it should be noted that the alarm data processing apparatus according to the embodiment of the present application may be used to execute the alarm data processing method according to the embodiment of the present application. The following describes an alarm data processing apparatus provided in an embodiment of the present application.
Fig. 2 is a schematic diagram of an alarm data processing apparatus according to an embodiment of the present application. As shown in fig. 2, the apparatus includes:
the first acquiring unit 10 is configured to acquire alarm data to be processed, where the alarm data to be processed includes a plurality of pieces of attribute information and a plurality of pieces of feature information;
the first screening unit 20 is configured to screen the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target screening condition to obtain target attribute information and target feature information;
the first input unit 30 is configured to input the target attribute information and the target feature information into a first model, and process the target attribute information and the target feature information to obtain an alarm handling scheme corresponding to alarm data to be processed, where the first model is obtained by training multiple groups of first sample data, and each group of the first sample data includes all attribute information, all feature information, and an alarm result corresponding to the sample alarm data of the sample alarm data.
The alarm data processing apparatus provided in the embodiment of the application acquires alarm data to be processed through the first acquiring unit 10, where the alarm data to be processed includes a plurality of pieces of attribute information and a plurality of pieces of feature information; the first screening unit 20 screens the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target screening condition to obtain target attribute information and target feature information; the first input unit 30 inputs the target attribute information and the target feature information into a first model, and processes the target attribute information and the target feature information to obtain an alarm handling scheme corresponding to alarm data to be processed, wherein the first model is obtained by training a plurality of groups of first sample data, and each group of first sample data includes all attribute information, all feature information of the sample alarm data and an alarm result corresponding to the sample alarm data. The problem that an alarm processing method corresponding to alarm data cannot be accurately obtained through a machine learning algorithm in the related art is solved. By splitting the alarm data, target characteristic information and target attribute information which can be used for determining an alarm result are selected, and the alarm data are preprocessed to obtain the alarm data which can be more easily identified and judged by a machine learning model, so that the effect of improving the accuracy of the alarm result obtained by machine learning is achieved.
Optionally, in the alarm data processing apparatus provided in the embodiment of the present application, the first filtering unit 20 includes: the first screening module is used for screening the plurality of pieces of attribute information according to a first screening condition to obtain target attribute information; the second screening module is used for screening the plurality of pieces of feature information according to a second screening condition to obtain initial feature information; the first segmentation module is used for segmenting each piece of initial characteristic information according to fields to obtain a plurality of pieces of field information corresponding to the initial characteristic information; and the third screening module is used for screening the plurality of field information according to a third screening condition to obtain a plurality of target field information, and determining the plurality of target field information as target characteristic information.
Optionally, in the alarm data processing apparatus provided in the embodiment of the present application, the apparatus further includes: the second acquisition unit is used for acquiring a plurality of historical alarm data and a historical alarm result corresponding to each historical alarm data, wherein each historical alarm data comprises a plurality of pieces of historical attribute information and a plurality of pieces of historical characteristic information; the second screening unit is used for screening the plurality of pieces of historical attribute information and the plurality of pieces of historical characteristic information in each piece of historical alarm data according to preset screening conditions to obtain historical target attribute information and historical target characteristic information; the first determining unit is used for determining the prediction accuracy of the first model according to historical target attribute information and historical target characteristic information of a plurality of historical alarm data; the first adjusting unit is used for adjusting a preset screening condition under the condition that the prediction accuracy of the first model is smaller than the accuracy condition, re-executing the step of screening the plurality of pieces of historical attribute information and the plurality of pieces of historical characteristic information in each piece of historical alarm data according to the preset screening condition to obtain the historical target attribute information and the historical target characteristic information, and determining the prediction accuracy of the first model according to the historical target attribute information and the historical target characteristic information of the plurality of pieces of historical alarm data until the prediction accuracy of the first model is larger than or equal to the accuracy condition; and the second determining unit is used for determining the adjusted preset screening condition as the target screening condition.
Optionally, in the alarm data processing apparatus provided in the embodiment of the present application, the first determining unit includes: the first input module is used for respectively inputting the historical target attribute information and the historical target characteristic information of each historical alarm data into the first model and processing to obtain a predicted alarm result corresponding to each historical alarm data; the first comparison module is used for comparing the historical alarm result corresponding to each historical alarm data with the corresponding predicted alarm result respectively to obtain a plurality of comparison results; and the first determining module is used for determining the prediction accuracy of the first model according to the comparison results.
Optionally, in the alarm data processing apparatus provided in the embodiment of the present application, the apparatus further includes: the first conversion unit is used for converting each character in each target characteristic information according to a preset conversion rule to obtain character string information; and the first updating unit is used for inputting the character string information into the second model for processing to obtain standard character string information, and determining the standard character string information as updated target characteristic information, wherein the number in the standard character string information is positioned in a preset interval.
Optionally, in the alarm data processing apparatus provided in the embodiment of the present application, the apparatus further includes: the third acquisition unit is used for acquiring a plurality of sample character string information, converting each sample character string according to a preset interval conversion rule to obtain converted digital information, and determining the combination of the sample character strings and the converted digital information as a second sample data to obtain a plurality of second sample data; and the first training unit is used for training the preset model through a plurality of second sample data to obtain a second model.
The alarm data processing device comprises a processor and a memory, wherein the first acquiring unit 10, the first screening unit 20, the first input unit 30 and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.
The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, and the problem that the alarm processing method corresponding to the alarm data cannot be accurately obtained through a machine learning algorithm in the related technology is solved by adjusting the kernel parameters.
The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.
An embodiment of the present invention provides a computer-readable storage medium, on which a program is stored, which, when executed by a processor, implements the alarm data processing method.
The embodiment of the invention provides a processor, which is used for running a program, wherein the alarm data processing method is executed when the program runs.
As shown in fig. 3, an embodiment of the present invention provides an electronic device, where the electronic device 40 includes a processor, a memory, and a program stored in the memory and executable on the processor, and the processor executes the program to implement the following steps: acquiring alarm data to be processed, wherein the alarm data to be processed comprises a plurality of pieces of attribute information and a plurality of pieces of characteristic information; screening the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target screening conditions to obtain target attribute information and target feature information; and inputting the target attribute information and the target characteristic information into a first model, and processing to obtain an alarm handling scheme corresponding to alarm data to be processed, wherein the first model is obtained by training a plurality of groups of first sample data, and each group of first sample data comprises all attribute information, all characteristic information and alarm results corresponding to the sample alarm data of the sample alarm data. The device herein may be a server, a PC, a PAD, a mobile phone, etc.
The present application further provides a computer program product adapted to perform a program for initializing the following method steps when executed on a data processing device: acquiring alarm data to be processed, wherein the alarm data to be processed comprises a plurality of pieces of attribute information and a plurality of pieces of characteristic information; screening the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target screening conditions to obtain target attribute information and target feature information; and inputting the target attribute information and the target characteristic information into a first model, and processing to obtain an alarm handling scheme corresponding to alarm data to be processed, wherein the first model is obtained by training a plurality of groups of first sample data, and each group of first sample data comprises all attribute information, all characteristic information and alarm results corresponding to the sample alarm data of the sample alarm data.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims (10)
1. An alarm data processing method, characterized by comprising:
acquiring alarm data to be processed, wherein the alarm data to be processed comprises a plurality of pieces of attribute information and a plurality of pieces of characteristic information;
screening the plurality of pieces of attribute information and the plurality of pieces of feature information according to target screening conditions to obtain target attribute information and target feature information;
and inputting the target attribute information and the target characteristic information into a first model, and processing to obtain an alarm result corresponding to the alarm data to be processed, wherein the first model is obtained by training a plurality of groups of first sample data, and each group of the first sample data comprises all attribute information, all characteristic information and an alarm handling scheme corresponding to the sample alarm data.
2. The method of claim 1, wherein the screening the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target screening condition to obtain the target attribute information and the target feature information comprises:
screening the plurality of pieces of attribute information according to a first screening condition to obtain target attribute information;
screening the plurality of pieces of feature information according to a second screening condition to obtain initial feature information;
segmenting each piece of initial characteristic information according to fields to obtain a plurality of pieces of field information corresponding to the initial characteristic information;
and screening the plurality of field information according to a third screening condition to obtain a plurality of target field information, and determining the plurality of target field information as the target characteristic information.
3. The method of claim 1, wherein before the filtering the plurality of pieces of attribute information and the plurality of pieces of feature information according to the target filtering condition to obtain the target attribute information and the target feature information, the method further comprises:
acquiring a plurality of historical alarm data and a historical alarm result corresponding to each historical alarm data, wherein each historical alarm data comprises a plurality of pieces of historical attribute information and a plurality of pieces of historical characteristic information;
screening the plurality of pieces of historical attribute information and the plurality of pieces of historical characteristic information in each piece of historical alarm data according to preset screening conditions to obtain historical target attribute information and historical target characteristic information;
determining the prediction accuracy of the first model according to the historical target attribute information and the historical target characteristic information of the plurality of historical alarm data;
under the condition that the prediction accuracy of the first model is smaller than the accuracy condition, adjusting the preset screening condition, re-executing the step of screening the plurality of pieces of historical attribute information and the plurality of pieces of historical characteristic information in each piece of historical alarm data according to the preset screening condition to obtain historical target attribute information and historical target characteristic information, and determining the prediction accuracy of the first model according to the historical target attribute information and the historical target characteristic information of the plurality of pieces of historical alarm data until the prediction accuracy of the first model is greater than or equal to the accuracy condition;
and determining the adjusted preset screening condition as the target screening condition.
4. The method of claim 3, wherein determining the prediction accuracy of the first model based on the historical target attribute information and the historical target characteristic information for the plurality of historical alarm data comprises:
respectively inputting the historical target attribute information and the historical target characteristic information of each historical alarm data into the first model, and processing to obtain a predicted alarm result corresponding to each historical alarm data;
comparing the historical alarm result corresponding to each historical alarm data with the corresponding predicted alarm result to obtain a plurality of comparison results;
and determining the prediction accuracy of the first model according to the comparison results.
5. The method of claim 1, wherein after the plurality of pieces of attribute information and the plurality of pieces of feature information are filtered according to target filtering conditions to obtain target attribute information and target feature information, the method further comprises:
converting each character in each target characteristic information according to a preset conversion rule to obtain character string information;
and inputting the character string information into a second model for processing to obtain standard character string information, and determining the standard character string information as updated target characteristic information, wherein the number in the standard character string information is positioned in a preset interval.
6. The method of claim 5, wherein prior to entering the string information into the second model for processing to obtain standard string information, the method further comprises:
obtaining a plurality of sample character string information, converting each sample character string according to a preset interval conversion rule to obtain converted digital information, and determining the combination of the sample character strings and the converted digital information as a second sample data to obtain a plurality of second sample data;
and training a preset model through the plurality of second sample data to obtain the second model.
7. An alarm data processing apparatus, comprising:
the device comprises a first acquisition unit, a second acquisition unit and a processing unit, wherein the first acquisition unit is used for acquiring alarm data to be processed, and the alarm data to be processed comprises a plurality of pieces of attribute information and a plurality of pieces of characteristic information;
the first screening unit is used for screening the plurality of pieces of attribute information and the plurality of pieces of feature information according to target screening conditions to obtain target attribute information and target feature information;
and the first input unit is used for inputting the target attribute information and the target characteristic information into a first model and processing to obtain an alarm result corresponding to the alarm data to be processed, wherein the first model is obtained by training a plurality of groups of first sample data, and each group of the first sample data comprises all attribute information and all characteristic information of the sample alarm data and an alarm handling scheme corresponding to the sample alarm data.
8. The apparatus of claim 7, wherein the first screening unit comprises:
the first screening module is used for screening the plurality of pieces of attribute information according to a first screening condition to obtain target attribute information;
the second screening module is used for screening the plurality of pieces of feature information according to a second screening condition to obtain initial feature information;
the first segmentation module is used for segmenting each piece of initial characteristic information according to fields to obtain a plurality of pieces of field information corresponding to the initial characteristic information;
and the third screening module is used for screening the plurality of field information according to a third screening condition to obtain a plurality of target field information, and determining the plurality of target field information as the target characteristic information.
9. A non-volatile storage medium, characterized in that the non-volatile storage medium comprises a stored program, wherein the program controls a device in which the non-volatile storage medium is located to execute the alarm data processing method according to any one of claims 1 to 6 when running.
10. An electronic device comprising one or more processors and memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the alert data processing method of any of claims 1 to 6.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111447628.2A CN114238402A (en) | 2021-11-30 | 2021-11-30 | Alarm data processing method and device, storage medium and electronic equipment |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202111447628.2A CN114238402A (en) | 2021-11-30 | 2021-11-30 | Alarm data processing method and device, storage medium and electronic equipment |
Publications (1)
Publication Number | Publication Date |
---|---|
CN114238402A true CN114238402A (en) | 2022-03-25 |
Family
ID=80752376
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202111447628.2A Pending CN114238402A (en) | 2021-11-30 | 2021-11-30 | Alarm data processing method and device, storage medium and electronic equipment |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN114238402A (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114613103A (en) * | 2022-05-10 | 2022-06-10 | 广东阿尔创通信技术股份有限公司 | Alarm positioning processing method and system based on big data |
CN115396280A (en) * | 2022-08-29 | 2022-11-25 | 中国农业银行股份有限公司 | Alarm data processing method, device, equipment and storage medium |
CN116451116A (en) * | 2023-04-03 | 2023-07-18 | 深圳供电局有限公司 | Fault identification model construction method, device, computer equipment and storage medium |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110601894A (en) * | 2019-09-18 | 2019-12-20 | 中国工商银行股份有限公司 | Alarm processing method and device, electronic equipment and readable storage medium |
CN110943857A (en) * | 2019-11-20 | 2020-03-31 | 国网湖北省电力有限公司信息通信公司 | Power communication network fault analysis and positioning method based on convolutional neural network |
CN111738870A (en) * | 2020-07-28 | 2020-10-02 | 工保科技(浙江)有限公司 | Method and platform for identifying insurance risk of engineering performance guarantee based on characteristic engineering |
WO2021042843A1 (en) * | 2019-09-06 | 2021-03-11 | 平安科技(深圳)有限公司 | Alert information decision method and apparatus, computer device and storage medium |
WO2021174835A1 (en) * | 2020-03-04 | 2021-09-10 | 平安科技(深圳)有限公司 | Alarm information processing method and apparatus, and computer apparatus and storage medium |
-
2021
- 2021-11-30 CN CN202111447628.2A patent/CN114238402A/en active Pending
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2021042843A1 (en) * | 2019-09-06 | 2021-03-11 | 平安科技(深圳)有限公司 | Alert information decision method and apparatus, computer device and storage medium |
CN110601894A (en) * | 2019-09-18 | 2019-12-20 | 中国工商银行股份有限公司 | Alarm processing method and device, electronic equipment and readable storage medium |
CN110943857A (en) * | 2019-11-20 | 2020-03-31 | 国网湖北省电力有限公司信息通信公司 | Power communication network fault analysis and positioning method based on convolutional neural network |
WO2021174835A1 (en) * | 2020-03-04 | 2021-09-10 | 平安科技(深圳)有限公司 | Alarm information processing method and apparatus, and computer apparatus and storage medium |
CN111738870A (en) * | 2020-07-28 | 2020-10-02 | 工保科技(浙江)有限公司 | Method and platform for identifying insurance risk of engineering performance guarantee based on characteristic engineering |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN114613103A (en) * | 2022-05-10 | 2022-06-10 | 广东阿尔创通信技术股份有限公司 | Alarm positioning processing method and system based on big data |
CN114613103B (en) * | 2022-05-10 | 2022-08-23 | 广东阿尔创通信技术股份有限公司 | Alarm positioning processing method and system based on big data |
CN115396280A (en) * | 2022-08-29 | 2022-11-25 | 中国农业银行股份有限公司 | Alarm data processing method, device, equipment and storage medium |
CN115396280B (en) * | 2022-08-29 | 2024-03-19 | 中国农业银行股份有限公司 | Alarm data processing method, device, equipment and storage medium |
CN116451116A (en) * | 2023-04-03 | 2023-07-18 | 深圳供电局有限公司 | Fault identification model construction method, device, computer equipment and storage medium |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN114238402A (en) | Alarm data processing method and device, storage medium and electronic equipment | |
CN112200081B (en) | Abnormal behavior identification method, device, electronic device and storage medium | |
CN114580263A (en) | Knowledge graph-based information system fault prediction method and related equipment | |
CN113965389B (en) | Network security management method, device and medium based on firewall log | |
CN110399268A (en) | A kind of method, device and equipment of anomaly data detection | |
CN112988509A (en) | Alarm message filtering method and device, electronic equipment and storage medium | |
CN111582341A (en) | User abnormal operation prediction method and device | |
CN113610156A (en) | Artificial intelligence model machine learning method and server for big data analysis | |
CN112565422A (en) | Method, system and storage medium for identifying fault data of power internet of things | |
CN118261695A (en) | Risk assessment method and device, computer storage medium and electronic equipment | |
CN114024761A (en) | Network threat data detection method and device, storage medium and electronic equipment | |
WO2022144908A1 (en) | Method and network node for detecting anomalous access behaviours | |
CN110532773B (en) | Malicious access behavior identification method, data processing method, device and equipment | |
CN118694609B (en) | Abnormal data monitoring method and system based on industrial Internet of Things | |
CN118740892A (en) | An industrial gas analysis method based on the Internet of Things | |
CN110458383A (en) | Demand handles implementation method, device and the computer equipment of serviceization, storage medium | |
CN114385876B (en) | Model search space generation method, device and system | |
CN116170829A (en) | Operation and maintenance scene identification method and device for independent private network service | |
CN106776623B (en) | User behavior analysis method and device | |
CN115237734A (en) | Task risk detection method and device | |
CN107122480A (en) | A kind of cloud analysis method and system based on traffic violation data | |
US11151473B1 (en) | Systems and methods for machine-learning augmented application monitoring | |
CN113128734A (en) | Method and device for predicting oil field yield | |
CN115794195B (en) | Banking business management and maintenance method and device based on Viper framework | |
CN119248739A (en) | Log processing method, device and electronic equipment |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |