[go: up one dir, main page]

CN118863546B - A risk classification method and system based on deep learning - Google Patents

A risk classification method and system based on deep learning Download PDF

Info

Publication number
CN118863546B
CN118863546B CN202411327991.4A CN202411327991A CN118863546B CN 118863546 B CN118863546 B CN 118863546B CN 202411327991 A CN202411327991 A CN 202411327991A CN 118863546 B CN118863546 B CN 118863546B
Authority
CN
China
Prior art keywords
model
data
sub
input
calculation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202411327991.4A
Other languages
Chinese (zh)
Other versions
CN118863546A (en
Inventor
路笃辉
王目凯
郝艳荣
仇志阳
唐潇宇
丁彦龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Qinghai Zhongtejian Special Equipment Testing Co ltd
Zhongjian Metrology Co ltd
China Special Equipment Inspection and Research Institute
Original Assignee
Qinghai Zhongtejian Special Equipment Testing Co ltd
Zhongjian Metrology Co ltd
China Special Equipment Inspection and Research Institute
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Qinghai Zhongtejian Special Equipment Testing Co ltd, Zhongjian Metrology Co ltd, China Special Equipment Inspection and Research Institute filed Critical Qinghai Zhongtejian Special Equipment Testing Co ltd
Priority to CN202411327991.4A priority Critical patent/CN118863546B/en
Publication of CN118863546A publication Critical patent/CN118863546A/en
Application granted granted Critical
Publication of CN118863546B publication Critical patent/CN118863546B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management
    • G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063—Operations research, analysis or management
    • G06Q10/0635—Risk analysis of enterprise or organisation activities
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Business, Economics & Management (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Human Resources & Organizations (AREA)
  • Strategic Management (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Economics (AREA)
  • Probability & Statistics with Applications (AREA)
  • Evolutionary Biology (AREA)
  • Development Economics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Educational Administration (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Game Theory and Decision Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本申请提供了一种基于深度学习的风险分类方法及系统，涉及风险分类的技术领域，所述方法包括第一建模、第一输入、第一获取、第一计算、第二输入、第三输入、第二计算和第三计算等步骤；所述系统包括建立模型模块、数据处理模块和计算模块。本申请兼顾了HAZOP分析报告的局部特征和长距离信息特征，提高了HAZOP分析报告中文本分类的准确性。

The present application provides a risk classification method and system based on deep learning, which relates to the technical field of risk classification. The method includes the steps of first modeling, first input, first acquisition, first calculation, second input, third input, second calculation and third calculation; the system includes a model building module, a data processing module and a calculation module. The present application takes into account the local features and long-distance information features of the HAZOP analysis report, and improves the accuracy of text classification in the HAZOP analysis report.

Description

Risk classification method and system based on deep learning
Technical Field
The application relates to the technical field of risk classification, in particular to a risk classification method and system based on deep learning.
Background
The production process in the industries of chemical engineering, manufacturing and the like is usually accompanied by huge safety risks, and it is extremely important to carry out safety analysis on the whole process before an accident occurs. The hazard and operability analysis (HAZOP) method is a safety analysis technology widely applied to the fields of chemical industry, manufacturing, electric power and the like, and is beneficial to finding potential safety hazards in the production process. Unstructured text knowledge multiplexing capability in the HAZOP analysis report is poor, which is always an important problem in the HAZOP application field, and text classification is an effective method for improving the knowledge multiplexing capability of the HAZOP analysis report.
Text classification of HAZOP analysis reports is generally divided into two periods throughout industry development, wherein the first period is that people collect information of HAZOP analysis reports by using a manual method, such as establishing a keyword search system and manually constructing a knowledge graph. The second stage is to use machine learning and deep learning techniques instead of manual methods with the popularity of computer technology, automatically extract features from text using models, and automatically perform model training and prediction. The machine learning and deep learning technologies thoroughly change the traditional HAZOP analysis field, and bring great development potential to the intelligent process safety analysis field.
However, the text in the HAZOP analysis report has a large number of professional words, and the generalized Hurst index of the whole text language structure is higher, and the language logic and causality are stronger. That is, the HAZOP text has more compact local features and long-distance information features of a specific structure, and in related researches, only a basic deep learning model (e.g., LSTM model) is often used to perform feature extraction on the HAZOP analysis report, and language characteristics of the text in the HAZOP analysis report are not explored, so that using only the basic deep learning model cannot well improve the accuracy of risk classification of the HAZOP analysis report.
Disclosure of Invention
In order to improve the accuracy of HAZOP analysis report risk classification, the application provides a risk classification method and system based on deep learning.
In a first aspect, the present application provides a risk classification method based on deep learning, which adopts the following technical scheme:
a risk classification method based on deep learning, comprising:
collecting HAZOP analysis reports to be classified, and recording the HAZOP analysis reports as first reports;
First modeling, establishing a T-BTM model, wherein the T-BTM model comprises a BERT sub-model, a TextCNN sub-model, a BILSTM sub-model and a full connection layer;
The first input, namely taking a first report as the input of the BERT sub-model, generating a word vector with a label by utilizing an embedding layer of the BERT sub-model, and recording the word vector as a first word vector;
first obtaining, namely respectively inputting the first word vectors into the BERT submodel Layer self-attention layer, obtaining outputFeatures ofAnd sets the weight of each feature;
First calculation based onFeatures ofCalculating first input data, wherein a calculation model of the first input data is as follows:
;
wherein A is the first input data, Is the firstA plurality of features; Is the first Weights of the individual features; Is the first A plurality of features;
Second input, inputting the first input data into TextCNN submodels to obtain first output data ;
Third input, inputting the first input data into BILSTM submodels to obtain second output data;
Second calculation of the first output dataAnd second output dataPerforming linear combination to obtain third output data;
Third calculation of third output dataInput into the full connection layer of the T-BTM model to obtain fourth output dataFourth output data using Softmax functionNormalizing to obtain a risk type of the first report.
By adopting the technical scheme, the T-BTM model is built and comprises a BERT sub-model, a TextCNN sub-model, a BILSTM sub-model and a full-connection layer, wherein an embedded layer of the BERT sub-model is used for generating a high-quality first word vector, the BERT sub-model is used for extracting n characteristics by utilizing n self-attention layers based on the first word vector, first input data are calculated based on the n characteristics and are input into the TextCNN sub-model for capturing local characteristics, the first input data are input into the BILSTM sub-model for capturing long-distance dependency, the outputs of the two sub-models are combined linearly, the local characteristics and the long-distance characteristics are fused, the combined characteristics are input into the full-connection layer, the full-connection layer can learn a higher-level characteristic representation, third output data are output, the output of the third output data is normalized through a Softmax function, and the type of a HAZOP analysis report is output. Because the HAZOP analysis report has more important word characteristics than a general text, the T-BTM model extracts characteristic information of different self-attention layers of the BERT sub-model to be combined, and local characteristics and long-distance characteristics are learned through the TextCNN sub-model and the BILSTM sub-model, so that the T-BTM model can learn the characteristic information of sentences in a text structure of the HAZOP classification report as much as possible.
Optionally, after performing the first modeling, before performing the first input, further comprising:
training the T-BTM model, wherein the training comprises secondary acquisition, training and verification;
Collecting a historical HAZOP analysis report, recording the report as a second report, and dividing the second report into a training set and a verification set;
training the T-BTM model by adopting a training set, and recording the prediction probability of the risk type of each sample in the training process;
And (3) verifying the T-BTM model by adopting a verification set.
By adopting the technical scheme, the method adopts the historical HAZOP analysis report as the training set and the verification set, the training set is adopted to train the T-BTM model, and in the training process, the T-BTM model optimizes the prediction capacity by continuously adjusting parameters, so that the prediction probability gradually approaches to the real risk type, and the process not only improves the classification accuracy of the T-BTM model, but also enhances the robustness of the T-BTM model. And verifying the T-BTM model by adopting a verification set, and judging whether the T-BTM model is over-fitted or under-fitted or not by using performance indexes (such as accuracy, recall, F1 score and the like) of the verification set so as to adjust a training strategy. Through the close combination of the second acquisition, training and verification, the T-BTM model can not only learn enough knowledge in the training process, but also keep certain generalization capability.
Optionally, after performing training and before performing verification, the method further comprises:
calculating loss, namely defining a loss function, fourth calculation, optimization, first judgment and second judgment;
defining a loss function, namely defining the loss function, wherein a calculation model of the loss function is as follows:
;
Wherein, The prediction probability corresponding to the correct risk type; the prediction probability corresponding to the risk type of the ith error; M is the number of risk types for errors;
Calculating the loss of the T-BTM model by using a loss function based on the risk type and the prediction probability;
Optimizing, namely updating parameters of a T-BTM model by using a gradient descent method to obtain updating times and minimum loss;
the first judgment, namely judging whether the minimum loss is smaller than a preset minimum loss threshold value, if so, executing verification, and if not, executing the second judgment;
and judging whether the updating times are larger than a preset iteration times threshold, if so, executing verification, and if not, executing optimization.
By adopting the technical scheme, the application provides an improved cross entropy loss function, which can calculate generalized cross entropy loss and simultaneously consider incorrect labels and probabilities thereof, so that after T-BTM model training, when the probabilities of the correct labels approach 1, the polynomial corresponding to the incorrect labels approaches 0 infinitely, and the loss function of the application enhances the information transmission of back propagation in the T-BTM model training process.
Optionally, after performing the second acquisition, before performing the training, further comprising:
collecting keywords, namely collecting keywords of a second report through a web crawler technology;
establishing a word stock, which comprises collecting corpus, word segmentation and first combination;
Collecting corpus, namely acquiring an accident report and acquiring accident information;
word segmentation, namely converting accident information into text data, and performing word segmentation operation on the text data to obtain first data;
And the first combination is to combine the keywords with the first data to obtain second data, and the second data is used as a dictionary.
By adopting the technical scheme, the application collects the keywords of the second report through the web crawler technology, increases the number of professional words in the training set, and can extract the key information related to risk analysis, accident reasons, safety measures and the like. The accident report obtained by the application contains abundant practical cases and experience training, and has important significance for improving the capabilities of the model in the aspects of risk identification, accident prevention and the like. The accident information is converted into text data and word segmentation operation is a basic step of text processing, and continuous texts can be segmented into independent word units by word segmentation, so that the T-BTM model is facilitated to better understand and process the text data. The method combines the keywords with the first data (namely text data after word segmentation), takes the combined content as a dictionary, and the dictionary contains all words which need to be identified in the training process of the T-BTM model, thereby having important significance for improving the classification accuracy and generalization capability of the T-BTM model.
Optionally, after the word stock is built, before the training is performed, the method further includes:
converting the first data and the keywords into Word vectors through a Word2Vec model;
screening, including first extraction, fifth calculation, sorting, second extraction and second combination;
extracting a keyword in a dictionary, and marking the keyword as a first keyword;
Fifth calculation, namely calculating the similarity of the first keywords and all the first data in the dictionary, and recording the similarity as the first similarity;
Sequencing, namely re-sequencing the first similarity according to the sequence from the big to the small, and marking the first similarity as a first sequence;
second extracting, namely extracting first m pieces of first data in the first sequence;
Combining the first keywords with the corresponding m first data to obtain third data, and taking the third data as a sub-dictionary;
Judging whether first keywords which are not similar to all first data in the dictionary are present in the dictionary or not, if yes, executing screening, otherwise, executing updating the training set;
updating the training set, namely taking all sub-dictionaries as a new training set.
By adopting the technical scheme, the Word2Vec model is utilized to convert the first data and the keywords into Word vectors, and the Word vectors can capture semantic relations among words, so that the T-BTM model can better understand the intrinsic meaning of text data in the training process. The method comprises the steps of extracting a keyword from a dictionary as a starting point, recording the keyword as a first keyword, calculating the similarity between the first keyword and all first data in the dictionary, finding out first data similar to the meaning of the first keyword through calculation of the similarity, sequencing the similarity according to the sequence from large to small, extracting first m pieces of first data from the sequenced sequence, taking the first keyword as m pieces of first data most relevant to the first keyword, combining the first keyword with m pieces of first data corresponding to the first keyword to obtain third data, taking the third data as a sub-dictionary, helping the T-BTM model to focus on the first data relevant to the first keyword in the training process, taking the sub-dictionary as a new training set, and improving the performance of the T-BTM model on specific tasks because the training set focuses on the first data relevant to the first keyword.
Optionally, after performing the third determination, before performing updating the training set, further comprising:
Deleting, namely deleting the first data which are not stored in the sub-dictionary in the dictionary.
By adopting the technical scheme, after the new sub-dictionary is combined, the first data which are not stored in the sub-dictionary are deleted from the original dictionary, so that redundancy is reduced.
Optionally, after performing the third calculation, further comprising:
establishing an SPN model, wherein the SPN model comprises a multi-head self-attention layer, a multi-head cross-attention layer and a full-connection layer;
A sixth calculation of taking the first word vector as the input of the multi-head self-attention layer, calculating the attention weight of each first word vector and the rest word vectors, taking the attention weight as the input of the multi-head cross-attention layer, and obtaining fifth output data ;
Seventh calculation of fifth output dataInput full connection layer of SPN model, output sixth output dataFor the sixth output data by Softmax functionAnd carrying out normalization processing to obtain a prediction result of the entity-relation triplet in the first report.
By adopting the technical scheme, the SPN model can more comprehensively capture semantic information in the text of the HAZOP analysis report by introducing self-attention and cross-attention mechanisms, and particularly the complex relationship between the entity and the relationship is beneficial to improving the accuracy of the SPN model in the entity-relationship extraction task. The multi-head mechanism enables the SPN model to learn the characteristics of data from multiple angles, and the generalization capability of the SPN model is enhanced. By visualizing the attention weights, it is possible to learn what parts of the information the SPN model focuses on when making predictions.
Optionally, after performing the sixth calculation, before performing the seventh calculation, further comprising:
Third modeling, namely establishing ACMix a model, wherein the ACMix model comprises a full connection layer and a self-attention layer, and the ACMix model is calculated as follows:
;
Wherein, Full connection layer output for ACMix model; Self-attention layer output for ACMix model; Weights output for the full connection layer of ACMix model; weights output for the self-attention layer of ACMix model; The output of ACMix models;
Eighth calculation will be Features ofAs an input to the ACMix model, seventh output data is obtained;
Ninth calculation of fifth output dataAnd seventh output dataAfter combining, as new fifth output data。
By adopting the technical scheme, the application establishes the ACMix model, and the ACMix model can learn more comprehensive and deep characteristic representation by combining the output of the full-connection layer and the self-attention layer, so that the accuracy of entity-relation triplet prediction is improved. The ACMix model reduces the dependence on single characteristics or single models by weighting and fusing the characteristic representations of different sources, thereby improving the robustness and generalization capability of the ACMix model.
Optionally, the prediction result is represented by an offset from the beginning to the end of the entity.
By adopting the technical scheme, the accuracy of entity extraction can be remarkably improved by using the offset to represent the predicted result, and the extracted entity can be judged whether to be correct according to whether the offset in the predicted result is correct or not because the offset is directly corresponding to a specific position in the text.
In a second aspect, the present application provides a risk classification system based on deep learning, which adopts the following technical scheme:
A deep learning-based risk classification system, comprising:
the data acquisition module is used for acquiring HAZOP analysis reports to be classified;
The building model module is used for building a T-BTM model, wherein the T-BTM model comprises a BERT sub-model, a TextCNN sub-model, a BILSTM sub-model and a full connection layer;
the data processing module comprises a first processing unit, a second processing unit and a third processing unit;
the first processing unit is used for inputting the HAZOP analysis report to be classified into the BERT sub-model, obtaining a first word vector after the processing of an embedding layer of the BERT sub-model, inputting the first word vector into an attention layer of the BERT sub-model and outputting a plurality of characteristics;
A second processing unit for inputting the first input data into TextCNN submodels to obtain first output data ;
A third processing unit for inputting the first input data into BILSTM submodels to obtain second output data;
The computing module comprises a first computing unit, a second computing unit and a third computing unit;
a first calculation unit configured to calculate first input data based on a plurality of features;
a second calculation unit for outputting the first output data And second output dataPerforming linear combination to obtain third output data;
A third calculation unit for outputting the third output dataInput to the full connection layer to obtain fourth output dataFourth output data using Softmax functionNormalizing to obtain a risk type of the first report.
By adopting the technical scheme, the T-BTM model is constructed and comprises the BERT sub-model, the TextCNN sub-model, the BILSTM sub-model and the full-connection layer, and the BERT sub-model can capture the context information in the text and improve the quality of text representation through the strong pre-training capability and the depth bidirectional coding. The TextCNN sub-model is good at capturing local features of text, such as keywords, phrases, etc., and is very effective for recognition of short text or specific patterns. The BILSTM sub-model is capable of handling long-term dependency problems in sequence data, and is helpful for understanding timing relationships and semantic structures in text. The full connection layer is used as a core part of the classifier and is responsible for integrating the output of each submodel and carrying out final classification decision. The computing module is used for computing input and output data, combining the output data of each sub-module to obtain total output data, inputting the total output data to the full-connection layer, processing the output of the full-connection layer through a Softmax function, and outputting probability distribution of each risk type to obtain the most probable risk type. The system can automatically collect and process a large amount of text data, and quickly identify the risk type in the HAZOP analysis report.
In summary, the present application includes at least one of the following beneficial technical effects:
1. The T-BTM model comprises a BERT sub-model, a TextCNN sub-model, a BILSTM sub-model and a full-connection layer, wherein an embedding layer of the BERT sub-model is used for generating a high-quality first word vector, the BERT sub-model is used for extracting n characteristics by using n self-attentive layers of the BERT sub-model based on the first word vector, first input data are calculated based on the n characteristics and are input into the TextCNN sub-model for capturing local characteristics, the first input data are also input into the BILSTM sub-model for capturing long-distance dependency, the outputs of the two sub-models are combined linearly and fused with the long-distance characteristics, the combined characteristics are input into the full-connection layer, the full-connection layer can learn a higher-level characteristic representation, third output data are output, the output of the third output data is normalized through a Softmax function, and the risk type of a HAZOP analysis report is output. Because the HAZOP analysis report has more important word characteristics than a general text, the T-BTM model extracts characteristic information of different self-attention layers of the BERT sub-model to be combined, and local characteristics and long-distance characteristics are learned through the TextCNN sub-model and the BILSTM sub-model, so that the T-BTM model can learn the characteristic information of sentences in a text structure of the HAZOP classification report as much as possible.
2. According to the application, the keywords of the second report are collected through the web crawler technology, so that the number of professional words in the training set is increased, and key information related to risk analysis, accident reasons, safety measures and the like can be extracted. The accident report obtained by the application contains abundant practical cases and experience training, and has important significance for improving the capabilities of the model in the aspects of risk identification, accident prevention and the like. The accident information is converted into text data and word segmentation operation is a basic step of text processing, and continuous texts can be segmented into independent word units by word segmentation, so that the T-BTM model is facilitated to better understand and process the text data. The method combines the keywords with the first data (namely text data after word segmentation), takes the combined content as a dictionary, and the dictionary contains all words which need to be identified in the training process of the T-BTM model, thereby having important significance for improving the classification accuracy and generalization capability of the T-BTM model.
3. The application builds a T-BTM model, which comprises a BERT sub-model, a TextCNN sub-model, a BILSTM sub-model and a full connection layer, wherein the BERT sub-model can capture context information in a text through strong pre-training capability and depth bidirectional coding, and improve the quality of text representation. The TextCNN sub-model is good at capturing local features of text, such as keywords, phrases, etc., and is very effective for recognition of short text or specific patterns. The BILSTM sub-model is capable of handling long-term dependency problems in sequence data, and is helpful for understanding timing relationships and semantic structures in text. The full connection layer is used as a core part of the classifier and is responsible for integrating the output of each submodel and carrying out final classification decision. The computing module is used for computing input and output data, combining the output data of each sub-module to obtain total output data, inputting the total output data to the full-connection layer, processing the output of the full-connection layer through a Softmax function, and outputting probability distribution of each risk type to obtain the most probable risk type. The system can automatically collect and process a large amount of text data, and quickly identify the risk type in the HAZOP analysis report.
Drawings
FIG. 1 is a flow chart of the method of embodiment 1 of the present application;
FIG. 2 is a flow chart of a method of training a T-BTM model according to embodiment 1 of the present application;
fig. 3 is a flow chart of a method of processing S4 data in embodiment 1 of the present application.
Detailed Description
The application is described in further detail below in connection with fig. 1 to 3.
Embodiment 1 discloses a risk classification method based on deep learning, referring to fig. 1, the method comprises the following steps of S1, first acquisition, S2, first modeling, S3, training of a T-BTM model, S4, data processing, S5, entity-relation extraction, wherein firstly, HAZOP analysis reports to be classified are acquired, then a T-BTM model is built, then the T-BTM model is trained, then the HAZOP analysis reports to be classified are processed through the T-BTM model, and then the entity-relation extraction is carried out on the processed HAZOP analysis reports, and the specific process of the method is as follows:
S1, collecting HAZOP analysis reports to be classified, and recording the HAZOP analysis reports as first reports.
The first acquisition step is focused on collecting and collating HAZOP (hazard and operability analysis) analysis reports to be subjected to safety analysis. These analytical reports contain detailed analysis of potential hazards and operational problems in industrial processes, equipment or systems, and are important documents to ensure production safety and prevent accidents. In this step, the latest HAZOP analysis report is automatically retrieved and downloaded through a designated path or database interface, so as to ensure timeliness and integrity of the data. These HAZOP analysis reports are noted as "first reports" and stored in a special folder or database for ready recall and access.
S2, first modeling, and building a T-BTM model, wherein the T-BTM model comprises a BERT sub-model, a TextCNN sub-model, a BILSTM sub-model and a full connection layer.
The method comprises the steps of taking a first report as an embedding layer of a BERT sub-model, generating a word vector with a label, recording the word vector as the first word vector, respectively inputting the first word vector into n layers of attention layers of the BERT sub-model, respectively extracting characteristics of a HAZOP analysis report by the n layers of attention layers of the BERT sub-model, converting unstructured text information into characteristic vectors, simultaneously processing text long-distance characteristic information in the HAZOP analysis report by using a BILSTM sub-model, and introducing a TextCNN sub-model to process the local information of the text in the HAZOP analysis report so as to obtain more perfect chemical professional word characteristics, and rapidly learning the local characteristics in the text by using a TextCNN sub-model through local connection and weight sharing characteristics of a Convolutional Neural Network (CNN), so as to further increase understanding of the T-BTM model on text contents.
As shown in FIG. 2, the S3: T-BTM model training includes S31: second acquisition, S32: reset training set, S33: training, S34: calculate loss, and S35: validation.
And S31, second collection, namely collecting a historical HAZOP analysis report, recording the historical HAZOP analysis report as a second report, and dividing the second report into a training set and a verification set according to the proportion.
S32, resetting the training set, including S321, collecting keywords, S322, establishing a word stock and S323, and resetting.
And S321, collecting keywords, and collecting keywords of the second report through a web crawler technology.
The present embodiment utilizes advanced web crawler technology to automatically collect keywords related to the second report. A web crawler is an automated script or program that is able to browse the world wide web and extract information. Crawler programs are tailored to this task to target searches and parse content in websites, databases, or file systems that store these reports.
The crawler will simulate browser behavior, access the specified URL addresses, extract text data in the HTML page, and use regular expressions, XPath, or other parsing techniques to identify and extract keywords related to the HAZOP analysis report, which may include, but are not limited to, process names, device types, chemicals, security risk categories, failure modes, etc.
In order to improve the comprehensiveness and accuracy of keyword collection, a crawler can be further set to traverse related links in a recursion mode, the searching range is further enlarged, and the embodiment further follows the moral specification used by the web crawler by setting the authority of the crawler program.
S322, establishing a word stock, wherein the word stock comprises S3221, collecting corpus, S3222, word segmentation and S3223, and the first combination.
S3221, collecting corpus, acquiring accident reports, and acquiring accident information, wherein the accident reports are descriptions of industrial accidents, particularly a complete accident analysis report, and the whole text of the accident report comprises basic conditions of accident enterprises, accident passing, rescue and post-treatment conditions, accident reasons and properties, treatment suggestions of accident-related responsible personnel and responsible institutions and treatment suggestions of related problems, wherein the accident reasons and properties are part of important contents of the chemical accident report, and comprise factors such as direct reasons and indirect reasons of the accident, particularly industrial equipment, personnel operation, industrial materials, safety devices and the like involved in the accident.
In order to ensure the accuracy of the accident information, after the accident information is acquired, whether the problem of messy codes or illegal characters exist is checked manually, if yes, the problem of messy codes or illegal characters are cleared manually, and if not, S3222 is executed for word segmentation.
S3222, word segmentation is carried out, accident information is converted into text data, word segmentation operation is carried out on the text data, and first data are obtained. The present embodiment uses specialized word segmentation tools or algorithms, such as jieba word segmentation, hanLP, etc., to perform word segmentation processing on the text data.
S3223, integrating the keywords and the first data, and taking the integrated content as a dictionary. For example, the keyword and the first data are stored together in a database as a dictionary. For another example, the keywords are stored in one sequence table, the first data is stored in one sequence table, and both sequence tables are stored together in the database. For another example, one keyword and all the first data are stored in one sequence table, and if the second report has several keywords, several sequence tables are stored correspondingly, and these sequence tables are stored together in the database.
S323, first reset, comprising S3231 conversion, S3232 screening, S3233 third judging, S3234 deleting and S3235 updating training set.
The conversion of S3231, for each Word in the first data, the Word2Vec model outputs a fixed length vector, which is a representation of the Word in vector space. Similarly, this step also converts the collected keywords into word vectors for subsequent interaction and comparison with the word vectors of the first data.
S3232, screening, including S32321, first extraction, S32322, fifth calculation, S32323, ordering, S32324, second extraction, and S32325, second combination.
And S32321, first extraction, namely extracting one keyword in the dictionary based on the dictionary established in the first combination in S3223, and marking the keyword as a first keyword.
And S32322, fifth calculation, namely calculating the similarity of the first keyword and all the first data in the dictionary, obtaining a series of similarity scores, and recording the similarity scores as first similarity, wherein the first similarity reflects the semantic proximity degree between the first keyword and each first data.
And S32323, sequencing, namely re-sequencing the first similarity according to the sequence from the big to the small, and marking the first similarity as a first sequence.
S32324, second extraction, namely extracting first m pieces of first data in the first sequence according to the sequencing result of the first sequence.
And S32325, combining the first keywords with the m pieces of first data extracted in the second extraction as a sub dictionary. The combination form of the first keywords and the m first data comprises the steps of storing the keywords and the m first data in a database together to serve as a dictionary, storing the keywords in a sequence table, storing the m first data in the sequence table, storing the two sequence tables together in the database, storing the keywords and the m first data in the sequence table, storing a plurality of keywords in a second report, and storing a plurality of sequence tables correspondingly, and storing the sequence tables together in the database.
S3233, third judging whether first keywords which are not similar to all first data in the dictionary exist in the dictionary, if yes, executing S3232, screening based on the first keywords which are not similar to all the first data in the dictionary, and if not, executing S3234, deleting.
And S3234, deleting the first data which is not stored in the sub-dictionary in the dictionary.
And S3235, updating the training set, and taking all the sub-dictionaries as a new training set.
And S33, training, namely, training the T-BTM model by adopting a new training set obtained in the updated training set, and recording the prediction probability of the risk type of each sample in the training process, wherein the prediction probability is used for calculating the loss in the step S34.
The step S34 is to calculate the loss, including step S341 is to define a loss function, step S342 is to define a fourth calculation, step S343 is to optimize, step S344 is to first judge and step S345 is to second judge.
S341, defining a loss function, wherein the loss function is defined by a calculation model as follows:
;
Wherein, The prediction probability corresponding to the correct risk type; The prediction probability corresponding to the risk type of the ith error is obtained, and M is the number of the risk types of the errors.
And S342, fourth calculation, namely calculating loss by using a loss function based on the risk types of the samples in the training set and the prediction probability of each sample risk type.
And S343, optimizing, namely iteratively updating parameters of the T-BTM model by using a gradient descent method, recording the updating times once every updating, and defining the initial updating times as 0. Minimal loss is obtained by continually iteratively updating the parameters of the T-BTM model.
S344 is a first judgment, after each update, judging whether the minimum loss is smaller than a preset minimum loss threshold, if yes, executing S35 is verification, and if not, executing S345 is a second judgment.
S345, judging whether the updating times are larger than a preset iteration times threshold, if so, executing S35, verifying, and if not, executing S343, optimizing.
And S35, verifying the T-BTM model by adopting a verification set, wherein the verification set is used for verifying that the performance of the T-BTM model meets the expectations. If the validation results are not expected, the calculation of the penalty may be re-performed S34, optimizing the T-BTM model.
As shown in fig. 3, S4: data processing includes S41: first input, S42: first acquisition, S43: first calculation, S44: second input, S45: third input, S46: second calculation, and S47: third calculation.
S41, a first input, namely, using a first report as an input of the BERT sub-model, generating a word vector with labels by utilizing an embedding layer of the BERT sub-model, and recording the word vector as a first word vector.
S42, first obtaining, namely respectively inputting the first word vectors into the BERT submodelLayer self-attention layer, outputFeatures ofAnd sets the weight of each feature。
S43, first calculation based onFeatures ofAnd the weight of each featureCalculating first input data, wherein a calculation model of the first input data is as follows:
;
wherein A is the first input data, Is the firstA plurality of features; Is the first Weights of the individual features; Is the first And features.
S44, a second input, namely inputting the first input data into TextCNN submodels, carrying out convolution and pooling on the linear combination of the first input data through different convolution checks to obtain local characteristics of the HAZOP analysis report, namely the first output data。
S45, third input, inputting the first input data into BILSTM submodels, obtaining long-distance characteristics of the first input data, and obtaining second output data。
S46, second calculating the first output dataAnd second output dataPerforming linear combination to obtain third output data。
S47, third calculation, to output third dataInput into the full connection layer of the T-BTM model to obtain fourth output dataFourth output data using Softmax functionNormalizing to obtain a risk type of the first report.
Assume that a classification case of a certain sample has 10 different risk type labels (risk type label 1, risk type label 2,..and risk type label 10), and of the 10 risk type labels, only 1 is a correct label of the current sample, the remaining 9 are error labels, the risk type label 3 is set to be a correct label, and a common deep learning model is adopted for prediction, wherein the risk type and the prediction probability of the sample are as follows:
In the first case, the prediction probability of the risk type tag 3 is 0.46, the prediction probability of the risk type tag 2 is 0.46, and the rest risk type tags occupy 0.08 together.
After training, the T-BTM model of the embodiment is adopted, and the risk type and the prediction probability of the sample are as follows:
In the second case, the prediction probability of the risk type tag 3 is still 0.46, and the prediction probability of all the remaining risk type tags is 0.06.
In the first case, the wrong label (wrong risk type label 2) will generate a strong misleading to the correct label (correct risk type label 3), and in the second case, the correct label (correct risk type label 3) can be correctly found, and the same output can be obtained by selecting the conventional cross entropy loss function, that is, the conventional cross entropy loss function only focuses on the matching degree of the prediction probability distribution of each sample and the real label, and does not focus on the wrong label. The loss function adopted in the embodiment can consider all the error labels while performing generalized cross entropy loss calculation, so that the error labels approach to a relative average state, and the influence of extreme values of the error labels on the correct labels is reduced. By iterative training, the probability of the correct label is made to approach 1, the more the loss function approaches 0.
S5, entity-relation extraction, including S51, second modeling, S52, sixth calculation, S53, third modeling, S54, eighth calculation, S55, ninth calculation and S56, seventh calculation.
S51, second modeling, and establishing an SPN model, wherein the SPN model comprises a multi-head self-attention layer, a multi-head cross-attention layer and a full-connection layer. A multi-headed self-attention layer for capturing internal dependencies between input word vectors, each head independently calculating attention weights, and then combining these weights to generate a richer representation. A multi-headed cross-attention layer receives as input the output of the multi-headed self-attention layer and possibly performs cross-attention calculations with representations of other sequences (e.g., context, query, etc.) to further fuse the information.
S52, sixth calculating, namely taking the first word vector as the input of the multi-head self-attention layer, calculating the attention weights of the first word vector and the rest word vectors for each first word vector, taking the attention weights as the input of the multi-head cross-attention layer, and obtaining fifth output data。
S53, third modeling, namely establishing ACMix a model, wherein the ACMix model comprises a full connection layer and a self-attention layer, and the ACMix model is calculated as follows:
;
Wherein, Full connection layer output for ACMix model; Self-attention layer output for ACMix model; Weights output for the full connection layer of ACMix model; weights output for the self-attention layer of ACMix model; Is the output of ACMix models.
The ACMix model also includes a plurality of different convolution kernels for mapping the output of the BERT sub-model to different feature sets.
S54 eighth calculation, toFeatures ofThe data are input into a convolution layer of ACMix models to obtain a plurality of feature sets, and then the feature sets are divided into N groups, wherein each group is internally provided with three features.
And taking the three features as a query value key value and a value in the self-attention layer to participate in the calculation of the self-attention model, and obtaining an attention result.
And carrying out operation on the data of the N groups of feature sets through a full connection layer of ACMix models, and carrying out shift operation on operation results to obtain different new N groups of feature sets.
The output is then calculated from the calculation model of ACMix models:
;
Wherein, N groups of different feature sets are output for the full connection layer of the ACMix model; attention results output for the self-attention layer of ACMix model; Weights output for the full connection layer of ACMix model; weights output for the self-attention layer of ACMix model; Seventh output data output for ACMix models.
S55, ninth calculation, to output fifth output dataAnd seventh output dataAfter combining, as new fifth output data。
S56 seventh calculation, the fifth output data in the S55 ninth calculationInput full connection layer, output sixth output dataFor the sixth output data by Softmax functionAnd carrying out normalization processing to obtain a predicted result of the entity-relation triplet in the first report, and representing the predicted result by the offset of the head and the tail of the entity.
The embodiment provides a risk classification method based on deep learning, firstly, a HAZOP analysis report to be classified is collected as a first report. Subsequently, by building a T-BTM model, the model includes a BERT sub-model, a TextCNN sub-model, a BILSTM sub-model, and a full connection layer. The present embodiment inputs the first report into the BERT sub-model to generate a tagged word vector (i.e., a first word vector), and extracts features and sets weights using the self-attention layer of BERT. Based on these features and their weights, first input data is calculated. Then, the first input data is fed into TextCNN and BILSTM sub-models, respectively, to obtain two independent output data (first output dataAnd second output data). Then, by linearly combining the two output data, third output data is generatedAnd finally, carrying out normalization processing through the full connection layer and the Softmax function to predict the risk type of the first report. The method fully utilizes the advantages of different submodels and improves the accuracy and the efficiency of the classification of the HAZOP analysis report.
Embodiment 2 this embodiment discloses a risk classification system based on deep learning, the system includes:
And the data acquisition module is used for acquiring HAZOP analysis reports to be classified.
And the building model module is used for building a T-BTM model, wherein the T-BTM model comprises a BERT sub-model, a TextCNN sub-model, a BILSTM sub-model and a full connection layer.
The data processing module comprises a first processing unit, a second processing unit and a third processing unit.
The first processing unit is used for inputting the HAZOP analysis report to be classified into the BERT sub-model, converting the text into tagged word vectors (namely first word vectors) through an embedding layer of the BERT sub-model, wherein the word vectors are rich in semantic information of the text, and sending the first word vectors into an attention layer of the BERT sub-model, and extracting a plurality of key features through an attention mechanism, wherein the features are used for subsequent calculation and classification.
A second processing unit for inputting the first input data into TextCNN submodels, performing convolution and pooling by different convolution checks to obtain local features of the HAZOP analysis report, namely the first output data。
A third processing unit for inputting the first input data into BILSTM submodels to obtain long-distance characteristics of the first input data and obtain second output data。
The computing module comprises a first computing unit, a second computing unit and a third computing unit;
and a first calculation unit for calculating first input data based on the plurality of features output by the BERT sub-model.
A second calculation unit for outputting the first output dataAnd second output dataPerforming linear combination to obtain third output data。
A third calculation unit for outputting the third output dataInput to the full connection layer to obtain fourth output dataFourth output data using Softmax functionNormalizing to obtain a risk type of the first report.
The embodiment provides a risk classification system based on deep learning, wherein a data acquisition module is responsible for collecting HAZOP analysis reports to be classified, and then a model building module is used for building an integrated T-BTM model, and the T-BTM model is combined with BERT sub-model, textCNN sub-model, BILSTM sub-model and full connection layer advantages. Units in the data processing module convert the report into word vectors and extract features, while the input TextCNN and BILSTM models obtain output data. The computing module integrates these outputs and ultimately determines the reported risk type classification via the full connection layer and Softmax functions.
The above embodiments are not intended to limit the scope of the application, so that the equivalent changes of the structure, shape and principle of the application are covered by the scope of the application.

Claims (7)

1.一种基于深度学习的风险分类方法，包括，第一采集：采集待分类的HAZOP分析报告，记为第一报告；其特征在于，所述分类方法还包括：1. A risk classification method based on deep learning, comprising: first collection: collecting a HAZOP analysis report to be classified, recorded as a first report; characterized in that the classification method also includes: 第一建模：建立T-BTM模型，所述T-BTM模型包括：BERT子模型、TextCNN子模型、BILSTM子模型和全连接层；First modeling: establishing a T-BTM model, wherein the T-BTM model includes: a BERT sub-model, a TextCNN sub-model, a BILSTM sub-model, and a fully connected layer; T-BTM模型训练：包括第二采集、训练和验证；T-BTM model training: including second acquisition, training and verification; 第二采集：采集历史HAZOP分析报告，记为第二报告，将所述第二报告划分为训练集和验证集；Second collection: collect historical HAZOP analysis reports, record them as second reports, and divide the second reports into training sets and validation sets; 收集关键词：通过网络爬虫技术收集第二报告的关键词；Collect keywords: collect keywords for the second report through web crawler technology; 建立词库：包括收集语料、分词和第一结合；Building a vocabulary: including collecting corpus, word segmentation and first combination; 收集语料：获取事故报告，采集事故信息；Collect corpus: obtain accident reports and collect accident information; 分词：将事故信息转化为文本数据，对文本数据进行分词操作，获得第一数据；Word segmentation: convert the accident information into text data, perform word segmentation on the text data, and obtain the first data; 第一结合：将关键词和第一数据结合，获得第二数据，将第二数据作为词典；First combination: combining the keyword with the first data to obtain the second data, and using the second data as a dictionary; 转化：通过Word2Vec模型，分别将第一数据和关键词转化为词向量；Conversion: The first data and keywords are converted into word vectors through the Word2Vec model; 筛选：包括第一提取、第五计算、排序、第二提取和第二结合；Screening: includes first extraction, fifth calculation, sorting, second extraction and second combination; 第一提取：提取词典中的一个关键词，记为第一关键词；First extraction: extract a keyword from the dictionary and record it as the first keyword; 第五计算：计算第一关键词和词典中所有第一数据的相似度，记为第一相似度；Fifth calculation: calculating the similarity between the first keyword and all the first data in the dictionary, recorded as the first similarity; 排序：将第一相似度按照从大到小的顺序重新排序，记为第一序列；Sorting: re-arrange the first similarity from large to small, and record it as the first sequence; 第二提取：提取第一序列中前m个第一数据；Second extraction: extract the first m first data in the first sequence; 第二结合：将第一关键词和对应的m个第一数据结合，获得第三数据，将第三数据作为子词典；Second combination: combining the first keyword with the corresponding m first data to obtain third data, and using the third data as a sub-dictionary; 第三判断：判断词典内是否存在未与词典中所有第一数据计算相似度的第一关键词，若是，则执行筛选；若否，则执行更新训练集；Third judgment: judging whether there is a first keyword in the dictionary whose similarity is not calculated with all the first data in the dictionary, if yes, performing screening; if no, performing updating of the training set; 更新训练集：将所有子词典作为新的训练集；Update training set: take all sub-dictionaries as new training sets; 训练：采用训练集对T-BTM模型进行训练，并在训练过程中，记录每个样本的风险类型的预测概率；Training: Use the training set to train the T-BTM model, and during the training process, record the predicted probability of the risk type of each sample; 验证：采用验证集对T-BTM模型进行验证；Verification: The T-BTM model is verified using the verification set; 第一输入：将第一报告作为BERT子模型的输入，利用BERT子模型的嵌入层，生成带标签的词向量，记为第一词向量；First input: The first report is used as the input of the BERT sub-model. The embedding layer of the BERT sub-model is used to generate a labeled word vector, which is recorded as the first word vector. 第一获取：将所述的第一词向量分别输入BERT子模型的层自注意力层，获得输出的个特征，并设定每个特征的权重；First acquisition: Input the first word vector into the BERT sub-model Layer self-attention layer, get the output Features , and set the weight of each feature ; 第一计算：基于个特征，计算第一输入数据，所述第一输入数据的计算模型如下：First calculation: based on Features , calculate the first input data, the calculation model of the first input data is as follows: ； ; 其中，A为第一输入数据，为第个特征；为第个特征的权重；为第个特征；Where A is the first input data, For the Features For the The weight of each feature; For the Features 第二输入：将第一输入数据输入TextCNN子模型中，获得第一输出数据；Second input: Input the first input data into the TextCNN sub-model to obtain the first output data ; 第三输入：将第一输入数据输入BILSTM子模型中，获得第二输出数据；Third input: Input the first input data into the BILSTM sub-model to obtain the second output data ; 第二计算：将第一输出数据和第二输出数据进行线性组合，获得第三输出数据；Second calculation: convert the first output data and the second output data Perform linear combination to obtain the third output data ; 第三计算：将第三输出数据输入至T-BTM模型的全连接层，获得第四输出数据，利用Softmax函数将第四输出数据归一化，获得第一报告的风险类型。Third calculation: The third output data Input to the fully connected layer of the T-BTM model to obtain the fourth output data , use the Softmax function to convert the fourth output data Normalized to obtain the first reported risk type. 2.根据权利要求1所述的基于深度学习的风险分类方法，其特征在于，在执行训练之后，执行验证之前，还包括：2. The risk classification method based on deep learning according to claim 1, characterized in that after performing training and before performing verification, it also includes: 计算损失：包括定义损失函数、第四计算、优化、第一判断和第二判断；Calculating loss: including definition of loss function, fourth calculation, optimization, first judgment and second judgment; 定义损失函数：定义损失函数，所述损失函数的计算模型如下：Define loss function: Define loss function, the calculation model of the loss function is as follows: ； ; 其中，为正确的风险类型所对应的预测概率；为第i个错误的风险类型所对应的预测概率；M为错误的风险类型的数量；in, The predicted probability corresponding to the correct risk type; is the predicted probability corresponding to the i-th wrong risk type; M is the number of wrong risk types; 第四计算：基于风险类型和预测概率，利用损失函数计算T-BTM模型的损失；Fourth calculation: Based on the risk type and predicted probability, the loss of the T-BTM model is calculated using the loss function; 优化：使用梯度下降法更新T-BTM模型的参数，获得更新次数和最小损失；Optimization: Use the gradient descent method to update the parameters of the T-BTM model and obtain the number of updates and the minimum loss; 第一判断：判断最小损失是否小于预设最小损失阈值，若是，则执行验证；若否，则执行第二判断；First judgment: judge whether the minimum loss is less than the preset minimum loss threshold, if so, perform verification; if not, perform the second judgment; 第二判断：判断更新次数是否大于预设迭代次数阈值，若是，则执行验证；若否，则执行优化。Second judgment: judge whether the number of updates is greater than the preset iteration number threshold, if so, perform verification; if not, perform optimization. 3.根据权利要求1所述的基于深度学习的风险分类方法，其特征在于，在执行第三判断之后，执行更新训练集之前，还包括：3. The risk classification method based on deep learning according to claim 1, characterized in that after executing the third judgment and before executing the update of the training set, it also includes: 删除：在词典中删除未存入子词典的第一数据。Delete: Delete the first data in the dictionary that is not stored in the sub-dictionary. 4.根据权利要求1-3任一项所述的基于深度学习的风险分类方法，其特征在于，在执行第三计算后，还包括：4. The deep learning-based risk classification method according to any one of claims 1 to 3, characterized in that after performing the third calculation, it also includes: 第二建模：建立SPN模型，所述SPN模型包括：多头自注意力层、多头交叉注意力层和全连接层；Second modeling: establishing an SPN model, wherein the SPN model includes: a multi-head self-attention layer, a multi-head cross-attention layer, and a fully connected layer; 第六计算：将第一词向量作为多头自注意力层的输入，分别计算每个第一词向量和剩余词向量的注意力权重，将所述注意力权重作为多头交叉注意力层的输入，获得第五输出数据；The sixth calculation: take the first word vector as the input of the multi-head self-attention layer, calculate the attention weights of each first word vector and the remaining word vectors respectively, take the attention weights as the input of the multi-head cross attention layer, and obtain the fifth output data ; 第七计算：将第五输出数据输入SPN模型的全连接层，输出第六输出数据，通过Softmax函数对第六输出数据进行归一化处理，获得第一报告中实体-关系三元组的预测结果。Seventh calculation: convert the fifth output data Input the fully connected layer of the SPN model and output the sixth output data , the sixth output data is processed by the Softmax function Normalization processing is performed to obtain the prediction results of the entity-relationship triples in the first report. 5.根据权利要求4所述的基于深度学习的风险分类方法，其特征在于，在执行第六计算之后，执行第七计算之前，还包括：5. The risk classification method based on deep learning according to claim 4, characterized in that after performing the sixth calculation and before performing the seventh calculation, it also includes: 第三建模：建立ACMix模型，所述ACMix模型包括：全连接层和自注意力层，所述ACMix模型的计算模型如下：The third modeling: establishing an ACMix model, the ACMix model includes: a fully connected layer and a self-attention layer, and the calculation model of the ACMix model is as follows: ； ; 其中，为ACMix模型的全连接层输出；为ACMix模型的自注意力层输出；为ACMix模型的全连接层输出的权重；为ACMix模型的自注意力层输出的权重；为ACMix模型的输出；in, It is the fully connected layer output of the ACMix model; It is the self-attention layer output of the ACMix model; is the weight of the fully connected layer output of the ACMix model; is the weight of the self-attention layer output of the ACMix model; is the output of the ACMix model; 第八计算：将个特征作为ACMix模型的输入，获得第七输出数据；8th calculation: Features As input to the ACMix model, the seventh output data is obtained ; 第九计算：将第五输出数据和第七输出数据结合后，作为新的第五输出数据。Ninth calculation: convert the fifth output data and the seventh output data After combining, the new fifth output data . 6.根据权利要求5所述的基于深度学习的风险分类方法，其特征在于，以实体首尾的偏移量表示所述预测结果。6. The deep learning-based risk classification method according to claim 5 is characterized in that the prediction result is represented by the offset between the head and tail of the entity. 7.一种基于深度学习的风险分类系统，所述系统适用于如权利要求1-6任一项所述的方法，包括：数据采集模块，用于采集待分类的HAZOP分析报告；其特征在于，所述系统还包括：7. A risk classification system based on deep learning, the system being applicable to the method according to any one of claims 1 to 6, comprising: a data acquisition module for collecting HAZOP analysis reports to be classified; characterized in that the system further comprises: 建立模型模块，用于建立T-BTM模型，所述T-BTM模型包括：BERT子模型、TextCNN子模型、BILSTM子模型和全连接层；Establish a model module for establishing a T-BTM model, wherein the T-BTM model includes: a BERT sub-model, a TextCNN sub-model, a BILSTM sub-model, and a fully connected layer; 数据处理模块，包括第一处理单元、第二处理单元和第三处理单元；A data processing module, comprising a first processing unit, a second processing unit and a third processing unit; 第一处理单元，用于将待分类的HAZOP分析报告输入至BERT子模型，经过BERT子模型的嵌入层处理后获得第一词向量；还用于将第一词向量输入至BERT子模型的多个注意力层中，输出多个特征；A first processing unit is used to input the HAZOP analysis report to be classified into the BERT sub-model, and obtain a first word vector after being processed by the embedding layer of the BERT sub-model; and is also used to input the first word vector into multiple attention layers of the BERT sub-model to output multiple features; 第二处理单元，用于将第一输入数据输入TextCNN子模型中，获得第一输出数据；The second processing unit is used to input the first input data into the TextCNN sub-model to obtain the first output data ; 第三处理单元，用于将第一输入数据输入BILSTM子模型中，获得第二输出数据；The third processing unit is used to input the first input data into the BILSTM sub-model to obtain the second output data ; 计算模块，包括第一计算单元、第二计算单元和第三计算单元；A computing module, comprising a first computing unit, a second computing unit and a third computing unit; 第一计算单元，用于基于多个特征，计算第一输入数据；A first calculation unit, configured to calculate first input data based on a plurality of features; 第二计算单元，用于将第一输出数据和第二输出数据进行线性组合，获得第三输出数据；The second computing unit is used to convert the first output data and the second output data Perform linear combination to obtain the third output data ; 第三计算单元，用于将第三输出数据The third computing unit is used to convert the third output data 第三计算单元，用于将第三输出数据输入至全连接层，获得第四输出数据，利用Softmax函数将第四输出数据归一化，获得第一报告的风险类型。The third computing unit is used to convert the third output data Input to the fully connected layer to obtain the fourth output data , use the Softmax function to convert the fourth output data Normalized to obtain the first reported risk type.
CN202411327991.4A 2024-09-24 2024-09-24 A risk classification method and system based on deep learning Active CN118863546B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202411327991.4A CN118863546B (en) 2024-09-24 2024-09-24 A risk classification method and system based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202411327991.4A CN118863546B (en) 2024-09-24 2024-09-24 A risk classification method and system based on deep learning

Publications (2)

Publication Number Publication Date
CN118863546A CN118863546A (en) 2024-10-29
CN118863546B true CN118863546B (en) 2024-11-29

Family

ID=93160571

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202411327991.4A Active CN118863546B (en) 2024-09-24 2024-09-24 A risk classification method and system based on deep learning

Country Status (1)

Country Link
CN (1) CN118863546B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110598209A (en) * 2019-08-21 2019-12-20 合肥工业大学 Method, system and storage medium for extracting keywords
CN113807715A (en) * 2021-09-23 2021-12-17 厦门标安科技有限公司 A kind of dynamic early warning method for chemical plant risk

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117151478B (en) * 2023-10-31 2024-01-26 南京南工应急科技有限公司 A risk early warning method and system for chemical enterprises based on convolutional neural network

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110598209A (en) * 2019-08-21 2019-12-20 合肥工业大学 Method, system and storage medium for extracting keywords
CN113807715A (en) * 2021-09-23 2021-12-17 厦门标安科技有限公司 A kind of dynamic early warning method for chemical plant risk

Also Published As

Publication number Publication date
CN118863546A (en) 2024-10-29

Similar Documents

Publication Publication Date Title
CN119646178B (en) Enhanced document generation and retrieval method based on knowledge graph
CN116127090B (en) Aviation system knowledge graph construction method based on fusion and semi-supervision information extraction
CN111382575A (en) An event extraction method based on joint annotation and entity semantic information
CN111159407A (en) Method, apparatus, device and medium for training entity recognition and relation classification model
CN113806547A (en) A deep learning multi-label text classification method based on graph model
CN118964648B (en) Enhanced retrieval method and system for power vector knowledge base based on artificial intelligence
CN117933249A (en) Intelligent interaction method and system for equipment fault knowledge
CN119739867A (en) Medical knowledge graph retrieval system and method based on potential relationship reasoning
CN119046433A (en) Output method, device, equipment and storage medium for search enhancement generation type question and answer
CN119669530B (en) Knowledge graph generation-assisted teaching question answering method and system based on LLM
CN116842194A (en) An electric power semantic knowledge graph system and method
CN119067457A (en) A green energy power industry procurement document compliance inspection method and system
CN118690841A (en) A method and device for constructing an electricity safety knowledge graph
CN120632110B (en) Method, device, equipment and readable storage medium for processing archive task
CN120407775B (en) A method and system for automatically annotating text data by integrating pre-trained NLP models
CN117056510A (en) A method for automatic collection of information on multiple social conflicts and disputes
CN116955818A (en) Recommendation system based on deep learning
CN119128155B (en) A method for classifying news events
CN118863546B (en) A risk classification method and system based on deep learning
CN113378024A (en) Deep learning-based public inspection field-oriented related event identification method
CN119938846A (en) Method and device for generating question and answer based on knowledge graph
CN119003769A (en) Netizen view analysis method based on double large models
Deng Research and Application of Webpage Information Recognition Method Based on KNN Algorithm.
Che et al. A feature and deep learning model recommendation system for mobile application
CN119003789B (en) Technical methods for mining and aggregation of scientific and technological terms in professional fields

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant