CN110321394A - The network security data method for organizing and computer storage medium of knowledge based map - Google Patents

Abstract

The invention discloses a network security data organization method and a computer storage medium based on a knowledge graph. The method comprises the following steps: 1) collecting massive network security data, and preprocessing by cleaning and filtering; 2) constructing network security data Knowledge base; 3) Formulate a feature template according to the context information of the data, and combine the feature template to generate a word vector from the data; 4) Input the word vector into the BiLSTM model, and complete network security entity recognition and network security entity relationship extraction through underlying parameter sharing; 5 ) combines the results of network security entity identification and network security entity relationship extraction with the network security knowledge base to construct a network security data knowledge graph. The invention solves the problems of weak data correlation, low organizational efficiency and single expression form faced by the network security data organization and management in the prior art, effectively improves the data correlation and fusion degree, and improves the data organization efficiency.

Description

Network security data organization method and computer storage medium based on knowledge graph

technical field

The invention relates to a network security data organization method and a computer storage medium, in particular to a network security data organization method and computer storage medium based on a knowledge graph.

Background technique

With the continuous innovation and improvement of computer network and communication technology, cyberspace is also facing unprecedented opportunities and challenges. Faced with such a severe security situation, network security data that reflects network trends in real time has become a key link to gain insight into network conditions, analyze network anomalies, and evaluate network environment. Network security data specifically refers to the time-sensitive and relevant data that is generated by the interaction and fusion of the three-dimensional world of "human, machine, and object" in cyberspace, and which can be obtained on the Internet and reflect the network status. As the carrier of network security information, the organization and association of network security data is of great significance to the development and utilization of information resources.

However, most of the existing data organization methods focus on the storage of data, and the association rules between data are not considered enough, resulting in a decrease in the degree of fusion between data, so the value of data cannot be fully reflected, and eventually it will show "information overload" , lack of knowledge, etc. In addition, most of the above data organization methods are limited to specific scenarios in a professional field, and there is little research on data organization in the multi-source heterogeneous, massive and complex network security environment. With the rise of emerging technologies such as artificial intelligence and machine learning, cyberspace security issues have become increasingly urgent, and a comprehensive and efficient data organization method is urgently needed to provide data security.

SUMMARY OF THE INVENTION

Purpose of the invention: The technical problem to be solved by the present invention is to provide a network security data organization method and computer storage medium based on knowledge graph, which solves the problems of weak data correlation and organizational efficiency faced by network security data organization and management in the prior art. Problems such as low level and single form of expression effectively improve the correlation and integration of data, and improve the efficiency of data organization.

Technical solution: The method for organizing network security data based on knowledge graph according to the present invention includes the following steps:

(1) Collect massive amounts of network security data and preprocess them by cleaning and filtering;

(2) Build a network security knowledge base;

(3) formulating a feature template according to the context information of the data, and generating a word vector from the data in combination with the feature template;

(4) Input the word vector into the BiLSTM model, and complete network security entity recognition and network security entity relationship extraction through underlying parameter sharing;

(5) Combine the results of network security entity identification and network security entity relationship extraction in step (4) with the network security knowledge base to construct a network security data knowledge graph.

Further, the network security data described in step (1) includes collection from five aspects: network asset information, network threat information, network status information, network vulnerability information and security event information.

Further, the preprocessing in step (1) is specifically:

(1) Select the original data in a standardized format as the basis for formulating filtering rules, and screen out data with irregular data values, data types and data formats according to the corresponding regular expressions defined by them, and correct them. Uncorrectable data to discard;

(2) Use Bloom-Filter algorithm to remove duplicate data;

(3) The incomplete data values are supplemented by means of mean value interpolation. For categorical data, the mode is used for interpolation, and for quantitative data, the mean value is used for interpolation.

Further, the network security knowledge base described in step (2) includes five ontologies of physical security, host security, network structure security, application security and data security.

Further, the feature template is a set of identification words formed by the current identification word and the number of identification words set before and after the current identification word.

Further, the method for generating a word vector from the data in step (3) is specifically: reading the data according to the feature template, and generating the word vector through the Bert model.

Further, the method for completing network security entity identification and network security entity relationship extraction in step (4) is: input the word vector into the BiLSTM model for network security entity recognition, and the BiLSTM model includes an input layer, a feature template, a word embedding layer, BiLSTM layer and CRF layer, and then replace the CRF layer of the BiLSTM model with the Attention layer and the Softmax layer for output, and complete the network security entity relationship extraction.

Further, the network security data knowledge graph described in step (5) includes two parts, one is a general knowledge graph, including previously known network vulnerability information, attack threat information and security announcement information; the other is an extended knowledge graph, It mainly includes network node information, network topology information, network connectivity information, and network operation and maintenance information.

Further, step (5) also includes storing the network security data knowledge graph using an OrientDB graph database.

The computer storage medium of the present invention has a computer program stored thereon, and when the computer program is executed by a computer processor, the above-mentioned network security data organization method based on knowledge graph is realized.

Beneficial effects: Aiming at the characteristics of massive, polymorphic and heterogeneous network security data, the present invention utilizes the establishment of a knowledge graph to effectively improve the correlation and degree of fusion between data, enhance the efficiency of data organization, and reduce the burden of a large number of manual annotations. It reflects a good data management effect, gives full play to the value of data, and then promotes the transformation of data advantages into decision-making advantages through knowledge advantages. The invention has the advantages of strong data correlation and high degree of integration, which can improve the efficiency of data organization and maximize the value of data.

Description of drawings

1 is an overall architecture diagram of an embodiment of the present invention;

Fig. 2 is the network security knowledge base structure diagram of this embodiment;

Fig. 3 is the network security entity recognition model diagram of the present embodiment;

Fig. 4 is the network security relation extraction model diagram of the present embodiment;

FIG. 5 is a structural diagram of a knowledge graph of network security data in this embodiment.

Detailed ways

The method steps of the embodiment of the present invention are shown in Figure 1, and the specific steps are:

Step 1. Collect massive amounts of network security data from different levels as data sources, and then preprocess the accessed data by cleaning and filtering.

Among them, network security data is collected from five aspects: network asset information, network threat information, network status information, network vulnerability information and security event information. Among them, vulnerability information mainly comes from various vulnerability databases such as CVE, NVD, and China National Vulnerability Database; attack threat data and security bulletin information are collected from information security websites and security emergency response centers. The former mainly includes Kanxue Forum, Toast Forum and Freebuf, The latter are mainly the National Internet Emergency Response Center, Tencent, and Ctrip Security Emergency Response Center; other data use open source tools for asset scanning detection, vulnerability scanning, and topology discovery to obtain relevant raw data on the network structure.

For the acquired data source, the specific steps of cleaning and filtering are:

(1) Preliminary filtering: Select the original data with standard format as the basis for formulating filtering rules, and screen out irregular data values, data types, and data formats according to the corresponding regular expressions defined by them, and correct them. Uncorrectable data is discarded.

(2) Redundancy removal: The Bloom-Filter algorithm is used to remove duplicate data. The bottom layer of the Bloom-Filter algorithm uses a bitmap. When an element is added to the set, the element is mapped into a bit array through K Hash functions. K points and set them to 1. When searching for redundancy, it is only necessary to check whether these points are all 1 to determine whether there is redundancy in the set. If these points appear 0, the probability of the existence of the retrieved element is 0; if the results are all 1, the probability of the existence of the retrieved element is very high.

(3) Compensation of incompleteness: The incomplete data value is processed by means of mean interpolation. For categorical data, mode is used for interpolation, and for quantitative data, mean value is used for interpolation.

Step 2: Construct a network security knowledge base based on the knowledge and experience in the field, comprehensively consider the context information of the data, formulate a feature template and generate a word vector. The knowledge and experience in this field is the knowledge summed up by experts in the field of practical experience in the field of network security, which can be obtained by summarizing the existing database in the actual process.

Among them, the network security knowledge base model is shown in Figure 2. The entire knowledge base contains five ontologies, namely physical security, host security, network structure security, application security and data security. Each part works independently of each other, and at the same time forms a large whole through the network system. The details of each ontology are as follows:

(1) Physical security: including the security of the environment where the entire network is located and the security of related equipment to which the network belongs;

(2) Host security: including operating system security and file security of servers and terminal equipment;

(3) Network structure security: including the entire network topology security, access control security, intrusion prevention security and equipment protection security;

(4) Application security: including network application software security and system application software security;

(5) Data security: including data integrity security and data confidentiality security.

Based on the network security knowledge base constructed according to the experience of experts, the local context features are extracted through preliminary screening of pre-constructed ontology relationships to form feature templates. Among them, the specific design of the feature template is as follows:

Described feature template is the set of recognition word that the current recognition word and the number of recognition words set before and after the current recognition word are formed. Make settings. That is, several adjacent positions of the current recognition word are defined as “monitoring windows”, and the size of the feature template, that is, the size of the monitoring window is positively correlated with the contained context information, and the “monitoring window” in this embodiment is set to 7.

The feature template of this embodiment is:

x[0,-3],x[0,-2],x[0,-1],x[0,0],x[0,1],x[0,2],x[0,3 ]

Among them, the format of the feature template is x[row,col], and the source of the feature is demarcated by row and col, row represents the row relative to the current character position, that is, the row value of the current row is 0, and col corresponds to the column of the selected feature.

Finally, using the Bert model to pre-train the word vector file, the result is that the input word or word is represented as a numerical vector of a certain length, and contains potential semantic relations.

Step 3. Taking the word vector combined with the local context feature as input, the BiLSTM model is used to complete the network security entity recognition and network security entity relationship extraction through the underlying parameter sharing.

Among them, the word vector combined with local context features is used as the input of the BiLSTM neural network for entity recognition, as shown in Figure 3. The network security entity recognition process is realized by the BiLSTM-CRF model, which includes the input layer, feature template, word embedding layer, BiLSTM layer and CRF layer. The definition of the correct rate output of the cybersecurity entity recognition model is as follows:

For a given input sequence X=(X ₁ ,X ₂ ,...,X _n ), suppose A is the score matrix of the output of a BiLSTM network of size n×k, k is the number of label types, A _i,j is the word The score for the jth label of . Then for the predicted label y=(y ₁ , y ₂ ,...,y _n ), its score is defined as:

where T is the transition score matrix of order k+2, T _i,j represents the score of the transition from label i to label j, y ₀ and _yn are the labels added at the beginning and end of the sentence, the probability of generating the label sequence y for:

Finally, the maximum likelihood estimation function is used to calculate the log probability of maximizing the correct label sequence as:

The specific training process of the network security entity recognition model is as follows:

(1) Enter the epoch cycle, and then enter the batch cycle;

(2) Initialization parameters;

(3) Extract character features according to the feature template and Bert model;

(4) Using the BiLSTM-CRF algorithm to forward transfer the extracted features;

(5) Use the CRF algorithm to calculate the probability of the global correct label;

(6) Using the BiLSTM-CRF algorithm to pass back the extracted features;

(7) Update parameters;

(8) End the batch loop and the epoch loop.

On the basis of the entity recognition model, by retaining the bottom layer and realizing parameter sharing, the CRF layer is replaced with the Attention layer and the Softmax layer for output, and a network security entity relationship extraction model is constructed, as shown in Figure 4. Among them, the Attention layer can generate a weight vector, and by multiplying this weight vector, the vocabulary-level features in each iteration are merged into sentence-level features, and the weight matrix of the Attention layer can be obtained as follows:

M=tanh(H)

α=softmax(ω ^T M)r=Hα ^T

in, d ^ω is the dimension of the word vector, and ω ^T is the transpose of the parameter vector obtained by training. Then, the sentences for classification are represented as follows:

h ^* =tanh(r)

Finally, a Softmax classifier is used to predict the labels, taking the h ^* obtained from the previous layer as input.

f(y|S)=softmax(W _s h ^* +b _s )

So, is the entity relationship category to be solved, where W _s represents the learning weight and b _s represents the bias parameter.

Step 4. Combine the results of entity recognition and relationship extraction with expert knowledge, build a knowledge graph of network security data and store it, and finally complete the organization of network security data.

Among them, the results of entity recognition and relation extraction are merged with expert knowledge, duplicate knowledge is filtered out and a complete knowledge graph is constructed. The designed network security knowledge graph includes two parts, as shown in Figure 5. One is the general knowledge graph, that is, the previously known network vulnerability information, attack threat information and security announcement information. The second is an extended knowledge graph covering the network structure, which mainly includes network node information, network topology information, network connectivity information, and network operation and maintenance information. Finally, the storage of the knowledge graph adopts the OrientDB graph database to organize and manage network security data.

If the embodiments of the present invention are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention may be embodied in the form of software products in essence or the parts that make contributions to the prior art. The computer software products are stored in a storage medium and include several instructions for A computer device (which may be a personal computer, a server, or a network device, etc.) is caused to execute all or part of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, removable hard disk, read only memory (ROM, Read Only Memory), magnetic disk or optical disk and other media that can store program codes. As such, embodiments of the present invention are not limited to any particular combination of hardware and software.

Correspondingly, embodiments of the present invention also provide a computer storage medium on which a computer program is stored. When the computer program is executed by the processor, the aforementioned method for organizing network security data based on knowledge graph can be implemented. For example, the computer storage medium is a computer-readable storage medium.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Claims (10)

1. A network security data organization method based on knowledge graph is characterized by comprising the following steps:

(1) acquiring mass network security data, and preprocessing the data in a cleaning and filtering mode;

(2) constructing a network security knowledge base;

(3) formulating a feature template according to the context information of the data, and generating a word vector from the data by combining the feature template;

(4) inputting the word vector into a BilSTM model, and completing network security entity identification and network security entity relationship extraction through bottom layer parameter sharing;

(5) and (4) combining the results of the network security entity identification and the network security entity relation extraction in the step (4) with the network security knowledge base to construct a network security data knowledge graph.

2. The knowledge-graph-based network security data organization method according to claim 1, wherein: the network security data in the step (1) is acquired from five aspects of network asset information, network threat information, network state information, network vulnerability information and security event information.

3. The knowledge-graph-based network security data organization method according to claim 1, wherein the preprocessing in the step (1) is specifically:

(1) selecting original data with a standard format as a formulation basis of a filtering rule, screening out data with an irregular data value, a data type and a data format according to a regular expression corresponding to the definition of the original data, and correcting the data;

(2) removing repeated data by adopting a Bloom-Filter algorithm;

(3) and (3) complementing the incomplete data value by adopting a mean interpolation mode, carrying out interpolation on the classified data by adopting a mode, and carrying out interpolation on the quantitative data by adopting a mean value.

4. The knowledge-graph-based network security data organization method according to claim 1, wherein: the network security knowledge base in the step (2) comprises five bodies of physical security, host security, network structure security, application security and data security.

5. The knowledge-graph-based network security data organization method according to claim 1, wherein: the characteristic template is a set of identifiers formed by the current identifier and identifiers with the number set before and after the current identifier.

6. The knowledge-graph-based network security data organization method according to claim 1, wherein the method for generating word vectors by the data in the step (3) is specifically as follows: and reading data according to the characteristic template, and generating a word vector through a Bert model.

7. The knowledge-graph-based network security data organization method according to claim 1, wherein the method for performing network security entity identification and network security entity relationship extraction in the step (4) comprises: inputting the word vector into a BilSTM model for network security entity identification, wherein the BilSTM model comprises an input layer, a characteristic template, a word embedding layer, a BilSTM layer and a CRF layer, and then replacing the CRF layer of the BilSTM model with an Attention layer and a Softmax layer for output, thereby completing network security entity relationship extraction.

8. The knowledge-graph-based network security data organization method according to claim 1, wherein: the network security data knowledge graph in the step (5) comprises two parts, wherein one part is a general knowledge graph which comprises previously known network vulnerability information, attack threat information and security bulletin information; and the other is to expand the knowledge graph, which mainly comprises network node information, network topology information, network communication information and network operation and maintenance information.

9. The knowledge-graph-based network security data organization method according to claim 1, wherein the step (5) further comprises storing the network security data knowledge graph using an OrientDB graph database.

10. A computer storage medium having a computer program stored thereon, characterized in that: the computer program, when executed by a computer processor, implementing the method of any one of claims 1 to 9.