CN111431821A - Method for rapidly detecting and identifying specific information in network large flow - Google Patents

Abstract

The invention discloses a method for rapidly detecting and identifying specific information in large network traffic, comprising a processor connected with a database module, an information identification module, an information storage module and a protection module, wherein the database module includes a data acquisition module, a sending module, a receiving module and a central control module; the information identification module includes an acquisition module and a screening module; the information storage module includes a data conversion unit, a communication unit, a cloud storage and a feedback system, and the protection module includes Wireless intrusion prevention systems, firewall systems. The present invention can quickly identify specific information content by using information fingerprint technology on the basis of classifying and identifying applications and protocols from large network traffic, and can greatly improve the accuracy of identifying and extracting specific information on the Internet; Information security is protected to avoid information loss.

Description

A method for rapid detection and identification of specific information in large network traffic

technical field

The invention relates to the technical field of network information processing, in particular to a method for rapidly detecting and identifying specific information in large network traffic.

Background technique

With the rapid development of the Internet and the increase of WEB information, it is necessary to perform quasi-real-time data collection, protocol classification, application identification, and feature extraction for massive Internet traffic. Just like finding a needle in a haystack, information processing technology has emerged. The page information of a large number of websites on the Internet is collected locally, and the information database and index database are established after processing, so as to respond to various searches proposed by users, and provide the information or related pointers required by users. The retrieval methods of users mainly include free-word full-text retrieval, keyword retrieval, classification retrieval and retrieval of other specific information. To this end, we propose a method to rapidly detect and identify specific information in large network traffic.

SUMMARY OF THE INVENTION

The purpose of the present invention is to solve the problems in the background technology, and propose a method for rapidly detecting and identifying specific information in large network traffic.

In order to achieve the above object, the present invention adopts the following technical scheme: a method for rapidly detecting and identifying specific information in large network traffic, comprising a processor, the processor is connected with a database module, an information identification module, an information storage module and A protection module, wherein the database module includes a data acquisition module, a sending module, a receiving module and a central control module; the information identification module includes an acquisition module and a screening module; the information storage module includes a data conversion unit, a communication unit, a cloud storage and a feedback system, the protection module includes a wireless intrusion protection system and a firewall system module.

In the above-mentioned method for rapidly detecting and identifying specific information in large network traffic, in the database module, the data collection module is used to collect data on the Internet, preliminarily classify the data, and classify the data of the same category. After the data is compressed, it is transmitted to the sending server; after the sending module receives the data sent by the data acquisition module, it saves the received data and establishes a sending queue table according to the data category and the receiving time; The receiving module groups are numbered, and the receiving module groups are classified according to the data. Different receiving module groups only receive data of the same type and save the received data; the central control module collects the status of the receiving modules, and establishes a receiving team according to the collected status. List, the receiving queue table saves the working status and busyness of each receiving module.

In the above-mentioned method for rapidly detecting and identifying specific information in large network traffic, in the information identification module, the acquisition module is used for acquiring a network information set; the screening module is used for, according to the specified information number, Filter the published information that matches the specified information number in the network information collection.

In the above-mentioned method for rapidly detecting and identifying specific information in large network traffic, in the information storage module, the data conversion unit can compress the stored information into an electrical signal, and send a request signal to the communication unit , the communication unit sends the request signal to the cloud storage, the cloud storage receives the request signal, the cloud storage receives the upload request and feeds back the signal allowing information uploading to the communication unit through the feedback system, and the communication unit receives the feedback signal Then start uploading information to achieve the effect of data saving.

In the above-mentioned method for rapidly detecting and identifying specific information in large network traffic, the database module is further provided with a data redundancy judgment module; the data redundancy judgment module is connected with the receiving module and the data acquisition module, and the data acquisition module is The collected data is judged redundantly. If the data stored in the receiving module is the same as the data collected by the data acquisition module, the same data will be discarded.

In the above-mentioned method for quickly detecting and identifying specific information in a large network traffic, the central control module is further provided with a power control module, and the power control module stores the state of the receiving module according to the receiving queue table and the data of the sending queue table The class controls the corresponding idle receiving server to be in the standby state.

The above-mentioned method for rapidly detecting and identifying specific information in large network traffic includes the following steps:

S1. Information collection and construction of the database: collect the IP addresses of different computers or smart devices, as well as the computer or smart device attribute data group, and convert all elements in the computer or smart device attribute data group into digital format after processing. Attribute data groups of computers or smart devices in digital format are classified and saved to build a database;

S2. Rapid detection and identification of information: input the acquired network information set into the database module, and filter the published information matching the specified information number in the network information set;

S3. Information storage: the data conversion unit can compress the stored information into an electrical signal, and send a request signal to the communication unit, the communication unit sends the request signal to the cloud storage, the cloud storage receives the request signal, and the cloud storage receives the upload request and passes the feedback The system feeds back the signal that allows information to be uploaded to the communication unit, and the communication unit starts to upload the information after receiving the feedback signal, and saves the data to prevent information loss;

S4. Information protection: The protection module can protect network security in real time. By controlling the IP address configured on the network card, it can give real-time warning to the IP address detected by the firewall, and back up the warning information to the intranet web server for log records.

Compared with the existing technology, the advantages of this method of rapidly detecting and identifying specific information in large network traffic are: classifying the data in the early stage, establishing a sending queue table, and then cyclically detecting the sending queue table, so that the The user's information is well identifiable, and at the same time, the readability and naturalness of the user's information are not damaged or seldom damaged, and the information belonging to a specific user can be quickly found from a large number of similar information, so that the obtained information can be When there is a large amount of similar information, using information fingerprints to accurately find out the information of a specific user can greatly improve the accuracy of identifying and extracting specific information on the Internet; and by setting up a protection module to protect information security and avoid information loss.

Description of drawings

FIG. 1 is a structural block diagram of a method for rapidly detecting and identifying specific information in a large network traffic according to the present invention.

Detailed ways

The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

Example

Referring to Fig. 1, a method for rapidly detecting and identifying specific information in large network traffic includes a processor, and the processor is connected with a database module, an information identification module, an information storage module and a protection module; wherein the database module includes a data acquisition module, The sending module, the receiving module and the central control module, the data collection module is used to collect data on the Internet, preliminarily classify the data, and compress the data of the same category and transmit it to the sending server; the sending module receives the data sent by the data collection module. After the data, the received data is saved and the sending queue table is established according to the data type and receiving time; the receiving modules are grouped according to the data type, and the receiving module groups are numbered. The receiving module groups are classified according to the data, and different receiving module groups only receive the same category data, and save the received data; the central control module collects the status of the receiving module, and establishes a receiving queue table according to the collected status, and the receiving queue table saves the working status and busyness of each receiving module; the database module also sets There is a data redundancy judgment module; the data redundancy judgment module is connected with the receiving module and the data acquisition module, and performs redundant judgment on the data collected by the data acquisition module. If the data stored in the receiving module is the same as the data collected by the data acquisition module, it will be The same data is discarded; a power control module is also set in the central control module, and the power control module controls the corresponding idle receiving server to be in a standby state according to the state of the receiving module saved in the receiving queue table and the data type of the sending queue table;

The information identification module includes an acquisition module and a screening module, the acquisition module is used to acquire the network information set; the screening module is used to filter the published information matching the specified information number in the network information set according to the specified information number;

The information storage module includes a data conversion unit, a communication unit, a cloud storage and a feedback system. The data conversion unit can compress the stored information into electrical signals and send a request signal to the communication unit, which sends the request signal to the cloud storage, and the cloud storage After receiving the request signal, the cloud storage receives the upload request and feeds back the signal allowing information uploading to the communication unit through the feedback system. After the communication unit receives the feedback signal, it starts to upload the information to achieve the effect of data preservation.

In the present invention, the method for rapidly detecting and identifying specific information in large network traffic includes the following steps:

S1. Information collection and construction of the database: collect the IP addresses of different computers or smart devices, as well as the computer or smart device attribute data group, and convert all elements in the computer or smart device attribute data group into digital format after processing. Attribute data groups of computers or smart devices in digital format are classified and saved to build a database;

S2. Rapid detection and identification of information: input the acquired network information set into the database module, and filter the published information matching the specified information number in the network information set;

S3. Information storage: the data conversion unit can compress the stored information into an electrical signal, and send a request signal to the communication unit, the communication unit sends the request signal to the cloud storage, the cloud storage receives the request signal, and the cloud storage receives the upload request and passes the feedback The system feeds back the signal that allows information to be uploaded to the communication unit, and the communication unit starts to upload the information after receiving the feedback signal, and saves the data to prevent information loss;

S4. Information protection: The protection module can protect network security in real time. By controlling the IP address configured on the network card, it can give real-time warning to the IP address detected by the firewall, and back up the warning information to the intranet web server for log records.

The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention. Inside.

Claims (7)

1. A method for rapidly detecting and identifying specific information in network large flow comprises a processor, and is characterized in that the processor is connected with a database module, an information identification module, an information storage module and a protection module, wherein the database module comprises a data acquisition module, a sending module, a receiving module and a central control module; the information identification module comprises an acquisition module and a screening module; the information storage module comprises a data conversion unit, a communication unit, a cloud storage and a feedback system, and the protection module comprises a wireless intrusion protection system and a firewall system.

2. The method for rapidly detecting and identifying the specific information in the large flow rate of the network according to claim 1, wherein in the database module, the data acquisition module is used for acquiring data on the internet, preliminarily classifying the data, compressing the data of the same category and transmitting the data to the sending server; after the sending module receives the data sent by the data acquisition module, the received data are stored, and a sending queue list is established according to the data type and the receiving time; the receiving modules are grouped according to data types, the receiving module groups are numbered, the receiving module groups are classified according to the data, different receiving module groups only receive the data of the same type, and the received data are stored; the central control module collects the states of the receiving modules, establishes a receiving queue list according to the collected states, and stores the working states and the busy degree of each receiving module in the receiving queue list.

3. The method for rapidly detecting and identifying specific information in large traffic volume of network according to claim 1, wherein in the information identification module, the obtaining module is used for obtaining network information set; the screening module is used for screening the release information matched with the specified information number in the network information set according to the specified information number.

4. The method according to claim 1, wherein in the information storage module, the data conversion unit compresses the stored information into an electrical signal and sends a request signal to the communication unit, the communication unit sends the request signal to a cloud storage, the cloud storage receives the request signal, the cloud storage receives an upload request and feeds back a signal allowing information to be uploaded to the communication unit through a feedback system, and the communication unit starts uploading the information after receiving the feedback signal, so that the effect of data storage is achieved.

5. The method for rapidly detecting and identifying the specific information in the large flow rate of the network according to claim 1, wherein a data redundancy judgment module is further arranged in the database module; the data redundancy judgment module is connected with the receiving module and the data acquisition module, redundancy judgment is carried out on data acquired by the data acquisition module, and if the data stored in the receiving module is the same as the data acquired by the data acquisition module, the same data is discarded.

6. The method according to claim 1, wherein a power control module is further disposed in the central control module, and the power control module controls the idle receiving servers to be in a standby state according to the state of the receiving module stored in the receiving queue table and the data type of the sending queue table.

7. The method for rapidly detecting and identifying the specific information in the large flow of the network according to claim 1, comprising the following steps:

s1, information acquisition and construction of the database: collecting IP addresses of different computers or intelligent equipment and an attribute data set of the computers or the intelligent equipment, converting all elements in the attribute data set of the computers or the intelligent equipment into a digital format after processing, and classifying and storing the attribute data set of the computers or the intelligent equipment in the digital format so as to construct a database;

s2, rapid detection and identification of information: inputting the acquired network information set into a database module to screen release information matched with the specified information number in the network information set;

s3, information storage: the data conversion unit can compress the stored information into an electric signal and send a request signal to the communication unit, the communication unit sends the request signal to the cloud storage, the cloud storage receives the request signal, the cloud storage receives the uploading request and feeds back a signal allowing information to be uploaded to the communication unit through the feedback system, the communication unit starts to upload the information after receiving the feedback signal and stores the data to prevent information loss;

s4, information protection: the protection module can protect network safety in real time, pre-warns the IP address detected by the firewall in real time by controlling the IP address configured on the network card, and performs log record backup on the pre-warning information in an intranet Web server.

Images