CN107579960A - A kind of data filtering method and device - Google Patents
A kind of data filtering method and device Download PDFInfo
- Publication number
- CN107579960A CN107579960A CN201710723706.4A CN201710723706A CN107579960A CN 107579960 A CN107579960 A CN 107579960A CN 201710723706 A CN201710723706 A CN 201710723706A CN 107579960 A CN107579960 A CN 107579960A
- Authority
- CN
- China
- Prior art keywords
- data message
- data
- keywords
- virus
- keyword
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 238000001914 filtration Methods 0.000 title claims abstract description 44
- 241000700605 Viruses Species 0.000 claims abstract description 123
- 238000012795 verification Methods 0.000 claims description 2
- 239000000284 extract Substances 0.000 abstract description 5
- 238000001514 detection method Methods 0.000 abstract 1
- 230000000717 retained effect Effects 0.000 abstract 1
- 230000005540 biological transmission Effects 0.000 description 21
- 238000010586 diagram Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 3
- 230000011664 signaling Effects 0.000 description 3
- 230000002159 abnormal effect Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 239000000463 material Substances 0.000 description 2
- 238000004891 communication Methods 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000005242 forging Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000007781 pre-processing Methods 0.000 description 1
Landscapes
- Information Transfer Between Computers (AREA)
Abstract
The present invention relates to a kind of data filtering method and device.This method includes：The data message that receiving end/sending end is sent；Extract at least one first class keywords in data message；At least one first class keywords are matched with the keyword in pre-configured antistop list respectively；If during Keywords matching at least one first class keywords in the keyword of the first predetermined number and pre-configured antistop list, data message is sent to receiving terminal；Otherwise, whether detection data message carries virus；And whether virus is carried according to data message, it is determined that directly deleting data message, or data message is preserved to local.With it, can determine whether data message carries virus.If carrying virus, directly give up the message.Virus is not carried, then is retained to local.Further analyzed.The filtering to data message is achieved in, ensures that valid data message is sent to receiving terminal.
Description
Technical Field
The invention relates to the technical field of Internet of things, in particular to a data filtering method and device.
Background
The internet of things contains a wide variety of types of interconnected devices. These devices generate a vast amount of data signals. It is a cumbersome and complicated procedure in itself if it is desired to derive valid data signals from these large data signals. Moreover, the server is overloaded due to the processing of a large amount of data, and the processing speed is reduced or even delayed, so that more data is processed. Such a vicious cycle eventually causes the system to crash. In fact, many of the massive data signals are useless signals, even carried virus signals, and if the signals are filtered and then transmitted to the server for processing, the data processing rate can be greatly improved, and the network can be prevented from being threatened by viruses and the like. How to filter out these unwanted signals becomes a technical problem to be solved urgently.
Disclosure of Invention
In order to solve the technical problem, the invention provides a data filtering method and a data filtering device.
In a first aspect, the present invention provides a data filtering method, including: receiving a data message sent by a sending end;
extracting at least one first-class keyword in the data message;
matching at least one first class keyword with keywords in a pre-configured keyword table respectively;
if a first preset number of keywords in at least one first type of keywords are matched with keywords in a pre-configured keyword table, sending the data message to a receiving end;
otherwise, detecting whether the data message carries viruses or not;
and determining to directly delete the data message or store the data message to the local according to whether the data message carries viruses or not.
According to the data filtering method provided by the embodiment of the invention, at least one first class keyword in the data message is extracted to be matched with the keyword in the preset keyword table. If the preset number of keywords in the extracted keywords all exist in the pre-configured keyword table, the data message is the data message to be received by the receiving end. Otherwise, it is likely that the data message itself is problematic or that the source is problematic. Further analysis is required to determine whether the data message carries a virus. If the data message carries the virus, the virus is directly deleted, if the virus is not detected, the data message cannot be completely determined to be a normal data message, and the virus which cannot be identified by the system temporarily is possibly carried. And need to be saved locally for further analysis by the staff.
Further, before sending the data message to the receiving end, the method further comprises: and eliminating redundant fields in the data messages, and compressing the data messages with the redundant fields eliminated.
In the above implementation, the data packet may include a header, a valid field, an end field, and some other fields. Some fields actually occupy a plurality of bytes and do not play any role. Then, the redundant fields in the data packets can be removed, and the removed redundant fields are compressed. And finally, sending the compressed data message to a receiving end. Therefore, the resource occupancy rate is reduced, and the data transmission efficiency is improved.
In a second aspect, the present invention provides a data filtering apparatus, the apparatus comprising:
a receiving unit, configured to receive a data packet sent by a sending end;
the extraction unit is used for extracting at least one first-class keyword in the data message;
the processing unit is used for respectively matching at least one first-class keyword with keywords in a pre-configured keyword table;
the sending unit is used for sending the data message to a receiving end when a first preset number of keywords in at least one first-class keyword are matched with keywords in a pre-configured keyword table;
the processing unit is further used for determining whether the data message carries viruses or not when no first preset number of keywords in the at least one first type of keywords are matched with keywords in a pre-configured keyword table;
and determining to directly delete the data message or store the data message to the local according to whether the data message carries viruses or not.
According to the data filtering device provided by the embodiment of the invention, the processing unit matches at least one first type keyword in the data message extracted by the extraction unit with a keyword in a preset keyword table. If the preset number of keywords in the extracted keywords all exist in the pre-configured keyword table, the data message is the data message to be received by the receiving end. Otherwise, it is likely that the data message itself is problematic or that the source is problematic. Further analysis is required to determine whether the data message carries a virus. If the data message carries the virus, the virus is directly deleted, if the virus is not detected, the data message cannot be completely determined to be a normal data message, and the virus which cannot be identified by the system temporarily is possibly carried. And need to be saved locally for further analysis by the staff.
Further, the processing unit is further configured to: and eliminating redundant fields in the data messages, and compressing the data messages with the redundant fields eliminated.
In the above implementation, the data packet may include a header, a valid field, an end field, and some other fields. Some fields actually occupy a plurality of bytes and do not play any role. Then, the redundant fields in the data messages are removed by the processing unit, and the removed redundant fields are compressed. And finally, sending the compressed data message to a receiving end. Therefore, the resource occupancy rate is reduced, and the data transmission efficiency is improved.
Drawings
FIG. 1 is a block diagram of a data filtering system according to an embodiment of the present invention;
fig. 2 is a schematic signaling flow diagram of a data filtering method according to an embodiment of the present invention;
FIG. 3 is a flow chart of a data filtering method according to an embodiment of the present invention;
FIG. 4 is a schematic flow chart of another data filtering method according to an embodiment of the present invention;
FIG. 5 is a schematic flow chart of another data filtering method according to an embodiment of the present invention;
FIG. 6 is a flow chart of another data filtering method according to an embodiment of the present invention;
fig. 7 is a schematic structural diagram of a data filtering apparatus according to an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth such as particular system structures, interfaces, techniques, etc. in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
Fig. 1 is an architecture diagram of a data filtering system according to an embodiment of the present invention.
As shown in fig. 1 in detail, the system includes: a sending end 10 of a data message, a receiving end 20 of a data message, and a forwarding node 30.
The data message sending end 10 and the receiving end 20 are two opposite entities. For example, the transmitting end 10 may be an intelligent terminal, and the receiving end 20 is a base station. Alternatively, the transmitting end 10 is a base station and the receiving end 20 is an intelligent terminal. Still alternatively, the sending end 10 may be an intelligent terminal, and the receiving end 20 is a server; conversely, the transmitting end 10 is a server, and the receiving end 20 is an intelligent terminal. Of course, the transmitting end 10 and the receiving end 20 may be both intelligent terminals. In short, during the data transmission process, the roles of the sender 10 and the receiver 20 are completely different according to different application scenarios. However, no matter what the sender 10 and the receiver 20 are, the data transmission process is based on network transmission. During network transmission, the forwarding node 30 is needed to support data transmission. Therefore, the data transmission process includes the transmitting end 10, the receiving end 20 and the forwarding node 30.
In this embodiment, for example, an application scenario is set as data transmission based on the internet of things, the sending end 10 is defined as an intelligent terminal, and the receiving end 20 is defined as a server.
When the sending end transmits the data packet to the forwarding node 30, the forwarding node needs to perform some preprocessing so as to shorten the number of bytes occupied by the data packet as much as possible. Meanwhile, in order to ensure that the transmitted data message is safe and effective, the server is prevented from sending out wrong instructions after receiving wrong data messages, and further unnecessary influences are caused to users.
Specifically, in order to introduce the processing performed in the forwarding node 30 and the interaction process with other components in the system in more detail, the embodiment of the present invention further provides a signaling flow diagram of the data filtering method. Of course, for the sake of simplicity, only the communication connection between one intelligent terminal, one forwarding node and the server is taken as an example for description. As shown in fig. 2, the method specifically includes:
step 1, a forwarding node receives a data message sent by an intelligent terminal.
And 2, extracting at least one first-class keyword in the data message.
Specifically, the data message may include a plurality of keywords, for example, the transmitted data message is data related to power consumption in a certain user's home, and includes, for example, the power consumption of the user's home in the last several months, the power consumption used in the current month, the remaining amount of power rate, and the like. Extracting at least one first-class keyword in the data message, and then respectively matching the at least one first-class keyword with keywords in a pre-configured keyword table, namely step 3.
And 3, respectively matching at least one first class keyword with keywords in a pre-configured keyword table.
And 4, if the keywords with the first preset number in the at least one first-class keyword are matched with the keywords in the pre-configured keyword table, sending the data message to a receiving end.
Specifically, if a first preset number of keywords in at least one first-class keyword match with keywords in a pre-configured keyword table, it is determined that the data packet is a normal data packet, and the normal data packet may be sent to the server through the forwarding node.
However, when none of the first predetermined number of keywords in the at least one first category of keywords matches a keyword in the pre-configured keyword table, it indicates that the data packet may have a problem itself or a problem with the data source. Then step 5 needs to be performed.
And 5, detecting whether the data message carries viruses or not.
And 6, determining to directly delete the data message or store the data message to the local according to whether the data message carries viruses or not.
The data messages are transmitted through the intelligent terminal, the forwarding nodes are used for filtering the data messages, the data messages which possibly have danger are intercepted, whether the data messages have viruses or not is further detected, if the viruses exist, the data messages are deleted, otherwise, the data messages are stored to the local, and further analysis is carried out. And determining whether the data message is sent to the server or not until the data packet message is determined to be not dangerous. Or keep a copy to local as backup. When the data message is received again next time, the data packet message can be directly sent to the server after determining that no danger exists in the data packet message. And if the data message is not dangerous, directly transmitting the data message to the server.
As can be seen from the above system signaling flow, in the system, the component that plays a role of being turned off is a forwarding node. Hence, in the following, the method steps performed by the forwarding node will be described in detail.
Fig. 3 is a schematic flow chart of a data filtering method according to an embodiment of the present invention. As shown in fig. 3, the method includes:
step 310, receiving a data message sent by a sending end.
At step 320, at least one first type keyword in the data message is extracted.
Specifically, the data message may include a plurality of keywords, for example, the transmitted data message is data related to power consumption in a certain user's home, and includes, for example, the power consumption of the user's home in the last several months, the power consumption used in the current month, the remaining amount of power rate, and the like. Extracting at least one first-class keyword from the data message, and then matching the at least one first-class keyword with the keywords in the pre-configured keyword table, respectively, that is, step 330.
At least one first category keyword is matched with keywords in a pre-configured keyword table, respectively, in step 330.
Step 340, if the first preset number of keywords in the at least one first type of keywords match with the keywords in the pre-configured keyword table, sending the data message to the receiving end.
Specifically, if a first preset number of keywords in at least one first-class keyword match with keywords in a pre-configured keyword table, it is determined that the data packet is a normal data packet, and the normal data packet may be sent to the server through the forwarding node.
However, when none of the first predetermined number of keywords in the at least one first category of keywords matches a keyword in the pre-configured keyword table, it indicates that the data packet may have a problem itself or a problem with the data source. Then step 350 needs to be performed.
Step 350, detecting whether the data message carries viruses.
And step 360, determining to directly delete the data message or store the data message to the local according to whether the data message carries viruses or not.
Specifically, when the data packet carries a virus, the data packet carrying the virus needs to be deleted directly, so as to prevent the virus from extending to other positions. And the problem that the server makes wrong instructions to influence users due to the fact that data messages carrying viruses are transmitted to the server is also avoided. If the data message can not be determined to carry the virus temporarily, the data message needs to be stored locally, so that the data message can be further analyzed by the staff conveniently. When the data message is ensured to be a new data message, the data message can be transmitted to the server and stored and backed up to the local. Or, when it is determined that the data packet carries a new virus, the data packet may be directly deleted.
The data filtering method provided by the embodiment of the invention transmits the data message through the intelligent terminal, filters the data message by using the forwarding node, intercepts the data message which possibly has danger, further detects whether the data message has viruses, deletes the data message if the viruses exist, and stores the data message to the local for further analysis if the viruses do not exist. And determining whether the data message is sent to the server or not until the data packet message is determined to be not dangerous. Or keep a copy to local as backup. When the data message is received again next time, the data packet message can be directly sent to the server after determining that no danger exists in the data packet message. And if the data message is not dangerous, directly transmitting the data message to the server.
In order to further enhance the security of data, ensure the safe and effective transmission of the data to a receiving end, avoid some lawbreakers from forging unsafe data carrying viruses and the like into a data transmitting end by an illegal means, send the data to a forwarding node, and transmit the data to a server by using the forwarding node. Therefore, after receiving the data packet sent by the sending end, some specific measures need to be taken. As follows. An embodiment of the present invention provides another data filtering method, specifically, as shown in fig. 4, the method includes:
step 410, receiving a data message sent by a sending end.
Step 420, the ID information of the sender and the interface information of the sender are obtained.
And step 430, matching the ID information and the interface information with the authority information in the interface index table stored locally.
Specifically, after receiving a data message sent by a sending end, before extracting at least one first type keyword in the data message, identity information and permission information of the sending end need to be obtained in advance.
For example, the ID information and the interface information of the sending end are obtained, and the ID information and the interface information of the sending end are matched with the authority information in the locally stored interface index table, so as to determine whether the sending end has the corresponding authority for sending the data message to the receiving end. When the matching is successful, the sending end can be determined to have the authority of sending the data message, and the data message is an effective data message. Otherwise, if the sending end does not have the authority to send the data message, the sending end is proved to be an 'abnormal sending end', the sending end is probably an illegal molecule to forge the sending end into a so-called 'sending end' by illegal means, and the sending end hopes to transmit wrong data, even virus data to the server through a data transmission channel. And if so, the data message is invalid, the data message is directly deleted, and the ID information and the interface information of the sending end are marked and added into a blacklist. And directly refusing to receive the data message sent by the sending end once the data message is subsequently received again.
Step 440, extracting at least one first type keyword in the data message.
Specifically, the data message may include a plurality of keywords, for example, the transmitted data message is data related to power consumption in a certain user's home, and includes, for example, the power consumption of the user's home in the last several months, the power consumption used in the current month, the remaining amount of power rate, and the like. Extracting at least one first-class keyword from the data message, and then matching the at least one first-class keyword with the keywords in the pre-configured keyword table, respectively, that is, step 450.
Step 450, at least one first category keyword is matched with keywords in a pre-configured keyword table respectively.
Step 460, if the first preset number of keywords in the at least one first type of keywords match the keywords in the pre-configured keyword table, sending the data message to the receiving end.
Specifically, if a first preset number of keywords in at least one first-class keyword match with keywords in a pre-configured keyword table, it is determined that the data packet is a normal data packet, and the normal data packet may be sent to the server through the forwarding node.
However, when none of the first predetermined number of keywords in the at least one first category of keywords matches a keyword in the pre-configured keyword table, it indicates that the data packet may have a problem itself or a problem with the data source. Then step 470 needs to be performed.
Step 470, detecting whether the data packet carries a virus.
Step 480, according to whether the data message carries viruses or not, determining to delete the data message directly or store the data message locally.
Specifically, when the data packet carries a virus, the data packet carrying the virus needs to be deleted directly, so as to prevent the virus from extending to other positions. And the problem that the server makes wrong instructions to influence users due to the fact that data messages carrying viruses are transmitted to the server is also avoided. If the data message can not be determined to carry the virus temporarily, the data message needs to be stored locally, so that the data message can be further analyzed by the staff conveniently. When the data message is ensured to be a new data message, the data message can be transmitted to the server and stored and backed up to the local. Or, when it is determined that the data packet carries a new virus, the data packet may be directly deleted.
In the data filtering method provided in the embodiment of the present invention, before the forwarding node extracts the keyword, the identity information and the authority of the sending end that sends the data packet are first determined, and a first round of filtering is performed. And when the sending end is determined to be legal and has the authority to send the data message, extracting keywords in the data message for matching, and performing a second round of filtering. And intercepting the data message which possibly has danger, further detecting whether the data message has viruses, deleting the data message if the viruses exist, and storing the data message to the local for further analysis if the viruses do not exist. And determining whether the data message is sent to the server or not until the data packet message is determined to be not dangerous. Or keep a copy to local as backup. When the data message is received again next time, the data packet message can be directly sent to the server after determining that no danger exists in the data packet message. And if the data message is not dangerous, directly transmitting the data message to the server.
In order to further ensure the high efficiency of data transmission, further processing is required before transmitting the data message. Specifically, as shown in fig. 5, fig. 5 is a schematic flow chart of another data filtering method according to an embodiment of the present invention, where the method includes:
step 510, receiving a data packet sent by a sending end.
Step 520, the ID information of the sending end and the interface information of the sending end are obtained.
Step 530, matching the ID information and the interface information with the authority information in the interface index table stored locally.
Specifically, after receiving a data message sent by a sending end, before extracting at least one first type keyword in the data message, identity information and permission information of the sending end need to be obtained in advance.
For example, the ID information and the interface information of the sending end are obtained, and the ID information and the interface information of the sending end are matched with the authority information in the locally stored interface index table, so as to determine whether the sending end has the corresponding authority for sending the data message to the receiving end. When the matching is successful, the sending end can be determined to have the authority of sending the data message, and the data message is an effective data message. Otherwise, if the sending end does not have the authority to send the data message, the sending end is proved to be an 'abnormal sending end', the sending end is probably an illegal molecule to forge the sending end into a so-called 'sending end' by illegal means, and the sending end hopes to transmit wrong data, even virus data to the server through a data transmission channel. And if so, the data message is invalid, the data message is directly deleted, and the ID information and the interface information of the sending end are marked and added into a blacklist. And directly refusing to receive the data message sent by the sending end once the data message is subsequently received again.
Step 540, at least one first type keyword in the data message is extracted.
Specifically, the data message may include a plurality of keywords, for example, the transmitted data message is data related to power consumption in a certain user's home, and includes, for example, the power consumption of the user's home in the last several months, the power consumption used in the current month, the remaining amount of power rate, and the like. Extracting at least one first-class keyword from the data packet, and then matching the at least one first-class keyword with the keywords in the pre-configured keyword table, respectively, that is, step 550.
At step 550, at least one first category keyword is matched with the keywords in the pre-configured keyword table.
Step 560, if the first preset number of keywords in the at least one first type of keywords match the keywords in the pre-configured keyword table, sending the data message to the receiving end.
Specifically, if a first preset number of keywords in at least one first-class keyword match with keywords in a pre-configured keyword table, it is determined that the data packet is a normal data packet, and the normal data packet may be sent to the server through the forwarding node.
However, when none of the first predetermined number of keywords in the at least one first category of keywords matches a keyword in the pre-configured keyword table, it indicates that the data packet may have a problem itself or a problem with the data source. Then step 570 needs to be performed.
Step 570, detecting whether the data packet carries a virus.
And 580, determining to directly delete the data message or store the data message to the local according to whether the data message carries the virus.
Specifically, when the data packet carries a virus, the data packet carrying the virus needs to be deleted directly, so as to prevent the virus from extending to other positions. And the problem that the server makes wrong instructions to influence users due to the fact that data messages carrying viruses are transmitted to the server is also avoided. If the data message can not be determined to carry the virus temporarily, the data message needs to be stored locally, so that the data message can be further analyzed by the staff conveniently. When the data message is ensured to be a new data message, the data message can be transmitted to the server and stored and backed up to the local. Or, when it is determined that the data packet carries a new virus, the data packet may be directly deleted.
Optionally, after step 510, the method may further include steps 585 and 590.
In step 585, when the same data packet sent by the same sending end is received, the same data packet is deleted.
Specifically, no matter whether the data message is a valid data message or an invalid data message, the system does not repeat transmission as long as the data message is identical. Once it is determined that the same data message sent by the same sending end is received, the data message can be directly deleted. One of the purposes is to reduce the data transmission amount as much as possible, reduce the load of a data transmission channel and improve the working efficiency. Yet another purpose is to avoid the occurrence of repeated transmission of data messages in order to avoid viruses on the transmitting side. And determining whether the data messages are the same or not, wherein the determination can be made by checking the bytes occupied by the data messages and whether the contents of the data messages are the same or not.
Step 590, counting the number of the same data packets sent by the same sending end within a preset time period, and sending a first warning message to the receiving end when the number of the same data packets sent by the same sending end is greater than or equal to a preset threshold.
Specifically, when the number of the same data packets sent by the same sending end is greater than or equal to the preset threshold value, the forwarding node may be a virus in the sending end or a virus carried in the data packets. Then, it is necessary to send the first warning message to the receiving end while deleting the data packet, so that the receiving end can remind the staff or the user to take corresponding effective measures.
In the data filtering method provided in the embodiment of the present invention, before the forwarding node extracts the keyword, the identity information and the authority of the sending end that sends the data packet are first determined, and a first round of filtering is performed. And when the sending end is determined to be legal and has the authority to send the data message, extracting keywords in the data message for matching, and performing a second round of filtering. And intercepting the data message which possibly has danger, further detecting whether the data message has viruses, deleting the data message if the viruses exist, and storing the data message to the local for further analysis if the viruses do not exist. And determining whether the data message is sent to the server or not until the data packet message is determined to be not dangerous. Or keep a copy to local as backup. When the data message is received again next time, the data packet message can be directly sent to the server after determining that no danger exists in the data packet message. And if the data message is not dangerous, directly transmitting the data message to the server.
In addition, if the data message is received for multiple times, it can also be determined that the sending end of the data message or the data message has a problem, and it can be directly said that the data message is deleted. Or when the counted receiving times are larger than or equal to the preset threshold value, the alarm information is directly sent to the receiving end.
In any of the above embodiments, the method for specifically detecting whether the data packet carries a virus may be implemented by the following steps, which are specifically shown in fig. 6:
step 610, extracting at least one second type keyword in the data packet.
Step 620, matching at least one second type keyword of the data message with keywords in a pre-stored virus library.
Step 630, when the second preset number of keywords in the at least one second type of keywords are successfully matched with the keywords in the pre-stored virus library, it is determined that the data packet carries viruses.
Step 640, when none of the second predetermined number of keywords in the at least one second type of keyword is successfully matched with the keywords in the pre-stored virus database, determining that the data packet is a damaged valid data packet or is a virus not stored in the virus database.
By the method, whether the data message is matched with the virus record existing in the current virus library or not can be clearly determined. Of course, if there is no match between the current data packet and the virus record stored in the current virus library, it cannot be directly stated that the data packet does not carry viruses. The virus library is actually the virus-related information acquired by the staff through different ways, and then is stored in the preset virus library as a record. If the virus library is not updated timely, new viruses are probably not detected. Therefore, it cannot be directly determined that the data packet is free of viruses. The data message needs to be stored locally and then further processed.
Or, the data message is likely to lose some bytes during transmission, resulting in incomplete data message. I.e. the actual corrupted valid data message. Then, the discrimination by the staff is also required. Namely, the data message needs to be stored locally, and the worker is informed to process the data message.
If the data message is directly determined to carry the virus, the data message needs to be directly deleted, and second alarm information is sent to the receiving end. So that the receiving end transmits the second alarm information to the staff or the user, and the related staff can effectively process the second alarm information.
In the method, at least one second-class keyword in the data message is extracted, the at least one second-class keyword in the data message is matched with keywords in a pre-stored virus library, and if the matching is successful, the data message is determined to carry viruses. Otherwise, the data message is determined to be a damaged effective data message. Or a virus not stored in the virus database. Both of these latter cases require further analysis by personnel. However, in either case, filtering of data packets can be achieved. And data messages possibly carrying virus danger are deleted or stored locally for further analysis, so that unnecessary troubles caused by the data messages to a server or a user are avoided.
Further optionally, in order to further reduce the number of bytes occupied by the data, the work efficiency of data transmission is improved. On the basis of any of the above embodiments, before sending the data packet to the receiving end, the method may further include a: and eliminating redundant fields in the data messages, and compressing the data messages with the redundant fields eliminated.
Specifically, the data packet may include a header, a valid field, a trailer field, and some other fields. Some of these fields may occupy a large number of bytes, but do not play any role, i.e. are redundant fields. Then, the redundant fields in the data packets can be removed, and the removed redundant fields are compressed. And finally, sending the compressed data message to a receiving end. Therefore, the resource occupancy rate is reduced, and the data transmission efficiency is improved.
Corresponding to the embodiment, the embodiment of the invention also provides a data filtering device. Specifically, as shown in fig. 7, fig. 7 is a schematic structural diagram of a data filtering apparatus according to an embodiment of the present invention. The device includes: a receiving unit 701, an extracting unit 702, a processing unit 703 and a transmitting unit 704. Wherein,
a receiving unit 701, configured to receive a data packet sent by a sending end;
an extracting unit 702 is configured to extract at least one first-class keyword in the data packet.
The processing unit 703 is configured to match at least one first-class keyword with keywords in a pre-configured keyword table, respectively.
A sending unit 704, configured to send the data packet to the receiving end when a first preset number of keywords in the at least one first type of keywords match keywords in a pre-configured keyword table.
The processing unit 703 is further configured to determine whether the data packet carries a virus when none of the first predetermined number of keywords in the at least one first type of keyword matches a keyword in a preconfigured keyword table; and determining to directly delete the data message or store the data message to the local according to whether the data message carries viruses or not.
Specifically, the processing unit 703 is configured to extract at least one second-type keyword from the data packet; matching at least one second-class keyword of the data message with keywords in a pre-stored virus library; when a second preset number of keywords in the at least one second type of keywords are successfully matched with keywords in a pre-stored virus library, determining that the data message carries viruses; otherwise, the data message is determined to be a damaged effective data message or a virus which is not stored in the virus database.
And when the data message is determined to carry the virus, directly deleting the data message and sending second alarm information to the receiving end. Or when the data message is determined to be a damaged effective data message or a virus stored in the virus database, the data message is stored locally so that a worker can analyze and process the data message.
In the data filtering device provided by the embodiment of the invention, the extraction unit extracts keywords in the data message, the processing unit filters the data message by matching the keywords in the data message with keywords in a pre-configured keyword table, intercepts the data message which may have danger, further detects whether the data message has viruses, deletes the data message if the viruses exist, and stores the data message to the local for further analysis if the viruses do not exist. And determining whether the data message is sent to the server or not until the data packet message is determined to be not dangerous. Or keep a copy to local as backup. When the data message is received again next time, the data packet message can be directly sent to the server after determining that no danger exists in the data packet message. And if the data message is not dangerous, directly transmitting the data message to the server.
Optionally, on the basis of the foregoing embodiment, the apparatus may further include: the verification unit 705 is configured to obtain ID information of a sending end and interface information of the sending end; matching the ID information and the interface information with authority information in an interface index table stored locally; when the matching is successful, determining that the sending end has the authority of sending the data message, and the data message is effective; otherwise, determining that the sending end does not have the authority of sending the data message, and deleting the data message if the data message is invalid.
Before the forwarding node extracts the keywords, firstly, the identity information and the authority of a sending end sending the data message are determined, and the first round of filtering is carried out. And when the sending end is determined to be legal and has the authority to send the data message, extracting keywords in the data message for matching, and performing a second round of filtering.
Optionally, on the basis of any of the above embodiments, the processing unit 703 is further configured to delete the same data packet when receiving the same data packet sent by the same sending end; counting the number of the same data messages sent by the same sending end received in a preset time period;
and when the number of the same data messages sent by the same sending end is greater than or equal to a preset threshold value, sending first alarm information to a receiving end.
Further optionally, on the basis of any of the above embodiments, the processing unit 703 is further configured to remove a redundant field in the data packet, and compress the data packet after the redundant field is removed.
The data packet may include a header, a valid field, an end field, and some other fields. Some fields actually occupy a plurality of bytes and do not play any role. Then, the processing unit 703 is used to remove the redundant fields in the data packets and compress the removed redundant fields. And finally, sending the compressed data message to a receiving end. Therefore, the resource occupancy rate is reduced, and the data transmission efficiency is improved.
Although specific functions among components in the data filtering device are not described in detail when describing the data filtering device provided in the embodiments of the present invention, it should be clear to those skilled in the art that the data filtering device corresponds to the data filtering method provided in the embodiments, and the functions performed by the data filtering device in the embodiments of the methods are described in detail and will not be described again here.
The reader should understand that in the description of this specification, reference to the description of the terms "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.
It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the several embodiments provided in the present application, it should be understood that the disclosed system and method may be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, a division of a unit is merely a logical division, and an actual implementation may have another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment of the present invention.
In addition, functional units in the embodiments of the present invention may be integrated into one processor, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product stored in a storage medium and including instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
While the invention has been described with reference to specific embodiments, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (10)
1. A method of filtering data, the method comprising:
receiving a data message sent by a sending end;
extracting at least one first-class keyword in the data message;
matching the at least one first-class keyword with keywords in a pre-configured keyword table respectively;
if the first preset number of keywords in the at least one first type of keywords are matched with the keywords in the pre-configured keyword table, sending the data message to a receiving end;
otherwise, detecting whether the data message carries viruses or not;
and determining to directly delete the data message or store the data message to the local according to whether the data message carries viruses or not.
2. The method of claim 1, wherein prior to extracting the at least one first-type keyword from the data packet, the method further comprises:
acquiring ID information of the sending end and interface information of the sending end;
matching the ID information and the interface information with authority information in an interface index table stored locally;
when the matching is successful, determining that the sending end has the authority of sending the data message, wherein the data message is effective;
otherwise, determining that the sending end does not have the authority of sending the data message, and deleting the data message if the data message is invalid.
3. The method according to claim 1, wherein after receiving the data packet sent by the sending end, the method further comprises:
when the same data message sent by the same sending end is received, deleting the same data message; counting the number of the same data messages sent by the same sending end received in a preset time period;
and when the number of the same data messages sent by the same sending end is greater than or equal to a preset threshold value, sending first alarm information to the receiving end.
4. The method according to any one of claims 1 to 3, wherein the detecting whether the data packet carries a virus specifically includes:
extracting at least one second type keyword in the data message;
matching at least one second type keyword of the keywords of the data message with keywords in a pre-stored virus library;
when a second preset number of keywords in at least one second type of keywords of the data message are successfully matched with the keywords in the pre-stored virus library, determining that the data message carries viruses;
otherwise, determining that the data message is a damaged effective data message or a virus not stored in the virus database.
5. The method according to claim 4, wherein the determining to directly delete the data packet or store the data packet locally according to whether the data packet carries a virus specifically comprises:
when the data message is determined to carry viruses, the data message is directly deleted, and second warning information is sent to a receiving end;
or,
and when the data message is determined to be a damaged effective data message or the virus stored in the virus database, storing the data message to the local so that the worker can analyze and process the data message conveniently.
6. A data filtering device, the device comprising:
a receiving unit, configured to receive a data packet sent by a sending end;
an extracting unit, configured to extract at least one first-class keyword in the data packet;
the processing unit is used for respectively matching the at least one first-class keyword with keywords in a pre-configured keyword table;
the sending unit is used for sending the data message to a receiving end when a first preset number of keywords in the at least one first type of keywords are matched with keywords in a pre-configured keyword table;
the processing unit is further configured to determine whether the data packet carries a virus or not when none of the first predetermined number of keywords in the at least one first category of keywords matches a keyword in a preconfigured keyword table;
and determining to directly delete the data message or store the data message to the local according to whether the data message carries viruses or not.
7. The apparatus of claim 6, further comprising:
the verification unit acquires the ID information of the sending end and the interface information of the sending end;
matching the ID information and the interface information with authority information in an interface index table stored locally;
when the matching is successful, determining that the sending end has the authority of sending the data message, wherein the data message is effective;
otherwise, determining that the sending end does not have the authority of sending the data message, and deleting the data message if the data message is invalid.
8. The apparatus of claim 6, wherein the processing unit is further configured to,
when the same data message sent by the same sending end is received, deleting the same data message; counting the number of the same data messages sent by the same sending end received in a preset time period;
and when the number of the same data messages sent by the same sending end is greater than or equal to a preset threshold value, sending first alarm information to the receiving end.
9. The apparatus according to any one of claims 6 to 8, wherein the processing unit is specifically configured to:
extracting at least one second type keyword in the data message;
matching at least one second-class keyword of the data message with keywords in a pre-stored virus library;
when the second preset number of keywords in the at least one second type of keywords are successfully matched with the keywords in the pre-stored virus library, determining that the data message carries viruses;
otherwise, determining that the data message is a damaged effective data message or a virus not stored in the virus database.
10. The apparatus of claim 9, wherein the processing unit is specifically configured to:
when the data message is determined to carry viruses, the data message is directly deleted, and second warning information is sent to a receiving end;
or,
and when the data message is determined to be a damaged effective data message or the virus stored in the virus database, storing the data message to the local so that the worker can analyze and process the data message conveniently.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710723706.4A CN107579960A (en) | 2017-08-22 | 2017-08-22 | A kind of data filtering method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710723706.4A CN107579960A (en) | 2017-08-22 | 2017-08-22 | A kind of data filtering method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107579960A true CN107579960A (en) | 2018-01-12 |
Family
ID=61033909
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710723706.4A Pending CN107579960A (en) | 2017-08-22 | 2017-08-22 | A kind of data filtering method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107579960A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112637013A (en) * | 2020-12-21 | 2021-04-09 | 苏州三六零智能安全科技有限公司 | CAN bus message abnormity detection method and device, equipment and storage medium |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101022343A (en) * | 2007-03-19 | 2007-08-22 | 杭州华为三康技术有限公司 | Network invading detecting/resisting system and method |
| CN101030859A (en) * | 2007-02-06 | 2007-09-05 | 上海交通大学 | Method and system for verifying distributed network |
| CN101079038A (en) * | 2006-06-28 | 2007-11-28 | 腾讯科技（深圳）有限公司 | System and method for implementing key word advertisement |
| CN101141390A (en) * | 2007-07-17 | 2008-03-12 | 武汉烽火网络有限责任公司 | Novel self-defining ethernet out-of-band data packet filtering method and device |
| CN101299719A (en) * | 2008-06-04 | 2008-11-05 | 北京星网锐捷网络技术有限公司 | Detection processing method for data flow, central processing unit and switch |
| CN101938565A (en) * | 2010-09-10 | 2011-01-05 | 中兴通讯股份有限公司 | Short message processing method and mobile terminal |
| CN102208992A (en) * | 2010-06-13 | 2011-10-05 | 天津海量信息技术有限公司 | Internet-facing filtration system of unhealthy information and method thereof |
| CN102255922A (en) * | 2011-08-24 | 2011-11-23 | 山东师范大学 | Intelligent multilevel junk email filtering method |
| CN102385554A (en) * | 2011-10-28 | 2012-03-21 | 华中科技大学 | Method for optimizing duplicated data deletion system |
| CN103179620A (en) * | 2009-01-19 | 2013-06-26 | 华为技术有限公司 | Implementation method of switching control, related equipment and communication system |
| CN103279542A (en) * | 2013-06-05 | 2013-09-04 | 中国电子科技集团公司第十五研究所 | Data importing processing method and data processing device |
| US20140223558A1 (en) * | 2007-12-13 | 2014-08-07 | International Business Machines Corporation | Method and device for integrating multiple threat security services |
| CN105376159A (en) * | 2014-08-25 | 2016-03-02 | 深圳市中兴微电子技术有限公司 | Packet processing and forwarding device and method |
| CN105511812A (en) * | 2015-12-10 | 2016-04-20 | 浪潮(北京)电子信息产业有限公司 | Method and device for optimizing big data of memory system |
-
2017
- 2017-08-22 CN CN201710723706.4A patent/CN107579960A/en active Pending
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101079038A (en) * | 2006-06-28 | 2007-11-28 | 腾讯科技（深圳）有限公司 | System and method for implementing key word advertisement |
| CN101030859A (en) * | 2007-02-06 | 2007-09-05 | 上海交通大学 | Method and system for verifying distributed network |
| CN101022343A (en) * | 2007-03-19 | 2007-08-22 | 杭州华为三康技术有限公司 | Network invading detecting/resisting system and method |
| CN101141390A (en) * | 2007-07-17 | 2008-03-12 | 武汉烽火网络有限责任公司 | Novel self-defining ethernet out-of-band data packet filtering method and device |
| US20140223558A1 (en) * | 2007-12-13 | 2014-08-07 | International Business Machines Corporation | Method and device for integrating multiple threat security services |
| CN101299719A (en) * | 2008-06-04 | 2008-11-05 | 北京星网锐捷网络技术有限公司 | Detection processing method for data flow, central processing unit and switch |
| CN103179620A (en) * | 2009-01-19 | 2013-06-26 | 华为技术有限公司 | Implementation method of switching control, related equipment and communication system |
| CN102208992A (en) * | 2010-06-13 | 2011-10-05 | 天津海量信息技术有限公司 | Internet-facing filtration system of unhealthy information and method thereof |
| CN101938565A (en) * | 2010-09-10 | 2011-01-05 | 中兴通讯股份有限公司 | Short message processing method and mobile terminal |
| CN102255922A (en) * | 2011-08-24 | 2011-11-23 | 山东师范大学 | Intelligent multilevel junk email filtering method |
| CN102385554A (en) * | 2011-10-28 | 2012-03-21 | 华中科技大学 | Method for optimizing duplicated data deletion system |
| CN103279542A (en) * | 2013-06-05 | 2013-09-04 | 中国电子科技集团公司第十五研究所 | Data importing processing method and data processing device |
| CN105376159A (en) * | 2014-08-25 | 2016-03-02 | 深圳市中兴微电子技术有限公司 | Packet processing and forwarding device and method |
| CN105511812A (en) * | 2015-12-10 | 2016-04-20 | 浪潮(北京)电子信息产业有限公司 | Method and device for optimizing big data of memory system |
Non-Patent Citations (1)
| Title |
|---|
| 莫礼平: "《多媒体技术与应用》", 30 August 2015 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112637013A (en) * | 2020-12-21 | 2021-04-09 | 苏州三六零智能安全科技有限公司 | CAN bus message abnormity detection method and device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109951500B (en) | Network attack detection method and device | |
| CN109829310B (en) | Similar attack defense method and device, system, storage medium, electronic device | |
| EP3598329B1 (en) | Information processing method, information processing system, and program | |
| CN106161451B (en) | Defend the method, apparatus and system of CC attack | |
| CN106713049B (en) | Monitoring alarm method and device | |
| EP2946332B1 (en) | Automated forensics of computer systems using behavioral intelligence | |
| CN110519150B (en) | Mail detection method, device, equipment, system and computer readable storage medium | |
| CN108521408A (en) | Resist method of network attack, device, computer equipment and storage medium | |
| CN112153062B (en) | Multi-dimension-based suspicious terminal equipment detection method and system | |
| CN107426202B (en) | Method for automatically testing WAF (Wireless Access Filter) interception rule | |
| CN110417717A (en) | The recognition methods of login behavior and device | |
| CN111092900A (en) | Method and device for monitoring abnormal connection and scanning behavior of server | |
| CN112887405B (en) | Intrusion prevention method, system and related equipment | |
| CN107645480B (en) | Data monitoring method, system and device | |
| CN106506630B (en) | Malicious network behavior discovery method based on HTTP content consistency | |
| CN112910918A (en) | Industrial control network DDoS attack traffic detection method and device based on random forest | |
| US9654491B2 (en) | Network filtering apparatus and filtering method | |
| CN105939314A (en) | Network protection method and device | |
| CN114285769A (en) | Shared internet access detection method, device, equipment and storage medium | |
| CN107579960A (en) | A kind of data filtering method and device | |
| WO2016037489A1 (en) | Method, device and system for monitoring rcs spam messages | |
| CN112769847B (en) | Safety protection method, device, equipment and storage medium for Internet of things equipment | |
| CN118018299A (en) | Network exception handling method and device based on flow analysis | |
| CN110048905B (en) | Internet of things equipment communication mode identification method and device | |
| CN113098852A (en) | Log processing method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180112 |
|
| RJ01 | Rejection of invention patent application after publication |