HK1183575A1 - Method for processing messages, device and system thereof - Google Patents

Abstract

The invention discloses a method, device and system for processing messages. The method includes: reading one or a plurality of messages from a data base and reading message type identifier of each message; obtaining a processing flow of the message corresponding to the message type identifier according to the message type identifiers; and generating one or a plurality of target messages by conducting de-weight processing and/or combination processing on the messages through the processing flow. By means of the method, device and system, invalid messages or redundancy messages interacted between systems can be greatly reduced. Due to the fact that the message type identifiers occupy less resource, system resource is saved, system processing efficiency is improved, namely system performance is further improved, and network bandwidth resource waste is reduced.

Description

Message processing method, device and system

Technical Field

The present application relates to the field of information interaction, and in particular, to a method, an apparatus, and a system for processing a message.

Background

At present, the situation of notifying messages as an interaction mode of an internet application system is increasing, and as the scale of internet applications is increasing, the number of generated messages is also increasing in a geometric basis, so that there is a strong demand for reducing the message amount by de-duplicating or merging the messages, thereby saving resources.

In the prior art, a method based on message deduplication mainly adopts comparison according to specific contents of messages and filters out messages with the same contents. For example, taking a piece of commercial advertisement information published on an e-commerce website as an example, the commercial advertisement information may change according to a received message, and the message carries a message content related to the commercial advertisement information, when a client user modifies a title of the commercial advertisement information twice within a time period, two pieces of message content carrying the title of the commercial advertisement information are generated, and at this time, the two pieces of message content may be "supply MP 3" and "supply MP 3" (i.e., the message contents of the two pieces of message are the same), and the prior art generally adopts a method of filtering messages with the same content to effectively filter messages with repeated message contents. There is no relevant technology at present based on a merging method between messages.

The core of the above prior art scheme is to perform deduplication determination based on message body content, and the method includes the following disadvantages:

firstly, because of the flexibility and diversity of the chinese expression, the duplication removal is realized by filtering the message content, which results in uncertainty of the filtering result, and the accurate duplication removal effect cannot be achieved, only one layer of filtering can be performed on part of redundant messages, and the efficiency and accuracy of the duplication removal on the mass data are relatively poor. For example, if the message content is based on the duplication removal, although the expressions of "supply MP 3" and "supply MP 3" are the same for the message content, the filtering result is obviously inaccurate by using the existing duplication removal technology only when the message content is supply MP3 "or" supply MP3 ", and the system processing efficiency is relatively low when the duplication removal processing of massive data is performed because the characters of the message content occupy relatively more resources.

In addition, in consideration of the diversity of the application scenarios, the message sent by each application scenario is also flexible and diverse, and therefore, the above message deduplication method in the same manner is difficult to satisfy all application scenarios simultaneously. Each type of application scene needs to be optimized in a targeted mode, so that the migration cost of the project is increased, and the usability of the project is reduced.

In the related art, an effective solution is not provided at present for the problems that filtering results obtained by removing duplicates of massive messages sent by a service terminal are poor in accuracy and low in efficiency, and massive messages cannot be combined.

Disclosure of Invention

The present application is provided for solving the problems that the filtering result obtained by removing the duplicate of the massive messages sent by the service terminal in the related art is poor in accuracy and the massive messages cannot be combined.

In order to achieve the above object, according to an aspect of the present application, there is provided a message processing method including: reading one or more messages and a message type identifier of each message from a database; acquiring a processing flow of the message corresponding to the message type identifier according to the message type identifier; and performing de-duplication processing and/or combination processing on the messages through the processing flow to generate one or more target messages.

Further, before reading the one or more messages and the message type identifier of each message from the database, the method further comprises: the service terminal generates one or more messages and generates a message type identifier of each message; and storing all the messages and the message type identifications corresponding to the messages in a database.

Further, the method for generating the message type identifier of each message comprises the following steps: setting a corresponding message type identifier for each message, and adding the message type identifier to the message; or one or more attribute values in the message are extracted, the obtained attribute values are extracted to obtain an extraction result, and the extraction result is used as a message type identifier corresponding to the message and is added to the message.

Further, performing deduplication processing and/or merging processing on the message through the processing flow to generate one or more target messages includes: under the condition that the message type identifications of one or more groups of messages in all the messages are detected to be the same, selecting the message with the latest message generation time in each group of messages as a target message of the group of messages; or judging whether a first message corresponding to any message type identifier is combined with messages corresponding to other message type identifiers, wherein under the condition of confirming the combination, reading a preset message needing to be combined with the first message, and combining the content of the first message with the content of the preset message to generate a target message; in the case where the acknowledgment does not merge, the first message is taken as the target message.

Further, performing deduplication processing and/or merging processing on the message through the processing flow to generate one or more target messages includes: under the condition that the type identifications of one or more groups of messages in all the messages are detected to be the same, only the message with the latest message generation time in each group of messages is stored to obtain a first message group; judging whether a first message corresponding to any message type identifier in a first message group is combined with messages corresponding to other message type identifiers or not, wherein under the condition of confirming combination, reading a preset message needing to be combined with the first message, and combining the content of the first message with the content of the preset message to generate a target message; in the case where the acknowledgment does not merge, the first message is taken as the target message.

Further, performing deduplication processing and/or merging processing on the message through the processing flow to generate one or more target messages includes: judging whether a first message corresponding to any message type identifier in all messages is combined with messages corresponding to other message type identifiers or not, wherein under the condition of confirming combination, reading a preset message needing to be combined with the first message, combining the content of the first message with the content of the preset message, and taking all combined messages obtained through combination as a second message group; taking all the first messages as a third group of messages under the condition of confirming that the messages are not combined; and under the condition that the type identifications of N messages in the second message group or the third message group are detected to be the same, only saving the message with the latest message generation time in the N messages as the target message.

In order to achieve the above object, according to another aspect of the present application, there is provided a message processing apparatus including: the reading module is used for reading one or more messages and the message type identification of each message from the database; the acquisition module is used for acquiring the processing flow of the message corresponding to the message type identifier according to the message type identifier; and the first processing module is used for carrying out de-duplication processing and/or combination processing on the messages through the processing flow so as to generate one or more target messages.

Further, the apparatus further comprises: the generating module is used for generating one or more messages by the service terminal and generating the message type identifier of each message; and the storage module is used for storing all the messages and the message type identifications corresponding to the messages to the database.

Further, the generation module includes: the setting module is used for setting a corresponding message type identifier for each message and adding the message type identifier to the message; or the second processing module is configured to extract one or more attribute values in the message, perform extraction processing on the obtained attribute values to obtain an extraction result, and add the extraction result to the message as a message type identifier corresponding to the message.

Further, the first processing module comprises: the duplication removal processing module is used for selecting the message with the latest message generation time in each group of messages as the target message of the group of messages under the condition that the message type identifications of one or more groups of messages in all the messages are detected to be the same; or, the merging processing module is configured to determine whether a first message corresponding to any one of the message type identifiers is merged with messages corresponding to other message type identifiers, where, in the case of confirming the merging, a predetermined message that needs to be merged with the first message is read, and the content of the first message is merged with the content of the predetermined message to generate a target message; in the case where the acknowledgment does not merge, the first message is taken as the target message.

In order to achieve the above object, according to another aspect of the present application, there is provided a message processing system including: the service terminal is used for generating one or more messages and the message type identifier of each message and storing all the messages and the message type identifiers corresponding to the messages to the database; and the notification message terminal is used for reading one or more messages and the message type identifier of each message from the database, acquiring the processing flow of the message corresponding to the message type identifier according to the message type identifier, and performing de-duplication processing and/or combination processing on the message through the processing flow to generate one or more target messages.

Further, the system further comprises: and the external terminal is used for receiving all the target messages subjected to format conversion.

In order to achieve the above object, according to another aspect of the present application, there is provided a message processing method including: the service terminal receives the modified new message sent by the client; the service terminal acquires a message type identifier corresponding to the new message; the service terminal stores the new message and the message type identifier corresponding to the new message into a database so that the notification message terminal reads the new message and the message type identifier of the message from the database; acquiring a processing flow of the message corresponding to the message type identifier according to the message type identifier; and performing de-duplication processing and/or combination processing on the messages through the processing flow to generate one or more target messages.

According to the method, one or more messages and the message type identifier of each message are read from a database; acquiring a processing flow of the message corresponding to the message type identifier according to the message type identifier; the embodiment reads the message type identifier added in the message, so that in the process of the deduplication processing and/or the merging processing, the preset processing flow is called according to the read message type identifier to obtain the corresponding target message, and because the content of the message type identifier is single and accurate, the deduplication processing and/or the merging processing based on the message type identifier does not change due to the flexibility and the diversity of the content of the message and the diversity of application scenes, obviously has higher accuracy, thereby solving the problems that the filtering result obtained by deduplication aiming at the massive messages sent by the service terminal in the prior art is poor in accuracy and the massive messages cannot be merged, and further, invalid messages and redundant messages interacted between the systems are greatly reduced, and because the message type identifiers occupy relatively few resources, system resources are saved, the processing efficiency of the system is improved, and the system performance is further improved, and the waste of network bandwidth resources is reduced.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:

FIG. 1 is a schematic diagram of a message processing system according to an embodiment of the present application;

FIG. 2 is a flow chart of a message processing method according to an embodiment of the application;

FIG. 3 is a detailed flow diagram of a message processing method according to the embodiment shown in FIG. 2;

FIG. 4 is a detailed flow diagram of another message processing method according to the embodiment shown in FIG. 2; and

fig. 5 is a schematic structural diagram of a message processing apparatus according to an embodiment of the present application.

Detailed Description

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

Fig. 1 is a schematic structural diagram of a message processing system according to an embodiment of the present application.

As shown in fig. 1, the message processing system includes: a service terminal 10, configured to generate one or more messages and a message type identifier of each message, and store all the messages and the message type identifiers corresponding to the messages in a database; and the notification message terminal 30 is configured to read one or more messages and a message type identifier of each message from the database, obtain a processing flow of the message corresponding to the message type identifier according to the message type identifier, and perform deduplication processing and/or merging processing on the message through the processing flow to generate one or more target messages.

According to the embodiment of the application, the message type identifier added in the message is read, so that in the process of duplicate removal processing and/or merging processing, the preset processing flow is called only according to the read message type identifier, and the corresponding target message can be obtained.

Specifically, in the process that more and more business entities organize information flow, fund flow and logistics through the internet at present, a notification function of an internet application system is implemented by using a message as an interaction mode, and as can be seen from the embodiment shown in fig. 1, first, a message type identifier is added to each message sent from a service terminal 10 in the internet application system, that is, a message type in a natural language recognizable by a terminal device is introduced to each message, and the message with the message type identifier added and the message type identifier thereof are stored in a database, for example, a message structure with the message type identifier added is described by taking a title of information issued on a user modification website as an example, and the message structure is shown in table 1 below:

table 1:

message type identification	Message content	Message generation time
			302	Supply MP3	2011-8-17 10:00:00
302	Supply MP3/MP4	2011-8-17 10:00:02

Then, the message processing apparatus in the notification message terminal 30 takes out the corresponding message and the message type identifier from the database, and obtains the corresponding processing flow (i.e., the rule for performing deduplication and/or merge processing) in the processing rule configuration apparatus according to the read message type identifier, then performs deduplication and/or merge processing on the message according to the message type identifier and the obtained processing rule, as shown in the example in table 1, when it is detected that the message type identifiers are consistent (both are 302), merges the two messages to obtain the target message, or uses one of the two messages with the latest message generation time as the target message, and prohibits sending the redundant other message, for example, only sends the message with the message generation time of 2011-8-1710: 00:02 as the target message.

The above-mentioned specific embodiment enables the notification message terminal 30 to determine the processing flow (or the processing rule) of the message according to the message type identifier in the message, thereby accurately and effectively performing deduplication and/or merging processing on the message, because the message type identifier has the characteristics of accuracy, single content, easy identification, and the like, the deduplication and/or merging processing based on the message type identifier is accurate and greatly reduces the interactive message volume between systems, so that the efficiency of reducing invalid messages and redundant messages is greatly improved, the system performance is further improved, and the waste of network bandwidth resources is reduced. In addition, the technical scheme of the application also completely gets rid of the restriction of projects on the manpower operation and maintenance cost and the application scenes, and can show accurate and efficient effects in all application scenes.

The system in the above embodiment of the present application may further include: and an external terminal 50 for receiving all the format-converted target messages. In order to enable the external terminal 50 to successfully recognize the target message obtained in the notification message terminal 30, the target message needs to be converted based on the format of the external terminal 50 and other requirements, so that the accuracy of message recognition is improved.

The product of the embodiment of the application can be applied to a plurality of interactive systems (e.g., Notify, synchronization system) of the backstage of the internet marketer.

In addition, it should be noted herein that the service terminal 10 referred to in the embodiments of the present application may refer to a terminal device applying various business application software, which is an initiator of a message notification; the notification client may refer to client software installed in the service terminal 10 for use by the business application. Here, the business application may register a message using a notification client. Once the message is successfully registered, the system can ensure that the message needs to reach the target receiver even if the message receiver is not on line or the network can not be connected temporarily; the database may be any general relational database that is responsible for storing messages to be sent or messages that have been sent unsuccessfully waiting for retransmission; the message refers to an information entity which can be identified by each system interaction; the notification message terminal 30 refers to an executor that notifies a specific consumer of a message; the external terminal refers to a certain commercial system, which is a specific message consumer and receiver; the message type identifier involved in the embodiment may specifically refer to a machine language that is expressed by the internal processing of the message in the notification message terminal 30. The following table examples:

message type identification	Description of functions
		300	Adding information type messages, e.g. release information for newly added goods
301	Deleting type messages, e.g. deleting merchandise release information
		302	Modifying an attribute of the information, e.g. title of the information issued from the article
303	Modifying an attribute of the information, e.g. the state of the information released by the article
		…………………….	………………………………….

FIG. 2 is a flow chart of a message processing method according to an embodiment of the application; FIG. 3 is a detailed flow diagram of a message processing method according to the embodiment shown in FIG. 2; and FIG. 4 is a detailed flow diagram of another message processing method according to the embodiment shown in FIG. 2.

As shown in fig. 2, the method comprises the following steps:

step S102, reading one or more messages and message type identifier of each message from the database is realized by the notification message terminal 30 in fig. 1.

Step S104, the notification message terminal 30 in fig. 1 executes a processing flow of acquiring a message corresponding to the message type identifier according to the message type identifier.

Step S106, executing, by the message processing device of the notification message terminal 30 in fig. 1, the message duplicate removal processing and/or the message merging processing through the processing flow to generate one or more target messages.

According to the embodiment of the application, the message type identifier added in the message is read, so that in the process of duplicate removal processing and/or merging processing, the preset processing flow is called only according to the read message type identifier, and the corresponding target message can be obtained.

In the above embodiments of the present application, before reading one or more messages and the message type identifier of each message from the database, the following steps may be further included: the service terminal 10 generates one or more messages and generates a message type identifier of each message; and storing all the messages and the message type identifications corresponding to the messages in a database. The embodiment confirms the message type identifier corresponding to each message and stores the message type identifier as a part of the message, and the independent generation and storage mode of the message type identifier enables the process of updating the message type identifier to be simple and easy to use, research personnel can accurately and efficiently determine the position of the message type identifier to be updated, and the flexibility and the usability of the system are improved. Meanwhile, after receiving a new message issued by the client due to the change of the commodity advertisement information, the service terminal in the embodiment directly acquires the message type identifier corresponding to the new message, and stores the combination of the acquired new message and the message type identifier thereof in the database, and the message can carry the message content related to the commodity advertisement information.

In the above embodiment of the present application, the method for generating the message type identifier of each message may include the following two implementation manners: the first is to set up the correspondent message type label to each message, and add the message type label to the message; or, the service terminal extracts the acquired attribute values by extracting one or more attribute values in the message to acquire an extraction result, and adds the extraction result to the message as a message type identifier corresponding to the message. The embodiment realizes two ways of generating the message type identifier, the first way is an automatic setting way which is simple and easy to implement, the second way is a way of automatically generating the message type identifier by using the existing content of the message, although the way is complex, the labor cost can be reduced, specifically, the message type identifier which needs to be added into the message can be determined by detecting the repetition degree of the message content or detecting the repetition range of the message generation time, the message content in the message is extracted as an example, when the current message content and the previous message content are detected to have two character repetitions (supply MP3 in two messages in table 1), the repeated content is extracted and inserted into the source message as the message type identifier as the basis of merging and deduplication processing.

In the above embodiments of the present application, in the implementation step of performing deduplication processing and/or merging processing on a message through a processing flow to generate one or more target messages, independent deduplication processing or independent merging processing may be performed respectively.

The independent duplicate removal processing may be performed by: and under the condition that the message type identifications of one or more groups of messages in all the messages are detected to be the same, selecting the message with the latest message generation time in each group of messages as the target message of the group of messages. The specific process of this embodiment may be described based on the embodiment shown in table 1, and after a plurality of messages are taken out from the database, the message processing apparatus in the message terminal 30 is notified to take out a corresponding deduplication processing flow from the processing rule configuration apparatus, where the deduplication processing flow is as follows, when it is detected that the message type identifiers of two messages are the same, it may be considered to send only one of the messages to the external terminal, and in order to improve the real-time performance and accuracy of sending the messages, a message with the latest message generation time of the two messages may be selected as a target message, for example, a message with the message generation time of 2011-8-1710: 00:02 is used as the target message, so that duplicate messages are removed, and system resources for processing the messages are reduced. For more complicated situations, if there are massive messages, the embodiment may divide the massive messages into message groups based on the same message type identifier, and since the message type identifiers of the messages in each group of messages are the same, each message group only sends one target message to the outside, so that the massive messages are compressed to the minimum number of messages for the system to process as much as possible.

The manner of independently completing the merging process may be: judging whether a first message corresponding to any message type identifier is combined with messages corresponding to other message type identifiers or not, wherein under the condition of confirming the combination, reading a preset message needing to be combined with the first message, and combining the content of the first message with the content of the preset message to generate a target message; in the case where the acknowledgment does not merge, the first message is taken as the target message. The specific merging process of this embodiment can be described based on the message structure shown in table 2. As can be seen from table 2, the message type identifier is a machine-recognizable natural language symbol, consistent with each message correspondence. The merge attribute value represents whether the message corresponding to the message type identifier needs to be merged and the merged preset message, and the "+" sign is used to indicate with which message or messages corresponding to which message type identifier the message corresponding to the current message type identifier needs to be merged.

Table 2:

message type identification	Merging attributes	Description of functions
			300	300	Newly added commodity release information
302	302+303	Modifying a title of merchandise release information
			303	302+303	Modifying the state of the commodity release information
…………………….	…………………….	…………………….

In this embodiment, after retrieving a plurality of messages from the database, the message processing apparatus in the notification message terminal 30 retrieves a corresponding merge processing flow from the processing rule configuration apparatus, where the merge processing flow is as follows: whether the message corresponding to the other message type identifier needs to be merged is determined according to the message type identifier (for example, the message with the message type identifier 302) of the current message, for example, it may be determined from table 2 that the message with the message type identifier 302 needs to be merged with the message type identifier 303, so that the message content with the message type identifier 302 and the message content with the message type identifier 303 are invoked to be merged to obtain a target message with merged message contents, which may be used to send some messages that are not necessarily sent singly in the actual use process although the contents are not completely consistent as a target message, thereby reducing redundant messages. If the message cannot be merged (e.g., a message with a message type identifier of 300), the message is sent directly as the target message.

In the foregoing embodiment, after the client modifies an attribute in the advertisement information, for example, a message content, at this time, after the service terminal 10 receives the new message content, the message type identifier corresponding to the message content is obtained based on the attribute definition of the message content, and after the message terminal 30 is notified to obtain the new message and the message type identifier thereof, the corresponding merged process flow is taken out from the processing rule configuration device for processing, and the specific flow is executed according to the steps described in the foregoing embodiment.

Similarly, in the case of the existence of the massive messages, the embodiment may preset, through the message type identifier, whether the current message needs to be merged, and which messages need to be merged, so that a plurality of combined and merged target messages may be obtained by merging a large number of messages according to the actual situation, and thus the massive messages are compressed to the minimum number of messages for the system to process. Since the number of message merges can be set on a case-by-case basis, the merge process allows for greater flexibility of the system.

Based on the foregoing embodiments, it can be seen that the present application can perform individual merging processing and individual deduplication processing on a massive message generated by the service terminal 10, and thus, the present application can also perform merging processing and deduplication processing for use, so that the massive message is further optimized, the system performance is further improved, and the waste of network bandwidth resources is reduced.

That is, the step of performing deduplication processing and/or merging processing on the messages through the processing flow to generate one or more target messages may also be an implementation step as follows: under the condition that the type identifications of one or more groups of messages in all the messages are detected to be the same, only the message with the latest message generation time in each group of messages is stored to obtain a first message group; judging whether a first message corresponding to any message type identifier in a first message group is combined with messages corresponding to other message type identifiers or not, wherein under the condition of confirming combination, reading a preset message needing to be combined with the first message, and combining the content of the first message with the content of the preset message to generate a target message; in the case where the acknowledgment does not merge, the first message is taken as the target message.

As shown in fig. 3, the above embodiment may include the following detailed steps:

step S201, a plurality of messages and message type identifiers of the messages are retrieved from the database.

In step S202, after the plurality of messages are retrieved from the database, the message processing apparatus in the message terminal 30 is notified to retrieve the processing flow corresponding to each message type identifier from the processing rule configuration apparatus. The processing flow obtained in this embodiment is a processing mode of performing deduplication before merging, and the following steps can be described with reference to the embodiments in table 1 and table 2.

Step S203, the message is processed for duplicate removal according to the message type identifier. The specific implementation process of this step may be that, in all the received messages, the messages with the same message type identifier are deduplicated (for example, the messages with the message type identifiers of 302 in table 1 are deduplicated), and one message with the latest message generation time is selected as the basis for the next merging processing. The deduplication processing performed in this step may also be considered as dividing all messages into message groups according to the same message type identifier, and taking only one message from each message group as a basis for the next merging processing.

Step S204, whether the message corresponding to other message type identifications needs to be merged or not is confirmed according to the message type identification of the current message, if yes, the step S205 is executed, otherwise, the step S206 is executed by taking the current message as a target message.

Step S205, obtaining the message corresponding to which message type identifier or message type identifiers the current message needs to be merged according to the message type identifier, for example, merging the message with the message type identifier 302 and the message with the message type identifier 303 in table 2, and similarly, obtaining the merging result of the messages corresponding to other message type identifiers, and finally obtaining the target message after two times of redundancy removal processing.

Step S206, the obtained target message is sent to an external terminal.

The embodiment of the present invention generally has an obvious effect in the case of processing a large amount of messages, and the embodiment may first divide a large amount of messages into message groups based on the same message type identifier, and since only one message is taken as a basis for the next combining after each message group is deduplicated, messages after deduplication of a large amount of messages may be regarded as a first message group, and then combine the first message group, that is, it is preset whether each message needs to be combined and which messages need to be combined by detecting the message type identifier of each message in the first message group, so that a plurality of combined target messages may be obtained by combining each message in the first message group according to the actual situation, and thus, the large amount of messages is compressed to the minimum number of messages for system processing.

For the same purpose as the embodiment shown in fig. 3, the step of performing deduplication processing and/or merging processing on the message through the processing flow to generate one or more target messages may also be implemented as another embodiment process: judging whether a first message corresponding to any message type identifier in all messages is combined with messages corresponding to other message type identifiers or not, wherein under the condition of confirming combination, reading a preset message needing to be combined with the first message, combining the content of the first message with the content of the preset message, and taking all combined messages obtained through combination as a second message group; taking all the first messages as a third group of messages under the condition of confirming that the messages are not combined; and under the condition that the type identifications of N messages in the second message group or the third message group are detected to be the same, only saving the message with the latest message generation time in the N messages as the target message.

Specifically, as shown in fig. 4, the above embodiment may include the following detailed steps:

step S301, a plurality of messages and message type identifications of the messages are taken out from the database.

Step S302, after taking out a plurality of messages from the database, notifies the message processing apparatus in the message terminal 30 to take out the processing flow corresponding to each message type identifier from the processing rule configuration apparatus. The processing flow obtained in this embodiment is a processing mode of performing combining first and then removing duplicate, and the following steps can be described with reference to the embodiments in table 1 and table 2.

Step S303, in all the received messages, it may be determined whether a message corresponding to another message type identifier needs to be merged according to the message type identifier of the current message, if so, step S304 is performed, otherwise, the current message is kept unchanged, which is used as a basis for the deduplication processing in step S305.

Step S304, obtaining, according to the message type identifier, which message or messages corresponding to which message type identifier the current message needs to be merged, for example, the message with the message type identifier 302 in table 2 needs to be merged with the message type identifier 303 to obtain a merged message, and similarly, obtaining the merged result of the messages corresponding to other message type identifiers to be used as the basis for the next deduplication processing.

Step S305, performing deduplication processing on all the messages obtained by merging in step S303 and the messages that are not merged in step S304, specifically performing deduplication processing according to the message type identifier.

The specific implementation procedure of this step may be to perform deduplication on messages with the same message type identifier among all the received messages (for example, perform deduplication on messages with message type identifiers of 302 in table 1), and select a message with the latest message generation time as the target message. The deduplication processing performed in this step may also be considered as screening all messages obtained after the merging processing according to the same condition of the message type identifier, dividing all messages into message groups, and obtaining only one message in each message group as the target message. And finally obtaining the target message subjected to redundancy removal processing twice.

And step S306, sending the obtained target message to an external terminal.

Similarly, in the case of processing a huge amount of messages, the method has the same and more obvious effect, and the embodiment can confirm whether each message needs to be merged according to the message type identifier of each message and with which messages to be merged, and after the merging processing of all messages is completed, can obtain a second message group after the message content is merged, and then divide the messages in the second message group into message groups based on the same message type identifier, that is, only one of the messages in the second message group with the same message type identifier is taken as a target message, so that several target messages are obtained by deduplicating each message in the second message group, and thus, the huge amount of messages is compressed to the minimum number of messages for system processing.

All target messages obtained in the above embodiments of the present application need to be processed in a format before being recognized by an external device, and therefore, the target messages obtained through processing by the message processing device need to be converted by the notification conversion device in the notification message terminal 30 and then sent to the external device.

The application can comprise a computer program for realizing the message processing method and a memory for storing the computer program, the computer program of the message processing method can be realized by adopting Java, and the computer program can run on a Linux system to achieve the best implementation effect

It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.

Fig. 5 is a schematic structural diagram of a message processing apparatus according to an embodiment of the present application.

As shown in fig. 5, the message processing apparatus includes: a reading module 20, configured to read one or more messages and a message type identifier of each message from a database; an obtaining module 40, configured to obtain, according to the message type identifier, a processing procedure of the message corresponding to the message type identifier; and a first processing module 60, configured to perform deduplication processing and/or merging processing on the messages through the processing flow to generate one or more target messages.

The above-mentioned embodiment of the present application reads the message type identifier already added to the message by the reading module 20, in the process of deduplication processing and/or merging processing, the obtaining module 40 only needs to call the preset processing flow according to the read message type identifier, so that the corresponding target message can be obtained through the first processing module 60, because the content of the message type identification is single and accurate, the content of the message cannot be changed due to the flexibility and variety of the content of the message and the variety of application scenes, thereby solving the problems that the filtering result obtained by removing the duplicate of the massive messages sent by the service terminal 10 in the prior art has poor accuracy and low efficiency and the massive messages can not be merged, and further, invalid messages and redundant messages interacted between the systems are greatly reduced, and the effects of further improving the system performance and reducing the waste of network bandwidth resources are further achieved.

The apparatus in the above embodiment of the present application may further include: a generating module 80, configured to generate one or more messages and generate a message type identifier of each message by the service terminal 10; and a storage module 100, configured to store all messages and the message type identifiers corresponding to the messages in a database. The embodiment confirms the message type identifier corresponding to each message and stores the message type identifier as a part of the message, and the independent generation and storage mode of the message type identifier enables the process of updating the message type identifier to be simple and easy to use, research personnel can accurately and efficiently determine the position of the message type identifier to be updated, and the flexibility and the usability of the system are improved.

The generating module 80 in the above embodiment of the present application may include: the setting module is used for setting a corresponding message type identifier for each message and adding the message type identifier to the message; or the second processing module is configured to extract one or more attribute values in the message, perform extraction processing on the obtained attribute values to obtain an extraction result, and add the extraction result to the message as a message type identifier corresponding to the message. The embodiment realizes two modes of generating the message type identifier, the first mode is an automatic setting mode, the mode is simple and easy to implement, the second mode is a mode of automatically generating the message type identifier by utilizing the existing content of the message, although the mode is complex, the labor cost can be reduced, and particularly, the message type identifier which needs to be added into the message can be determined by detecting the repetition degree of the message content or detecting the repetition range of the message generation time.

The first processing module 60 in the above embodiments of the present application may include: a deduplication processing module 601, configured to select, when it is detected that there are one or more groups of messages in all the messages and the message type identifiers of the messages are the same, a message with the latest message generation time in each group of messages as a target message of the group of messages; or, the merging processing module 602 is configured to determine whether a first message corresponding to any one message type identifier is merged with a message corresponding to another message type identifier, where in a case of confirming the merging, a predetermined message that needs to be merged with the first message is read, and content of the first message is merged with content of the predetermined message to generate a target message; in the case where the acknowledgment does not merge, the first message is taken as the target message. The merge processing module 601 and the deduplication processing module 602 in the above embodiment of the present application may independently complete the completion of massive messages, and greatly reduce the invalid messages and redundant messages exchanged between the systems, or may combine the two modules to process the messages, as shown in fig. 4, the function of the merge processing module 601 may be executed first, and then the function of the deduplication processing module 602 may be executed, but the starting sequence of the two modules in the processing process is not limited thereto, and may also be changed.

The above device embodiments of the present application classify messages by message type identifiers (machine-recognizable natural language), so as to achieve an effective message merging and deduplication method.

The application can also provide a message processing method, which comprises the following steps: the service terminal receives the modified new message sent by the client; the service terminal acquires a message type identifier corresponding to the new message; the service terminal stores the new message and the message type identifier corresponding to the new message into a database so that the notification message terminal reads the new message and the message type identifier of the message from the database; acquiring a processing flow of the message corresponding to the message type identifier according to the message type identifier; and performing de-duplication processing and/or combination processing on the messages through the processing flow to generate one or more target messages.

From the above description, it can be seen that the following technical effects are achieved by the present application: by introducing the message type (natural language which can be identified by a machine), invalid messages and redundant messages interacted between systems are greatly reduced. And the restriction of projects on the human operation and maintenance cost and application scenes is completely eliminated, the effect of reasonably removing redundant messages in all scenes can be realized, and the effects of further improving the system performance and reducing the waste of network bandwidth resources are also improved.

It will be apparent to those skilled in the art that the modules or steps of the present application described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and they may alternatively be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, or fabricated separately as individual integrated circuit modules, or fabricated as a single integrated circuit module from multiple modules or steps. Thus, the present application is not limited to any specific combination of hardware and software.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims (13)

1. A message processing method, comprising:

reading one or more messages and a message type identifier of each message from a database;

acquiring a processing flow of the message corresponding to the message type identifier according to the message type identifier; and

and performing de-duplication processing and/or combination processing on the message through the processing flow to generate one or more target messages.

2. The method of claim 1, wherein prior to reading the one or more messages and the message type identifier for each message from the database, the method further comprises:

the service terminal generates the one or more messages and generates a message type identifier of each message; and

and storing all the messages and the corresponding message type identifications to the database.

3. The method of claim 2, wherein generating the message type identifier for each message comprises:

setting a corresponding message type identifier for each message, and adding the message type identifier to the message;

alternatively, the first and second electrodes may be,

and extracting one or more attribute values in the message, extracting the obtained attribute values to obtain an extraction result, and adding the extraction result into the message as a message type identifier corresponding to the message.

4. The method according to any one of claims 1-3, wherein performing de-duplication processing and/or merging processing on the message through the processing flow to generate one or more target messages comprises:

under the condition that the message type identifications of one or more groups of messages in all the messages are detected to be the same, selecting the message with the latest message generation time in each group of messages as a target message of the group of messages; alternatively, the first and second electrodes may be,

judging whether the first message corresponding to any message type identification is merged with the messages corresponding to other message type identifications, wherein,

under the condition of confirming the combination, reading a preset message needing to be combined with the first message, and combining the content of the first message with the content of the preset message to generate the target message;

and in the case of confirming that the combination is not carried out, taking the first message as the target message.

5. The method according to any one of claims 1-3, wherein performing de-duplication processing and/or merging processing on the message through the processing flow to generate one or more target messages comprises:

under the condition that the type identifications of one or more groups of messages in all the messages are detected to be the same, only the message with the latest message generation time in each group of messages is stored to obtain a first message group;

judging whether the first message corresponding to any message type identifier in the first message group is merged with the messages corresponding to other message type identifiers or not, wherein,

under the condition of confirming the combination, reading a preset message needing to be combined with the first message, and combining the content of the first message with the content of the preset message to generate the target message;

and in the case of confirming that the combination is not carried out, taking the first message as the target message.

6. The method according to any one of claims 1-3, wherein performing de-duplication processing and/or merging processing on the message through the processing flow to generate one or more target messages comprises:

judging whether the first message corresponding to any message type identifier in all messages is combined with the messages corresponding to other message type identifiers, wherein,

under the condition of confirming the combination, reading a preset message which needs to be combined with the first message, combining the content of the first message with the content of the preset message, and taking all combined messages obtained through combination as a second message group;

taking all the first messages as a third group of messages under the condition of confirming that the messages are not combined;

and under the condition that the type identifications of N messages in the second message group or the third message group are detected to be the same, only saving the message with the latest message generation time in the N messages as the target message.

7. A message processing apparatus, comprising:

the reading module is used for reading one or more messages and the message type identification of each message from the database;

the acquisition module is used for acquiring the processing flow of the message corresponding to the message type identifier according to the message type identifier; and

and the first processing module is used for carrying out de-duplication processing and/or combination processing on the message through the processing flow so as to generate one or more target messages.

8. The apparatus of claim 7, further comprising:

a generating module, configured to generate the one or more messages and generate a message type identifier of each message by the service terminal; and

and the storage module is used for storing all the messages and the message type identifications corresponding to the messages to the database.

9. The apparatus of claim 8, wherein the generating module comprises:

the setting module is used for setting a corresponding message type identifier for each message and adding the message type identifier to the message; alternatively, the first and second electrodes may be,

and the second processing module is used for extracting one or more attribute values in the message, extracting the obtained attribute values to obtain an extraction result, and adding the extraction result into the message as a message type identifier corresponding to the message.

10. The apparatus of any one of claims 79, wherein the first processing module comprises:

the duplication removal processing module is used for selecting the message with the latest message generation time in each group of messages as the target message of the group of messages under the condition that the message type identifications of one or more groups of messages in all the messages are detected to be the same; alternatively, the first and second electrodes may be,

a merging processing module, configured to determine whether a first message corresponding to any one message type identifier is merged with messages corresponding to other message type identifiers, where, in the case of confirming the merging, a predetermined message that needs to be merged with the first message is read, and the content of the first message is merged with the content of the predetermined message to generate the target message; and in the case of confirming that the combination is not carried out, taking the first message as the target message.

11. A message processing system, comprising:

the service terminal is used for generating one or more messages and the message type identifier of each message and storing all the messages and the message type identifiers corresponding to the messages to the database; and

and the notification message terminal is used for reading one or more messages and the message type identifier of each message from the database, acquiring the processing flow of the message corresponding to the message type identifier according to the message type identifier, and performing de-duplication processing and/or merging processing on the message through the processing flow to generate one or more target messages.

12. The system of claim 12, further comprising:

and the external terminal is used for receiving all the target messages subjected to format conversion.

13. A message processing method, comprising:

the service terminal receives the modified new message sent by the client;

the service terminal acquires a message type identifier corresponding to the new message;

the service terminal stores the new message and the message type identifier corresponding to the new message into a database so that the notification message terminal reads the new message and the message type identifier of the message from the database;

acquiring a processing flow of the message corresponding to the message type identifier according to the message type identifier; and

and performing de-duplication processing and/or combination processing on the message through the processing flow to generate one or more target messages.