CN112948138B - A method and device for processing messages - Google Patents
A method and device for processing messages Download PDFInfo
- Publication number
- CN112948138B CN112948138B CN201911259680.8A CN201911259680A CN112948138B CN 112948138 B CN112948138 B CN 112948138B CN 201911259680 A CN201911259680 A CN 201911259680A CN 112948138 B CN112948138 B CN 112948138B
- Authority
- CN
- China
- Prior art keywords
- message
- processed
- idempotent
- tokens
- token
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000012545 processing Methods 0.000 title claims abstract description 65
- 238000000034 method Methods 0.000 title claims abstract description 40
- 238000004590 computer program Methods 0.000 claims description 9
- 230000003139 buffering effect Effects 0.000 claims description 7
- 239000002699 waste material Substances 0.000 abstract description 11
- 238000010586 diagram Methods 0.000 description 12
- 238000004891 communication Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 230000007246 mechanism Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000007123 defense Effects 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000002085 persistent effect Effects 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 231100000989 no adverse effect Toxicity 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computer And Data Communications (AREA)
Abstract
The invention discloses a method and a device for processing a message, and relates to the technical field of computers. The method comprises the steps of obtaining an idempotent token of a message to be processed, inquiring a network cache, a local cache and a message database by using the idempotent token to determine whether the message to be processed is processed, and processing the message to be processed and storing the message to be processed when the message to be processed is not processed. The embodiment can avoid repeated consumption of the message or excessive waste of the memory, realize real message idempotent, and trace the relation between the order number and the message.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a method and an apparatus for processing a message.
Background
In the existing message queues, the time-out, retransmission or confirmation mechanism adopted to ensure the accessibility of the messages may cause the message queue server or the service party to receive repeated messages, thereby affecting the service.
Message idempotency is the repetition of a message multiple times, the result of the consumer side consuming the repeated message multiple times is the same as one time, and multiple times of consumption have no adverse effect on the system. At present, a method for realizing message idempotent is to store a message in a Redis cache for a period of time, and when a certain message is consumed, if the message exists in the Redis cache, the message is represented to be repeated, and the message can be discarded.
In the process of implementing the present invention, the inventor finds that at least the following problems exist in the prior art:
if the Redis cache time is short, the repeated consumption of the message is easy to occur, if the Redis cache time is long, excessive waste of the memory is easy to occur, and the relation between the order number and the message cannot be traced.
Disclosure of Invention
In view of the above, the embodiment of the invention provides a method and a device for processing a message, which can avoid repeated consumption of the message or excessive waste of memory, realize real idempotent of the message and trace the relation between a task list and the message.
To achieve the above object, according to one aspect of an embodiment of the present invention, there is provided a method of processing a message.
The method for processing the message comprises the following steps:
acquiring an idempotent token of a message to be processed;
querying a network cache, a local cache and a message database by using the idempotent token to determine whether the message to be processed is processed;
and when the message to be processed is not processed, processing the message to be processed and storing the message to be processed.
Optionally, the idempotent token consists of a task identification of the message to be processed and a corresponding timestamp, and
Obtaining an idempotent token of a message to be processed, comprising:
Obtaining a message to be processed from a message queue;
and extracting an idempotent token of the message to be processed.
Optionally, querying a network cache, a local cache, and a message database with the idempotent token to determine whether the pending message is processed, including:
Querying the local cache for the idempotent tokens;
If the idempotent tokens exist in the local cache, the message to be processed is processed;
if the idempotent tokens do not exist in the local cache, setting the idempotent tokens as keys in a network cache;
If the setting fails, the message to be processed is processed;
if the setting is successful, the idempotent tokens are queried in a message database to determine whether the message to be processed is processed.
Optionally, querying the idempotent tokens in a message database to determine whether the pending message is processed, including:
judging whether the idempotent token is a primary key in the message database;
if yes, the message to be processed is processed;
if not, the message to be processed is not processed.
Optionally, processing the message to be processed and storing the message to be processed includes:
processing the message to be processed;
buffering the idempotent tokens in the local cache, and
And storing the message to be processed in the message database by taking the idempotent token as a primary key.
To achieve the above object, according to still another aspect of an embodiment of the present invention, there is provided an apparatus for processing a message.
The device for processing the message in the embodiment of the invention comprises the following components:
The acquisition module is used for acquiring an idempotent token of the message to be processed;
The query module is used for querying a network cache, a local cache and a message database by using the idempotent tokens so as to determine whether the message to be processed is processed or not;
and the processing module is used for processing the message to be processed and storing the message to be processed when the message to be processed is not processed.
Optionally, the idempotent token consists of a task identification of the message to be processed and a corresponding timestamp, and
The acquisition module is further configured to:
Obtaining a message to be processed from a message queue;
and extracting an idempotent token of the message to be processed.
Optionally, the query module is further configured to:
Querying the local cache for the idempotent tokens;
If the idempotent tokens exist in the local cache, the message to be processed is processed;
if the idempotent tokens do not exist in the local cache, setting the idempotent tokens as keys in a network cache;
If the setting fails, the message to be processed is processed;
if the setting is successful, the idempotent tokens are queried in a message database to determine whether the message to be processed is processed.
Optionally, the query module is further configured to:
judging whether the idempotent token is a primary key in the message database;
if yes, the message to be processed is processed;
if not, the message to be processed is not processed.
Optionally, the processing module is further configured to:
processing the message to be processed;
buffering the idempotent tokens in the local cache, and
And storing the message to be processed in the message database by taking the idempotent token as a primary key.
To achieve the above object, according to still another aspect of an embodiment of the present invention, there is provided an electronic device that processes a message.
An electronic device for processing a message according to an embodiment of the present invention includes one or more processors, and a storage device for storing one or more programs that, when executed by the one or more processors, cause the one or more processors to implement a method for processing a message according to an embodiment of the present invention.
To achieve the above object, according to still another aspect of the embodiments of the present invention, there is provided a computer-readable storage medium.
A computer readable storage medium of an embodiment of the present invention has stored thereon a computer program which, when executed by a processor, implements a method of processing a message of an embodiment of the present invention.
The embodiment of the invention has the advantages that the technical means that the message to be processed is processed and the message to be processed is stored when not processed are processed, so that the technical problems that the message is repeatedly consumed or the excessive waste of the memory is overcome, the technical problem that the relation between the task list and the message cannot be traced is solved, the problem that the message is repeatedly consumed or the excessive waste of the memory is avoided, the real message idempotent is realized, and the technical effect that the relation between the task list and the message can be traced is achieved.
Further effects of the above-described non-conventional alternatives are described below in connection with the embodiments.
Drawings
The drawings are included to provide a better understanding of the invention and are not to be construed as unduly limiting the invention. Wherein:
FIG. 1 is a schematic diagram of the main steps of a method of processing a message according to an embodiment of the invention;
FIG. 2 is a schematic diagram of an implementation flow of a method of processing a message according to one referenceable embodiment of the invention;
FIG. 3 is a schematic diagram of the main modules of an apparatus for processing messages according to an embodiment of the invention;
FIG. 4 is an exemplary system architecture diagram in which embodiments of the present invention may be applied;
Fig. 5 is a schematic diagram of a computer system suitable for use in implementing an embodiment of the invention.
Detailed Description
Exemplary embodiments of the present invention will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present invention are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the invention. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
It should be noted that the embodiments of the present invention and the technical features in the embodiments may be combined with each other without collision.
Fig. 1 is a schematic diagram of the main steps of a method of processing a message according to an embodiment of the invention.
As shown in fig. 1, the method for processing a message according to an embodiment of the present invention mainly includes the following steps:
and step S101, obtaining an idempotent token of the message to be processed.
Idempotent tokens are used to uniquely identify the same message to be processed and can be created when the message to be processed is generated. Due to the uniqueness of the idempotent tokens, it can be determined from the idempotent tokens whether the message to be processed has been processed.
In an embodiment of the invention, step S101 may be implemented by obtaining the message to be processed from the message queue and extracting an idempotent token of the message to be processed.
The idempotent tokens can be composed of task identifications and time stamps, the idempotent tokens can be obtained by processing the task identifications and the time stamps according to specific rules, and the task identifications and the time stamps can be directly spliced into the idempotent tokens in a preferred embodiment. The task identifier is used for identifying the message to be processed, an order number, a task name or the like can be used as the task identifier, and the timestamp is used for indicating the creation time of the message to be processed and determining the unique operation of the time point. An idempotent token consisting of a commonly used task identifier and a time stamp can uniquely represent a message to be processed, so that the message to be processed can be simply and conveniently inquired in a consumption process or a subsequent traceability process.
In the method for processing the message, the message to be processed comes from the message queue, and the idempotent token of the message to be processed is extracted before the message to be processed is processed. A message queue is a container that holds messages during their transmission, such as MQ (an application-to-application communication method), kafka (a high-throughput distributed publish-subscribe messaging system), and the like.
Step S102, query the network cache, the local cache and the message database by using the idempotent tokens to determine whether the message to be processed is processed.
The method for processing the message adopts a three-layer mechanism of combining double buffering with a message database, namely if an idempotent token of a certain message to be processed exists in one of the storage structures, the message to be processed is indicated to be processed, so that the unique constraint of the message to be processed is ensured. Although the network cache and the local cache have short cache time, most of repeated processing can be blocked, and the message database is used as the last defense line of the uniqueness constraint and stores all processed messages to be processed.
Specifically, the idempotent tokens may be queried in the network cache and the local cache first, if the idempotent tokens are not queried in the message database, if the idempotent tokens exist in the network cache, the local cache or the message database, the message to be processed is processed, and if the idempotent tokens do not exist in the network cache, the local cache and the message database, the message to be processed is not processed.
In the embodiment of the invention, the step S102 can be realized by inquiring an idempotent token in a local cache, if the idempotent token exists in the local cache, the message to be processed is processed, if the idempotent token does not exist in the local cache, the idempotent token is set as a key in a network cache, if the setting fails, the message to be processed is processed, and if the setting succeeds, the idempotent token is inquired in a message database to determine whether the message to be processed is processed.
The local buffer may be EhCache for temporarily storing idempotent tokens of the processed message, and if an idempotent token of a certain message exists in the local buffer, it indicates that the message has just been processed. EhCache is a pure Java in-process distributed cache framework, and has the characteristics of rapidness, exquisiteness and the like.
The network cache can be Redis, jimdb or a database based on Redis, and the like, and is used as a cache, and a lock mechanism is introduced (namely an idempotent token is set as a key), namely whether the message to be processed is processed or not can be determined by setting the idempotent token as the key through setnx command. Wherein, redis is an open source support network, can be based on a log type and Key Value pair (Key-Value) database which can also be persistent. Jimdb is a distributed, redis-based, memory-based, also persistent, log-type, key-Value database. setnx command is used to set certain string values in the Redis key (if that key does not exist in Redis).
In the embodiment of the invention, the idempotent tokens are queried in the message database to determine whether the message to be processed is processed or not, and the step of judging whether the idempotent tokens are primary keys in the message database or not can be realized by judging whether the idempotent tokens are primary keys in the message database, if yes, the message to be processed is processed, and if not, the message to be processed is not processed.
The message database can be a Key-Value database such as Hbase and the like, and is used as the last defense line of the uniqueness constraint, and the message database stores the message body of the message to be processed (namely the complete message to be processed) so as to obtain the processed message corresponding to the task list according to the task identifier. Wherein, in the message database, all processed messages to be processed are stored in the form of taking idempotent tokens as main keys and messages to be processed as values, so that if an idempotent token of a certain message to be processed is a main key in the message database, the message to be processed is processed. Hbase is a distributed, nematic, open source database.
And step S103, when the message to be processed is not processed, processing the message to be processed and storing the message to be processed.
When no idempotent tokens exist in the network cache, the local cache and the message database, the message to be processed is not processed, in this case, the message to be processed needs to be processed, and is stored after being processed, so that when the message to be processed is received again, the processed message can be determined to be processed without repeated processing.
In an embodiment of the present invention, step S103 may be implemented by referring to processing the message to be processed, buffering the idempotent tokens in a local cache, and storing the message to be processed in the message database with the idempotent tokens as primary keys.
The processed message to be processed can be stored in two ways simultaneously, namely, only idempotent tokens are stored in a local cache, the storage space occupied by the message is reduced under the condition that whether the requirement of repeated processing is queried when the new message to be processed is met, and the message to be processed is stored in a message database by taking the idempotent tokens as a main key, so that the relation between a follow-up task list and the message can be traced.
According to the method for processing the message, the technical problems that the message is repeatedly consumed or excessive waste of the memory and the relation between the task list and the message cannot be traced are solved, and therefore the real message idempotent can be realized by avoiding the repeated consumption or excessive waste of the memory of the message and tracing the relation between the task list and the message.
In order to further explain the technical idea of the invention, the technical scheme of the invention is described with specific application scenarios.
As shown in fig. 2, the method for processing a message according to the embodiment of the present invention may be implemented with reference to the following flow:
1. A Producer (Producer) creates a Message to be processed (Message) and sets MESSAGEID for the Message, which MESSAGEID consists of a service Id (i.e. task identity) and a timestamp at the time of creation of the Message;
2. Sending the Message to a Message queue MESSAGEID as an idempotent token while passing through the Message queue to the consumer (Constumer);
3. After receiving the Message by the consumer (Constumer) through the subscription mode, obtaining MESSAGEID of the Message;
4. Verifying whether MESSAGEID exists in EhCache, if so, indicating that the Message has been consumed, ending the flow, otherwise, continuing to step 5;
5. Based on the atomicity characteristic of the setnx command in Jimdb, MESSAGEID is used as a Key set value, if the setting fails, the Message is consumed, the flow is ended, and if the setting succeeds, the step 6 is continued;
6. MESSAGEID was used as the primary key (RowKey) of Hbase to verify if a Message was present. If the Message exists, the Message is consumed, and the flow is ended, otherwise, the step 7 is continued;
7. Consuming Message, and simultaneously, hbase stores the current Message and EhCache stores the current MESSAGEID.
Fig. 3 is a schematic diagram of main modules of an apparatus for processing a message according to an embodiment of the present invention.
As shown in fig. 3, the apparatus 300 for processing a message according to an embodiment of the present invention includes an acquisition module 301, a query module 302, and a processing module 303.
Wherein,
An acquisition module 301, configured to acquire an idempotent token of a message to be processed;
a query module 302 configured to query a network cache, a local cache, and a message database using the idempotent tokens to determine whether the message to be processed is processed;
and the processing module 303 is configured to process the message to be processed and store the message to be processed when the message to be processed is not processed.
Furthermore, the idempotent token consists of a task identification of the message to be processed and a corresponding timestamp.
In an embodiment of the present invention, the obtaining module 301 may further be configured to:
Obtaining a message to be processed from a message queue;
and extracting an idempotent token of the message to be processed.
In an embodiment of the present invention, the query module 302 may also be configured to:
Querying the local cache for the idempotent tokens;
If the idempotent tokens exist in the local cache, the message to be processed is processed;
if the idempotent tokens do not exist in the local cache, setting the idempotent tokens as keys in a network cache;
If the setting fails, the message to be processed is processed;
if the setting is successful, the idempotent tokens are queried in a message database to determine whether the message to be processed is processed.
In an embodiment of the present invention, the query module 302 may be further configured to:
judging whether the idempotent token is a primary key in the message database;
if yes, the message to be processed is processed;
if not, the message to be processed is not processed.
In an embodiment of the present invention, the processing module 303 may further be configured to:
processing the message to be processed;
buffering the idempotent tokens in the local cache, and
And storing the message to be processed in the message database by taking the idempotent token as a primary key.
The device for processing the message can be used for solving the technical problems that the message is repeatedly consumed or excessive waste of the memory and the relation between the task list and the message cannot be traced, so that the problem that the message is repeatedly consumed or excessive waste of the memory is avoided, the real message idempotent is realized, and the technical effect that the relation between the task list and the message can be traced is achieved.
Fig. 4 illustrates an exemplary system architecture 400 of a method of processing a message or an apparatus of processing a message to which embodiments of the present invention may be applied.
As shown in fig. 4, the system architecture 400 may include terminal devices 401, 402, 403, a network 404, and a server 405. The network 404 is used as a medium to provide communication links between the terminal devices 401, 402, 403 and the server 405. The network 404 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
A user may interact with the server 405 via the network 404 using the terminal devices 401, 402, 403 to receive or send messages or the like. Various communication client applications, such as shopping class applications, web browser applications, search class applications, instant messaging tools, mailbox clients, social platform software, etc., may be installed on the terminal devices 401, 402, 403.
The terminal devices 401, 402, 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smartphones, tablets, laptop and desktop computers, and the like.
The server 405 may be a server providing various services, such as a background management server providing support for shopping-type websites browsed by the user using the terminal devices 401, 402, 403. The background management server can analyze and other processing on the received data such as the product information inquiry request and the like, and feed back processing results (such as target push information and product information) to the terminal equipment.
It should be noted that, the method for processing a message provided in the embodiment of the present invention is generally performed by the server 405, and accordingly, the device for processing a message is generally disposed in the server 405.
It should be understood that the number of terminal devices, networks and servers in fig. 4 is merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
Referring now to FIG. 5, there is illustrated a schematic diagram of a computer system 500 suitable for use in implementing an embodiment of the present invention. The terminal device shown in fig. 5 is only an example, and should not impose any limitation on the functions and the scope of use of the embodiment of the present invention.
As shown in fig. 5, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, ROM 502, and RAM503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
Connected to the I/O interface 505 are an input section 506 including a keyboard, a mouse, and the like, an output section 507 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, a speaker, and the like, a storage section 508 including a hard disk, and the like, and a communication section 509 including a network interface card such as a LAN card, a modem, and the like. The communication section 509 performs communication processing via a network such as the internet. The drive 510 is also connected to the I/O interface 505 as needed. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as needed so that a computer program read therefrom is mounted into the storage section 508 as needed.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 509, and/or installed from the removable media 511. The above-described functions defined in the system of the present invention are performed when the computer program is executed by a Central Processing Unit (CPU) 501.
The computer readable medium shown in the present invention may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present invention, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The modules involved in the embodiments of the present invention may be implemented in software or in hardware. The described modules may also be provided in a processor, which may be described as, for example, a processor comprising an acquisition module, a query module, and a processing module. The names of these modules do not in any way constitute a limitation of the module itself, for example, the acquisition module may also be described as "module for acquiring idempotent tokens of the message to be processed".
As a further aspect, the invention also provides a computer readable medium which may be comprised in the device described in the above embodiments or may be present alone without being fitted into the device. The computer readable medium carries one or more programs which, when executed by a device, cause the device to include a step S101 of obtaining an idempotent token of a message to be processed, a step S102 of querying a network cache, a local cache, and a message database with the idempotent token to determine whether the message to be processed is processed, and a step S103 of processing the message to be processed and storing the message to be processed when the message to be processed is not processed.
According to the technical scheme of the embodiment of the invention, the idempotent tokens of the message to be processed are acquired, the network cache, the local cache and the message database are queried by utilizing the idempotent tokens to determine whether the message to be processed is processed, and when the message to be processed is not processed, the message to be processed is processed and the technical means of the message to be processed are stored, so that the technical problems that the message is repeatedly consumed or excessive waste of memory is overcome, and the relation between the task list and the message cannot be traced are solved, and further the technical effects that the message is prevented from being repeatedly consumed or excessive waste of memory, real message idempotent is realized, and the relation between the task list and the message can be traced are achieved.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives can occur depending upon design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A method of processing a message, comprising:
The method comprises the steps of obtaining an idempotent token of a message to be processed, wherein the idempotent token consists of a task identifier of the message to be processed and a corresponding time stamp;
Querying a network cache, a local cache and a message database by using the idempotent tokens to determine whether the message to be processed is processed, wherein the method comprises the steps of querying the idempotent tokens in the local cache, if the idempotent tokens exist in the local cache, the message to be processed is processed, ending the flow, if the idempotent tokens do not exist in the local cache, setting the idempotent tokens as keys in the network cache, if the setting fails, the message to be processed is processed, ending the flow, and if the setting is successful, querying the idempotent tokens in the message database to determine whether the message to be processed is processed;
and when the message to be processed is not processed, processing the message to be processed and storing the message to be processed.
2. The method of claim 1, wherein obtaining an idempotent token of the message to be processed comprises:
Obtaining a message to be processed from a message queue;
and extracting an idempotent token of the message to be processed.
3. The method of claim 1, wherein querying the idempotent tokens in a message database to determine whether the message to be processed is processed comprises:
judging whether the idempotent token is a primary key in the message database;
if yes, the message to be processed is processed;
if not, the message to be processed is not processed.
4. The method of claim 1, wherein processing the message to be processed and storing the message to be processed comprises:
processing the message to be processed;
buffering the idempotent tokens in the local cache, and
And storing the message to be processed in the message database by taking the idempotent token as a primary key.
5. An apparatus for processing a message, comprising:
The system comprises an acquisition module, a processing module and a processing module, wherein the acquisition module is used for acquiring an idempotent token of a message to be processed, and the idempotent token consists of a task identifier of the message to be processed and a corresponding time stamp;
The query module is used for querying a network cache, a local cache and a message database by utilizing the idempotent tokens to determine whether the message to be processed is processed or not, and comprises the steps of querying the idempotent tokens in the local cache, if the idempotent tokens exist in the local cache, the message to be processed is processed, ending the flow, if the idempotent tokens do not exist in the local cache, setting the idempotent tokens as keys in the network cache, if the setting fails, the message to be processed is processed, ending the flow, and if the setting is successful, querying the idempotent tokens in the message database to determine whether the message to be processed is processed or not;
and the processing module is used for processing the message to be processed and storing the message to be processed when the message to be processed is not processed.
6. The apparatus according to claim 5, wherein the sum
The acquisition module is further configured to:
Obtaining a message to be processed from a message queue;
and extracting an idempotent token of the message to be processed.
7. The apparatus of claim 5, wherein the query module is further to:
judging whether the idempotent token is a primary key in the message database;
if yes, the message to be processed is processed;
if not, the message to be processed is not processed.
8. The apparatus of claim 5, wherein the processing module is further configured to:
processing the message to be processed;
buffering the idempotent tokens in the local cache, and
And storing the message to be processed in the message database by taking the idempotent token as a primary key.
9. An electronic device for processing a message, comprising:
one or more processors;
storage means for storing one or more programs,
When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-4.
10. A computer readable medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911259680.8A CN112948138B (en) | 2019-12-10 | 2019-12-10 | A method and device for processing messages |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911259680.8A CN112948138B (en) | 2019-12-10 | 2019-12-10 | A method and device for processing messages |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112948138A CN112948138A (en) | 2021-06-11 |
| CN112948138B true CN112948138B (en) | 2025-03-18 |
Family
ID=76225586
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911259680.8A Active CN112948138B (en) | 2019-12-10 | 2019-12-10 | A method and device for processing messages |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112948138B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113641519A (en) * | 2021-08-17 | 2021-11-12 | 上海微盟企业发展有限公司 | Service idempotent calling method, device, equipment and storage medium |
| CN114116260A (en) * | 2021-11-29 | 2022-03-01 | 云知声智能科技股份有限公司 | A message processing method, device, terminal and storage medium |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107436767A (en) * | 2017-07-31 | 2017-12-05 | 杭州安恒信息技术有限公司 | The optimization method that idempotent operates in a kind of asynchronous framework |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8812711B2 (en) * | 2008-02-27 | 2014-08-19 | Red Hat, Inc. | Three-way communication protocol |
| US9185181B2 (en) * | 2011-03-25 | 2015-11-10 | International Business Machines Corporation | Shared cache for potentially repetitive message data in a publish-subscription environment |
| US20170230457A1 (en) * | 2016-02-05 | 2017-08-10 | Microsoft Technology Licensing, Llc | Idempotent Server Cluster |
| CN110266799B (en) * | 2019-06-21 | 2022-07-05 | 国网山西省电力公司忻州供电公司 | A method for implementing idempotency based on cache |
| CN110489437A (en) * | 2019-08-21 | 2019-11-22 | 中国工商银行股份有限公司 | Distributed transaction processing method, device, computer equipment and storage medium |
-
2019
- 2019-12-10 CN CN201911259680.8A patent/CN112948138B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107436767A (en) * | 2017-07-31 | 2017-12-05 | 杭州安恒信息技术有限公司 | The optimization method that idempotent operates in a kind of asynchronous framework |
Non-Patent Citations (1)
| Title |
|---|
| 基于微服务的消息中间件设计与实现;邵鹏;彭章友;;工业控制计算机;20190425(04);第112-115页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112948138A (en) | 2021-06-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107302597B (en) | Message file pushing method and device | |
| CN111038906B (en) | Order sorting method and device | |
| CN111460129B (en) | Method, device, electronic equipment and storage medium for generating identification | |
| CN111126948A (en) | Processing method and device for approval process | |
| CN110795315A (en) | Method and device for monitoring service | |
| CN116303608A (en) | Data processing method and device for application service | |
| CN112948138B (en) | A method and device for processing messages | |
| CN112948334A (en) | Log processing method and device | |
| CN110737655B (en) | Method and device for reporting data | |
| CN112860662A (en) | Data blood relationship establishing method and device, computer equipment and storage medium | |
| CN113722007B (en) | Configuration method, device and system of VPN branch equipment | |
| CN110109912B (en) | Identifier generation method and device | |
| CN109087097B (en) | Method and device for updating same identifier of chain code | |
| CN116450622B (en) | Method, apparatus, device and computer readable medium for data warehouse entry | |
| CN112579428B (en) | Interface testing method, device, electronic equipment and storage medium | |
| CN113704222A (en) | Method and device for processing service request | |
| CN113746661B (en) | A business processing method and device | |
| CN113726885B (en) | Flow quota adjusting method and device | |
| CN110858240A (en) | Front-end module loading method and device | |
| CN111460020B (en) | Method, device, electronic equipment and medium for resolving message | |
| CN112783924B (en) | Dirty data identification method, device and system | |
| CN114417318A (en) | Third-party page jumping method and device and electronic equipment | |
| CN109656519B (en) | Method and device for automatically accessing service data | |
| CN110909269B (en) | Log reporting method and device | |
| CN114091044A (en) | System authority management method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |