CN114356596A - Message processing method and device for distributed system - Google Patents
Message processing method and device for distributed system Download PDFInfo
- Publication number
- CN114356596A CN114356596A CN202111538371.1A CN202111538371A CN114356596A CN 114356596 A CN114356596 A CN 114356596A CN 202111538371 A CN202111538371 A CN 202111538371A CN 114356596 A CN114356596 A CN 114356596A
- Authority
- CN
- China
- Prior art keywords
- message
- event
- data
- service
- service modules
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title abstract description 8
- 238000012545 processing Methods 0.000 claims abstract description 61
- 238000000034 method Methods 0.000 claims abstract description 49
- 230000008569 process Effects 0.000 claims description 19
- 230000007246 mechanism Effects 0.000 claims description 11
- 230000004048 modification Effects 0.000 claims description 9
- 238000012986 modification Methods 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 6
- 238000004806 packaging method and process Methods 0.000 claims description 6
- 238000012544 monitoring process Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 description 6
- 230000000903 blocking effect Effects 0.000 description 3
- 238000013461 design Methods 0.000 description 2
- 239000003999 initiator Substances 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000005096 rolling process Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 1
- 230000014509 gene expression Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000003780 insertion Methods 0.000 description 1
- 230000037431 insertion Effects 0.000 description 1
- 230000009545 invasion Effects 0.000 description 1
- 238000002955 isolation Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a message processing method and a device of a distributed system, which is a distributed transaction solution, and the method comprises the following steps: determining a plurality of service modules of a target service in a distributed system, wherein each service module in the plurality of service modules is used for completing one or more service services in the target service; in order to solve the problem that one transaction spans a plurality of service modules and the call chain is short in a transaction scene, message middleware is configured for the plurality of service modules and used for processing event messages transmitted among the plurality of service modules, high concurrency, decoupling among the plurality of modules and high available message processing can be achieved by using the message middleware mq, distributed transactions and some fuzzy decoupling problems can be solved, and therefore final consistency of data is guaranteed.
Description
Technical Field
The invention relates to the field of internet, in particular to a message processing method and device for a distributed system.
Background
For distributed systems, to ensure data consistency, it may be implemented by distributed transactions. In a distributed system, a business operation may be performed by multiple services, different services are distributed at different nodes, and distributed transactions need to ensure that the services are either all successful or all failed.
Common scenarios involving distributed transactions include: cross-library transactions, where one service needs to operate different databases; the database and the table are divided, the data volume related in the service is large, and an independent database cannot be stored and needs to be horizontally split; invocation between services in microservice.
Seata and LCN are often used in the related art for implementation, Seata is an open source distributed transaction solution and aims to provide a high-performance and simple and easy-to-use distributed transaction service under a micro-service architecture, Seata can provide AT, TCC, SAGA and XA transaction modes for users, and as shown in FIG. 1, TM is a transaction initiator and defines the boundary of a transaction and is responsible for informing TC of the start, submission and rollback of the distributed transaction; the TC is a transaction coordinator and is responsible for generating a transaction D, registering, submitting, rolling back and the like; the RMs are resource managers, managing the resources of each branch transaction, and each RM registers as a branch transaction on the TC.
The distributed transaction lock flow is as follows: the local lock is acquired first, so that the local data can be modified, and only the local transaction can not be submitted; then, whether to submit is to see whether to obtain the global lock; if a global lock is acquired, meaning that it can be modified, then the local transaction is committed and the local lock is released; when a distributed transaction commits, the global lock is released, thus allowing other transactions to acquire the global lock and commit their modifications to the local data. Sea implements the read isolation principle: seata is done by proxying a SELECT FOR UPDATE statement, where execution of the SELECT FOR UPDATE statement applies FOR a global lock, and if the global lock is held by another transaction, the local lock is released (rolling back the local execution of the SELECT FOR UPDATE statement) and retried, where the query is blocked by the block and not returned until the global lock is taken, i.e., the read associated data is committed.
The LCN is a framework positioned in a transaction coordination mode, the framework does not produce transactions but is a coordinator of local transactions, and therefore the effect of transaction consistency is achieved, as shown in FIG. 2 (eureka: a registration center, a registration service module; TM, Tx-Manager, which is an independent service and is a controller of distributed transactions and coordinates submission and rollback of the distributed transactions; TC, Tx-Client, which is integrated by a business system, and a transaction initiator and a participant are both controlled by Tx-Client terminals; db, a database; redis: a database based on a memory), and the LCN is based on two-stage submission, as shown in FIG. 3.
In the related art, for example, when the payment requires the call of the payment service and the order service, the 2PC (i.e. the two-phase submission) is divided into two phases, the first phase is shown in fig. 4, and the second phase is shown in fig. 5. The 2PC has the disadvantage that TM has a single point of failure, occupies resources, and generates data inconsistency. Compared with 2PC, 3PC adds one stage Can (CanCommit) to 2PC (namely Three-stage commit, including Three stages CanCommit, PreCommit and DoCommit), and compared with 2PC, 3PC mainly solves the problem of single point failure and reduces blocking, because once a participant cannot timely receive information from a coordinator, commit is executed by default, and the participant cannot always hold transaction resources and is in a blocking state. This also brings new problems: this mechanism can cause data consistency problems because, due to network reasons, the abort response sent by the coordinator is not received by the participant in time, and the participant performs a commit operation after waiting for a timeout, which is a case of data inconsistency with other participants who receive the abort command and perform a rollback.
Compared with 2PC, 3PC has all set up the timeout time to Coordinator (Coordinator) and participant (participant), and 2PC only Coordinator has the timeout mechanism, this optimization point, mainly avoided the participant in the situation that can't communicate with Coordinator node for a long time (Coordinator hangs down), the problem that can't release the resource, because participant own possess the timeout mechanism can be after overtime, carry on local commit automatically and release the resource, and this kind of mechanism has also reduced the blocking time and scope of whole affairs laterally. Both Seata and LCN are optimized based on 2PC, and are strong in transaction consistency, moderate in complexity, small in invasion of a salesman, low in maintenance cost, but low in performance and not suitable for a particularly high-concurrency scene.
Disclosure of Invention
The invention mainly aims to provide a message processing method and a message processing device for a distributed system, so as to solve the technical problem that the technical scheme in the related technology is not suitable for a high-concurrency scene.
In order to achieve the above object, according to an aspect of the present invention, there is provided a message processing method for a distributed system, including: determining a plurality of service modules of a target business in a distributed system, wherein each service module in the plurality of service modules is used for completing one or more business services in the target business; and configuring message middleware for the plurality of service modules, wherein the message middleware is used for processing event messages transmitted among the plurality of service modules.
Optionally, after configuring message middleware for a plurality of service modules, the method further comprises: processing an event message sent between a plurality of service modules using message middleware; event messages received between the plurality of service modules are processed using message middleware.
Optionally, processing an event message sent between the plurality of service modules using message middleware, comprising: a sending end of the message middleware starts a timing task to continuously scan data with the type of 1 in an event table of the sending end according to the timing task, wherein the data is data sent from a first service module in the plurality of service modules to a second service module in the plurality of service modules, and the type of 1 indicates that the new service module is added; modifying the type of the data into 2 under the condition that the data with the type of 1 exists in the event table, wherein the type of 2 represents that the data is sent; packaging the data into an event message, and storing the event message into a message queue of message middleware; and processing the event message in the message queue by using the message middleware to send the event message to the second service module.
Optionally, in the process of processing an event message transmitted between the plurality of service modules by using the message middleware, the method further includes: when the event message in the message queue is processed by the message middleware, if the service is off-line, the data with the type 2 is rolled back to be restored to the state with the type 1.
Optionally, encapsulating the data into an event message, and storing the event message in a message queue of the message middleware, includes: and encapsulating the query data, the modification data and the sending data with the association into the same event message, and storing the same event message into a message queue of the message middleware.
Optionally, processing an event message received between the plurality of service modules using message middleware, comprising: monitoring an event message at a consumption end of message middleware; consuming the event message to store the data in the event message into an event table of a consuming end, and modifying the type of the data into 1, wherein the type of the data is 1 and indicates that the data is sent; and sending a notification of successful receiving to a sending end of the message middleware.
Optionally, in the process of consuming the event message and storing the data in the event message into the event table of the consuming side, the method further includes: and when the event message is repeated for N times and the consumption is not completed, storing the event message into a dead credit queue so as to run a compensation mechanism according to the dead credit queue.
In order to achieve the above object, according to an aspect of the present invention, there is provided a message processing apparatus of a distributed system, including: the system comprises a determining unit, a service processing unit and a service processing unit, wherein the determining unit is used for determining a plurality of service modules of a target service in a distributed system, and each service module in the plurality of service modules is used for completing one or more service services in the target service; the configuration unit is used for configuring message middleware for the plurality of service modules, wherein the message middleware is used for processing event messages transmitted among the plurality of service modules.
Optionally, the apparatus of the present application may further comprise: a message processing unit for processing an event message transmitted between the plurality of service modules using message middleware after configuring the message middleware for the plurality of service modules; event messages received between the plurality of service modules are processed using message middleware.
Optionally, the message processing unit is further configured to, in a process of processing an event message sent between the multiple service modules by using a message middleware, start a timing task by a sending end of the message middleware, so as to continuously scan data of a type 1 in an event table of the sending end according to the timing task, where the data is data sent from a first service module of the multiple service modules to a second service module of the multiple service modules, and the type 1 indicates a new addition; modifying the type of the data into 2 under the condition that the data with the type of 1 exists in the event table, wherein the type of 2 represents that the data is sent; packaging the data into an event message, and storing the event message into a message queue of message middleware; and processing the event message in the message queue by using the message middleware to send the event message to the second service module.
Optionally, the message processing unit is further configured to, in the process of processing an event message transmitted between the plurality of service modules by using the message middleware, when the event message in the message queue is processed by using the message middleware, if a problem of service offline occurs, roll back the data of type 2, and restore the state of type 1.
Optionally, the message processing unit is further configured to, in the process of encapsulating the data into an event message and storing the event message into a message queue of the message middleware, encapsulate the query data, the modification data, and the sending data, which have an association, into the same event message and store the same into the message queue of the message middleware.
Optionally, the message processing unit is further configured to, during processing of an event message received among the plurality of service modules by using message middleware, monitor the event message at a consuming side of the message middleware; consuming the event message to store the data in the event message into an event table of a consuming end, and modifying the type of the data into 1, wherein the type of the data is 1 and indicates that the data is sent; and sending a notification of successful receiving to a sending end of the message middleware.
Optionally, the message processing unit is further configured to store the event message into the deadlock queue when the event message is repeated N times and consumption is not completed in a process of consuming the event message and storing data in the event message into an event table of the consuming end, so as to run the compensation mechanism according to the deadlock queue.
According to another aspect of the embodiments of the present application, there is also provided an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the above method through the computer program.
According to an aspect of the application, a computer program product or computer program is provided, comprising computer instructions, the computer instructions being stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the steps of any of the embodiments of the method described above.
The distributed transaction solution of the invention is applied to determine a plurality of service modules of the target service in the distributed system, wherein each service module in the plurality of service modules is used for completing one or more service services in the target service; in order to solve the problem that one transaction spans a plurality of service modules and the call chain is short in a transaction scene, message middleware is configured for the plurality of service modules and used for processing event messages transmitted among the plurality of service modules, high concurrency, decoupling among the plurality of modules and high available message processing can be achieved by using the message middleware mq, distributed transactions and some fuzzy decoupling problems can be solved, and therefore final consistency of data is guaranteed.
In addition to the objects, features and advantages described above, other objects, features and advantages of the present invention are also provided. The present invention will be described in further detail below with reference to the drawings.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the invention and together with the description serve to explain the invention and not to limit the invention. In the drawings:
FIG. 1 shows a schematic diagram of a distributed transaction scheme in accordance with the present invention;
FIG. 2 shows a schematic diagram of a distributed transaction scheme in accordance with the present invention;
FIG. 3 shows a schematic diagram of a distributed transaction scheme in accordance with the present invention;
FIG. 4 shows a schematic diagram of a distributed transaction scheme in accordance with the present invention;
FIG. 5 shows a schematic diagram of a distributed transaction scheme in accordance with the present invention;
FIG. 6 is a flow chart illustrating a message processing method of a distributed system according to the present invention; and the number of the first and second groups,
fig. 7 shows a schematic diagram of a distributed transaction scheme according to the present invention.
Detailed Description
It should be noted that the embodiments and features of the embodiments may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.
In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the terms so used are interchangeable under appropriate circumstances for describing embodiments of the invention herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments according to the present application. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, and it should be understood that when the terms "comprises" and/or "comprising" are used in this specification, they specify the presence of stated features, steps, operations, devices, components, and/or combinations thereof, unless the context clearly indicates otherwise.
According to an aspect of the embodiments of the present application, a method embodiment of a message processing method for a distributed system is provided, as shown in fig. 6, including the following steps:
step S602, determining a plurality of service modules of the target service in the distributed system, where each of the plurality of service modules is used to complete one or more service services in the target service.
Step S604, configuring message middleware for the plurality of service modules, where the message middleware is configured to process event messages transmitted between the plurality of service modules.
Step S606, the event message sent between the plurality of service modules is processed by using the message middleware, or the event message received between the plurality of service modules is processed by using the message middleware.
Optionally, processing an event message sent between the plurality of service modules using message middleware, comprising: a sending end of the message middleware starts a timing task to continuously scan data with the type of 1 in an event table of the sending end according to the timing task, wherein the data is data sent from a first service module in the plurality of service modules to a second service module in the plurality of service modules, and the type of 1 indicates that the new service module is added; modifying the type of the data into 2 under the condition that the data with the type of 1 exists in the event table, wherein the type of 2 represents that the data is sent; packaging the data into an event message, and storing the event message into a message queue of message middleware; and processing the event message in the message queue by using the message middleware to send the event message to the second service module.
Optionally, in the process of processing an event message transmitted between the plurality of service modules by using the message middleware, the method further includes: when the event message in the message queue is processed by the message middleware, if the service is off-line, the data with the type 2 is rolled back to be restored to the state with the type 1.
Optionally, encapsulating the data into an event message, and storing the event message in a message queue of the message middleware, includes: and encapsulating the query data, the modification data and the sending data with the association into the same event message, and storing the same event message into a message queue of the message middleware.
Optionally, processing an event message received between the plurality of service modules using message middleware, comprising: monitoring an event message at a consumption end of message middleware; consuming the event message to store the data in the event message into an event table of a consuming end, and modifying the type of the data into 1, wherein the type of the data is 1 and indicates that the data is sent; and sending a notification of successful receiving to a sending end of the message middleware.
Optionally, in the process of consuming the event message and storing the data in the event message into the event table of the consuming side, the method further includes: and when the event message is repeated for N times and the consumption is not completed, storing the event message into a dead credit queue so as to run a compensation mechanism according to the dead credit queue.
The distributed transaction solution is adopted to determine a plurality of service modules of the target service in the distributed system, wherein each service module in the plurality of service modules is used for completing one or more service services in the target service; in order to solve the problem that one transaction spans a plurality of service modules and the call chain is short in a transaction scene, message middleware is configured for the plurality of service modules and used for processing event messages transmitted among the plurality of service modules, and by using the message middleware mq, high concurrency, decoupling among the plurality of modules and high available message processing can be realized, and the technical problem that the technical scheme in the related technology is not suitable for the high concurrency scene can be solved. The technical solution of the present application is further detailed below with reference to specific embodiments:
the application provides a high-performance solution of distributed transactions in a high concurrency environment, a plurality of modules are called, a plurality of services are decoupled by using mq as a middleware, and the solution is realized by taking a payment system as an example and by using a scheme of a local event table + a timing task + mq.
As shown in fig. 7, the distributed transaction is split into independent local transactions, and the design scheme of the sending-end event table (only important fields are designed, and the rest are self-extended according to the service) is shown in table 1:
TABLE 1
| Name of field | Type of field | Note |
| id | varcha | Business related order id |
| type | int | Event type, 1 indicates newly added, 2 indicates sent |
The design scheme of the consumer-side event table is shown in table 2:
TABLE 2
| Name of field | Type of field | Note |
| id | varcha | Business related order id |
| type | int | Event type, 1 indicates received, 2 indicates processed |
(1) Transaction 1 (sender of message):
and establishing a payment flow meter, updating the payment flow record in the payment flow meter, and asynchronously and directly sending the send to a message queue mq of the message middleware for the newly established event.
Step 1, inserting the event into an event table (including the newly created payment success event).
And 2-1, starting a timing task, and continuously scanning data with the type 1 in the event (inquiring a new event).
And step 2-2, changing the type of the data into 2, and sending the data.
And 2-3, sending the event message to a message queue and sending the event message in mq.
At this time, the query data, the modification data and the sending data are added into one transaction, if the problem of service offline occurs during sending, the data can be rolled back at this time, and the previous new state with the type being 1 is recovered, so that the accuracy of the data is ensured.
The order of step 2-2 and step 2-3 cannot be interchanged, if step 2-3 is earlier and there is a problem with the service after the message is sent to the message queue, at this point a rollback is performed, although it can be guaranteed that the data in the event table is restored to the previous state, but dirty data already exists in mq.
(2) The consuming side of the message:
step 3-1, monitoring messages and consuming messages.
And step 3-2, inserting the message content into an event table, wherein the type is 1 and indicates that the message content is received.
Step 3-3, reply to mq with an ack acknowledgment.
The primary key id and the unique key of the event table are used for ensuring that the message cannot be repeatedly consumed, if the service is unavailable in the step 3, the message can be repeatedly delivered, the repeated message is inserted into the event table at the moment, the insertion fails at the moment due to the constraint of the primary key and the unique key, when the message is repeated for 6 times and is not consumed, the message enters the m-q letter queue, and a compensation mechanism (processing data in the letter queue according to the service condition) can be carried out at the moment.
If the service is not available in step 2, the message is not deleted from the mq, and the mq has a retry mechanism, which can ensure that the message can be consumed certainly.
(3) Affairs 2 (modifying order state)
Step 4-1, the received event with type 1 in the consumption event table is queried.
And 4-2, running a timing task, executing the local service, changing the type of the data into 1 to 2, and processing the data.
And 4-3, modifying the order state into paid state and modifying the event table state into processed state.
The steps are put into a transaction to ensure that the state of the order received by the consumer can be successfully modified into the paid order.
By adopting the technical scheme, the decoupling is carried out on the multi-service calling by utilizing the mq, the performance of the service is improved, after the service is split, different members are responsible for different modules, the coupled maintenance call is reduced, the code intrusiveness is reduced, and the distributed complexity reduction is realized.
It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present application is not limited by the order of acts described, as some steps may occur in other orders or concurrently depending on the application. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required in this application.
Through the above description of the embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better implementation mode in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (e.g., a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present application.
According to another aspect of the embodiment of the application, an apparatus for implementing the method is also provided. The apparatus may include: the system comprises a determining unit, a service processing unit and a service processing unit, wherein the determining unit is used for determining a plurality of service modules of a target service in a distributed system, and each service module in the plurality of service modules is used for completing one or more service services in the target service; a configuration unit, configured to configure message middleware for the plurality of service modules, where the message middleware is configured to process event messages transmitted between the plurality of service modules.
Optionally, the apparatus of the present application may further comprise: a message processing unit for processing an event message transmitted between the plurality of service modules using message middleware after configuring the message middleware for the plurality of service modules; processing, with the message middleware, event messages received between the plurality of service modules.
Optionally, the message processing unit is further configured to, in the process of processing the event message sent between the multiple service modules by using the message middleware, start a timing task by a sending end of the message middleware, so as to scan data of a type 1 in an event table of the sending end uninterruptedly according to the timing task, where the data is data sent by a first service module of the multiple service modules to a second service module of the multiple service modules, and the type 1 indicates a new addition; modifying the type of the data into 2 under the condition that the data with the type of 1 exists in the event table, wherein the type of 2 represents that the data is sent; packaging the data into an event message, and storing the event message into a message queue of the message middleware; and processing the event message in the message queue by utilizing the message middleware so as to send the event message to the second service module.
Optionally, the message processing unit is further configured to, in a process of processing an event message transmitted between the plurality of service modules by using the message middleware, when the event message in the message queue is processed by using the message middleware, if a problem of service offline occurs, roll back the data of type 2, and restore the data to the state of type 1.
Optionally, the message processing unit is further configured to, in the process of encapsulating the data into an event message and storing the event message into the message queue of the message middleware, encapsulate query data, modification data, and sending data having an association into the same event message and store the same into the message queue of the message middleware.
Optionally, the message processing unit is further configured to, during the process of processing, by using the message middleware, the event message received among the plurality of service modules, monitor the event message at a consuming side of the message middleware; consuming the event message to store the data in the event message into an event table of a consuming end, and modifying the type of the data into 1, wherein the type of the data is 1 and indicates that the data is sent; and sending a notification of successful receiving to a sending end of the message middleware.
Optionally, the message processing unit is further configured to store the event message into the deadlock queue when the event message is repeated N times and consumption is not completed in a process of consuming the event message and storing data in the event message into an event table of the consuming end, so as to run the compensation mechanism according to the deadlock queue.
The method comprises the steps that a plurality of service modules of target business in a distributed system are determined by applying a distributed transaction solution of the application, and each service module in the plurality of service modules is used for completing one or more business services in the target business; in order to solve the problem that one transaction spans a plurality of service modules and the call chain is short in a transaction scene, message middleware is configured for the plurality of service modules and used for processing event messages transmitted among the plurality of service modules, and by using the message middleware mq, high concurrency, decoupling among the plurality of modules and high available message processing can be realized, and the technical problem that the technical scheme in the related technology is not suitable for the high concurrency scene can be solved.
The relative arrangement of the components and steps, the numerical expressions and numerical values set forth in these embodiments do not limit the scope of the present invention unless specifically stated otherwise. Meanwhile, it should be understood that the sizes of the respective portions shown in the drawings are not drawn in an actual proportional relationship for the convenience of description. Techniques, methods, and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail but are intended to be part of the specification where appropriate. In all examples shown and discussed herein, any particular value should be construed as merely illustrative, and not limiting. Thus, other examples of the exemplary embodiments may have different values. It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, further discussion thereof is not required in subsequent figures.
Spatially relative terms, such as "above … …," "above … …," "above … …," "above," and the like, may be used herein for ease of description to describe one device or feature's spatial relationship to another device or feature as illustrated in the figures. It will be understood that the spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. For example, if a device in the figures is turned over, devices described as "above" or "on" other devices or configurations would then be oriented "below" or "under" the other devices or configurations. Thus, the exemplary term "above … …" can include both an orientation of "above … …" and "below … …". The device may be otherwise variously oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein interpreted accordingly.
In the description of the present invention, it is to be understood that the orientation or positional relationship indicated by the orientation words such as "front, rear, upper, lower, left, right", "lateral, vertical, horizontal" and "top, bottom", etc. are usually based on the orientation or positional relationship shown in the drawings, and are only for convenience of description and simplicity of description, and in the case of not making a reverse description, these orientation words do not indicate and imply that the device or element being referred to must have a specific orientation or be constructed and operated in a specific orientation, and therefore, should not be considered as limiting the scope of the present invention; the terms "inner and outer" refer to the inner and outer relative to the profile of the respective component itself.
The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims (10)
1. A method for processing messages in a distributed system, comprising:
determining a plurality of service modules of a target business in a distributed system, wherein each service module in the plurality of service modules is used for completing one or more business services in the target business;
configuring message middleware for the plurality of service modules, wherein the message middleware is used for processing event messages transmitted among the plurality of service modules.
2. The method of claim 1, wherein after configuring message middleware for the plurality of service modules, the method further comprises:
processing, with the message middleware, event messages sent between the plurality of service modules;
processing, with the message middleware, event messages received between the plurality of service modules.
3. The method of claim 2, wherein processing, with the message middleware, event messages sent between the plurality of service modules comprises:
a sending end of the message middleware starts a timing task to continuously scan data with the type of 1 in an event table of the sending end according to the timing task, wherein the data is data sent from a first service module in the service modules to a second service module in the service modules, and the type of 1 indicates addition;
modifying the type of the data into 2 under the condition that the data with the type of 1 exists in the event table, wherein the type of 2 represents that the data is sent;
packaging the data into an event message, and storing the event message into a message queue of the message middleware;
and processing the event message in the message queue by utilizing the message middleware so as to send the event message to the second service module.
4. The method of claim 3, wherein in processing event messages transmitted between the plurality of service modules using the message middleware, the method further comprises:
when the event message in the message queue is processed by the message middleware, if the service is off-line, the data with the type 2 is rolled back to the state with the type 1.
5. The method of claim 3, wherein encapsulating the data as an event message for storage in a message queue of the message middleware comprises:
and packaging the query data, the modification data and the sending data with the association into the same event message, and storing the same event message into a message queue of the message middleware.
6. The method of claim 2, wherein processing, with the message middleware, event messages received between the plurality of service modules comprises:
monitoring an event message at a consuming end of the message middleware;
consuming the event message to store the data in the event message into an event table of a consuming end, and modifying the type of the data into 1, wherein the type of the data is 1 and indicates that the data is sent;
and sending a notification of successful receiving to a sending end of the message middleware.
7. The method of claim 6, wherein in the process of consuming the event message and storing the data in the event message into the event table of the consuming side, the method further comprises:
and when the event message is repeated for N times and the consumption is not completed, storing the event message into a dead credit queue so as to run a compensation mechanism according to the dead credit queue.
8. A message processing apparatus for a distributed system, comprising:
the system comprises a determining unit, a service processing unit and a service processing unit, wherein the determining unit is used for determining a plurality of service modules of a target service in a distributed system, and each service module in the plurality of service modules is used for completing one or more service services in the target service;
a configuration unit, configured to configure message middleware for the plurality of service modules, where the message middleware is configured to process event messages transmitted between the plurality of service modules.
9. A storage medium, characterized in that the storage medium comprises a stored program, wherein the program when executed performs the method of any of the preceding claims 1 to 7.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor executes the method of any of the preceding claims 1 to 7 by means of the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111538371.1A CN114356596A (en) | 2021-12-15 | 2021-12-15 | Message processing method and device for distributed system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111538371.1A CN114356596A (en) | 2021-12-15 | 2021-12-15 | Message processing method and device for distributed system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114356596A true CN114356596A (en) | 2022-04-15 |
Family
ID=81098601
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111538371.1A Pending CN114356596A (en) | 2021-12-15 | 2021-12-15 | Message processing method and device for distributed system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114356596A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116431367A (en) * | 2023-06-12 | 2023-07-14 | 中国航空结算有限责任公司 | Method, system and computer readable storage medium for modifying ticket information |
| CN118158184A (en) * | 2024-05-11 | 2024-06-07 | 中移(苏州)软件技术有限公司 | Information monitoring method, device, and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020108325A1 (en) * | 2018-11-27 | 2020-06-04 | 阿里巴巴集团控股有限公司 | Transaction processing method, apparatus and device |
| CN112671827A (en) * | 2020-11-25 | 2021-04-16 | 紫光云技术有限公司 | Distributed transaction final consistency method |
| CN113449039A (en) * | 2021-07-09 | 2021-09-28 | 中国银行股份有限公司 | System and method for processing data of different databases in distributed microservice |
-
2021
- 2021-12-15 CN CN202111538371.1A patent/CN114356596A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2020108325A1 (en) * | 2018-11-27 | 2020-06-04 | 阿里巴巴集团控股有限公司 | Transaction processing method, apparatus and device |
| CN112671827A (en) * | 2020-11-25 | 2021-04-16 | 紫光云技术有限公司 | Distributed transaction final consistency method |
| CN113449039A (en) * | 2021-07-09 | 2021-09-28 | 中国银行股份有限公司 | System and method for processing data of different databases in distributed microservice |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116431367A (en) * | 2023-06-12 | 2023-07-14 | 中国航空结算有限责任公司 | Method, system and computer readable storage medium for modifying ticket information |
| CN116431367B (en) * | 2023-06-12 | 2023-09-08 | 中国航空结算有限责任公司 | Method, system and computer readable storage medium for modifying ticket information |
| CN118158184A (en) * | 2024-05-11 | 2024-06-07 | 中移(苏州)软件技术有限公司 | Information monitoring method, device, and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111651244B (en) | Distributed transaction processing system | |
| EP2194495B1 (en) | A transaction aware, flexible interface for a state correlation and transition execution engine | |
| US11681683B2 (en) | Transaction compensation for single phase resources | |
| CN109213770B (en) | Data processing method, system, computer device and storage medium | |
| CN110502319B (en) | Distributed transaction processing method and device, electronic equipment and storage medium | |
| CN114356596A (en) | Message processing method and device for distributed system | |
| CN110888718A (en) | Method and device for realizing distributed transaction | |
| EP2196906B1 (en) | Cluster-based business process management through eager displacement and on-demand recovery | |
| CN112632093A (en) | Work order processing method, device, system, storage medium and program product | |
| US10425778B2 (en) | Distributed transactions on mobile devices via a messaging service provided by a mobile network operator | |
| CN114327799B (en) | Distributed transaction processing method and device, electronic equipment and storage medium | |
| US8606833B2 (en) | Method and system of committing operations of a synchronized transaction | |
| EP1671229B1 (en) | Automatic registration and deregistration of message queues | |
| CN112988775B (en) | Method, computing device and storage medium for processing batch transactions | |
| CN110647385A (en) | Distributed transaction execution method, device and system | |
| US8024371B2 (en) | Managing inverse references between objects in object-oriented software | |
| CN115208834A (en) | Service flow limiting method based on database storage process design | |
| CN117950813B (en) | Transaction management method, device and medium based on distributed application runtime | |
| Bernstein et al. | DECdta-Digital's distributed transaction processing architecture | |
| Böttcher et al. | Reducing sub-transaction aborts and blocking time within atomic commit protocols | |
| CN116541424A (en) | Online small batch suspicious record processing method and device | |
| CN113849278A (en) | Global transaction processing method and system | |
| CN119322658A (en) | Distributed transaction processing method, system, equipment and storage medium | |
| CN116954833A (en) | Distributed transaction consistency solving method, system, equipment and storage medium | |
| CN116719580A (en) | Service processing method, device, terminal equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |