Disclosure of Invention
An object of the present application is to provide a data consistency processing method and device, which not only complete execution of a task, but also perform fast retry and bottom retry on a task that fails to be executed to ensure that the task can be successfully executed, thereby achieving final consistency of data in a service system.
According to an aspect of the present application, a data consistency processing method is provided, wherein the method includes:
creating a message component in a business system, wherein the message component comprises a message queue and a work thread;
executing and monitoring a target task in the business system through the message component;
if the target task fails to be executed, carrying out retry operation on the target task through the message queue and recording retry times;
and if the target task still fails to be executed when the retry times reach a preset retry time threshold value, retrying the target task through the working thread according to a preset retry frequency until the target task is successfully executed.
Further, in the above method, the method further includes:
and if the target task fails to be executed, updating the task state of the target task from non-execution to execution failure.
Further, in the above method, the method further includes:
and if the target task fails to be executed, maintaining the task state of the target task as the execution failure.
Further, in the above method, the method further includes:
and if the target task is successfully executed, updating the task state of the target task from execution failure to execution success.
Further, in the above method, the method further includes:
and if the target task is successfully executed, updating the task state of the target task from non-execution to successful execution.
Further, in the above method, the message component further includes a task table, and the task table is used to store all tasks to be executed and all retried tasks in the service system.
According to another aspect of the present application, there is also provided a non-volatile storage medium having computer readable instructions stored thereon, which, when executed by a processor, cause the processor to implement the data consistency processing method as described above.
According to another aspect of the present application, there is also provided a data consistency processing apparatus, wherein the apparatus includes:
one or more processors;
a computer-readable medium for storing one or more computer-readable instructions,
when executed by the one or more processors, cause the one or more processors to implement the data coherency processing method described above.
Compared with the prior art, the method and the system have the advantages that a message component is created in the business system, and the message component comprises a message queue and a work thread; executing and monitoring a target task in the business system through the message component; if the target task fails to be executed, performing retry operation on the target task through the message queue and recording retry times so as to realize quick retry of the target task through the message queue when the target task fails to be executed; if the retry times reach a preset retry time threshold value, the target task is still executed unsuccessfully, the target task is retried through the working thread according to a preset retry frequency until the target task is executed successfully, so that the target task can be successfully executed through the working thread, all tasks in the business system can be guaranteed to be executed successfully finally, the purpose of successfully executing the target task is achieved, the task which is executed unsuccessfully is quickly retried and retried in a bottom-in-bottom mode, the task can be guaranteed to be executed successfully finally, and the final consistency of data in the business system is achieved.
Detailed Description
The present application is described in further detail below with reference to the attached figures.
In a typical configuration of the present application, the terminal, the device serving the network, and the trusted party each include one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include non-transitory computer readable media (transient media), such as modulated data signals and carrier waves.
As shown in fig. 1, one aspect of the present application provides a flow diagram of a data consistency processing method, which may employ a relational database management system MySQL, oracel, and other data stores. The method comprises a step S11, a step S12, a step S13 and a step S14, and specifically comprises the following steps:
step S11, creating a message component in the business system, wherein the message component comprises a message queue and a work thread; here, the message component is used to ensure final consistency of related data such as tasks in the service system, the message component is made in a form of JAR (Java Archive) package, and the message component made in the JAR package is introduced into the service system for integration, as shown in fig. 2.
It should be noted that, as shown in fig. 3, the Message components for ensuring data consistency in the service system include an API (Application Programming Interface), a task table, a Message Queue (MQ), and a Worker thread Worker. The API is used for providing task state in the processes of creating a task, inquiring the task, executing the task, supervising the task execution, updating the task state and the like; the task table is used for storing all tasks to be executed and all retried tasks in the service system; the message queue is used for carrying out retry operation on the task which is failed to be executed when the task is failed to be executed, but the retry operation of the task which is failed to be executed by the message queue MQ is limited retry, and after the retry operation reaches a preset retry number threshold, if the task is still failed to be executed, the task which is failed to be executed is carried out with bottom compensation operation through the working thread Worker, so that the task can be successfully executed. In the embodiment of the application, the message queue MQ can be replaced by other message queues such as a Rabbit MQ and a socket MQ; the Worker thread Worker may use a task scheduling tool xxl-joba or elastic-joba, etc.
Step S12, executing and monitoring target tasks in the service system through the message component;
step S13, if the target task fails to be executed, performing retry operation on the target task through the message queue and recording retry times, so that when the target task in the service system fails to be executed, the target task can be quickly retried through the message queue.
Step S14, if the target task still fails to be executed when the retry number reaches a preset retry number threshold, retrying the target task by the worker thread according to a preset retry frequency until the target task is successfully executed.
Through the steps S11 to S14, when the target task in the business system fails to be executed, the target task is preferentially retried through the message queue, and when the number of retries in the message queue reaches the preset retry number threshold and the execution still fails, the target task is retried through the work thread according to the preset retry frequency until the target task is successfully executed, so that the target task can be successfully executed through the work thread, and all tasks in the business system can be successfully executed finally.
Next to the foregoing embodiments of the present application, a data consistency processing method proposed by the present application further includes:
and if the target task fails to be executed, updating the task state of the target task from non-execution to execution failure.
For example, after the target task in the service system is executed and monitored through the message component in step S12, if the target task fails to be executed, the task state of the target task is updated from non-execution to failed execution, so as to achieve the purpose of monitoring and updating the task state of the target task in real time.
Next to the foregoing embodiments of the present application, a data consistency processing method proposed by the present application further includes:
and if the target task fails to be executed, maintaining the task state of the target task as the execution failure.
For example, in step S13, the message queue MQ performs retry operation on the target task and records retry times, and if the target task still fails to execute during retry of the target task within a preset retry time threshold, the task state of the target task is maintained as execution failure, so as to achieve the purpose of monitoring the task state of the target task in real time during retry of the target task through the message queue.
Next to the foregoing embodiments of the present application, a data consistency processing method proposed by the present application further includes:
and if the target task is successfully executed, updating the task state of the target task from execution failure to execution success.
For example, in step S13, the message queue MQ performs retry operation on the target task and records retry times, and if the target task is successfully executed in a certain retry in the process of performing retry within the preset retry time threshold on the target task, the task state of the target task is updated from execution failure to execution success, so as to achieve the purpose of monitoring and updating the task state in real time when the target task is successfully executed in the process of performing retry on the target task through the message queue.
Next to the foregoing embodiments of the present application, a data consistency processing method proposed by the present application further includes:
and if the target task is successfully executed, updating the task state of the target task from non-execution to successful execution.
For example, after the target task is executed and monitored in the service system through the message component in step S12, if the target task is successfully executed, the task state of the target task is updated from non-execution to successful execution, so as to achieve the purpose of monitoring and updating the task state of the target task in real time.
In an actual application scenario of the present application, as shown in fig. 4, after a task component for data consistency is created in a business system, when a task node in the business system executes a task, a write task is preferably operated first (here, a target task is preferably a write task), at this time, a task state of the write task is not executed, when the write task starts executing, the write task starts executing through the message component, and a task state of the write task in a task table is queried, if the write task is successfully executed, a task state of the write task is updated to be successful, and if the write task fails to execute, the task state of the write task is updated to be failed to be executed, and meanwhile, a retry policy of the task is triggered. For example, a quick retry operation is preferentially performed on the write task through the message queue MQ, if the retry number of the retry through the MQ does not reach the preset retry number threshold and the write task is successfully executed during retry, the task state of the write task is updated from execution failure to execution success, if the retry number of the retry through the MQ reaches the preset retry number threshold and the write task is still failed to be executed, the task state of the write task is continuously maintained as execution failure, and the target task is subjected to retry of bottom compensation through the working thread Worker according to the preset retry frequency until the target task is successfully executed, so that the write task can be successfully executed through the working thread, all tasks in the service system can be successfully executed finally, the purpose of successful execution of the write task is achieved, and the task which is failed to be executed is quickly retried and subjected to bottom retry to ensure that the task can be successfully executed finally, the final consistency of the data in the service system is achieved.
In the embodiment of the application, in a distributed system based on message driving, a task component for ensuring data consistency provides a rapid integration capability, a task node in a business system after integration is initialized in the task component, the task component monitors and processes a task execution step, if the execution fails, rapid retry and bottom-of-pocket compensation are carried out, and the task can be successfully executed to achieve final data consistency.
According to another aspect of the present application, there is also provided a non-volatile storage medium having computer readable instructions stored thereon, which, when executed by a processor, cause the processor to implement the data consistency processing method as described above.
According to another aspect of the present application, there is also provided a data consistency processing apparatus, wherein the apparatus includes:
one or more processors;
a computer-readable medium for storing one or more computer-readable instructions,
when executed by the one or more processors, cause the one or more processors to implement the data coherency processing method described above.
Here, for details of each embodiment in the data consistency processing apparatus, reference may be specifically made to corresponding parts of the embodiments of the data consistency processing method, and details are not described here again.
In summary, the application creates a message component in the service system, where the message component includes a message queue and a work thread; executing and monitoring a target task in the business system through the message component; if the target task fails to be executed, performing retry operation on the target task through the message queue and recording retry times so as to realize quick retry of the target task through the message queue when the target task fails to be executed; if the retry times reach a preset retry time threshold value, the target task is still executed unsuccessfully, the target task is retried through the working thread according to a preset retry frequency until the target task is executed successfully, so that the target task can be successfully executed through the working thread, all tasks in the business system can be guaranteed to be executed successfully finally, the purpose of successfully executing the target task is achieved, the task which is executed unsuccessfully is quickly retried and retried in a bottom-in-bottom mode, the task can be guaranteed to be executed successfully finally, and the final consistency of data in the business system is achieved.
It should be noted that the present application may be implemented in software and/or a combination of software and hardware, for example, implemented using Application Specific Integrated Circuits (ASICs), general purpose computers or any other similar hardware devices. In one embodiment, the software programs of the present application may be executed by a processor to implement the steps or functions described above. Likewise, the software programs (including associated data structures) of the present application may be stored in a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. Additionally, some of the steps or functions of the present application may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.
In addition, some of the present application may be implemented as a computer program product, such as computer program instructions, which when executed by a computer, may invoke or provide methods and/or techniques in accordance with the present application through the operation of the computer. Program instructions which invoke the methods of the present application may be stored on a fixed or removable recording medium and/or transmitted via a data stream on a broadcast or other signal-bearing medium and/or stored within a working memory of a computer device operating in accordance with the program instructions. An embodiment according to the present application comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to perform a method and/or a solution according to the aforementioned embodiments of the present application.
It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the apparatus claims may also be implemented by one unit or means in software or hardware. The terms first, second, etc. are used to denote names, but not any particular order.