CN111488367A - Method, device and equipment for realizing data consistency and computer storage medium - Google Patents

Abstract

The invention relates to the technical field of financial science and technology, and discloses a method, a device, equipment and a computer storage medium for realizing data consistency. The method comprises the following steps: when receiving an affair opening instruction, calling a preset API (application program interface) to generate an affair context; when a data write-in command is received, calling a preset API (application programming interface) to generate a corresponding forward transaction element and a corresponding reverse transaction element, and caching the forward transaction element and the reverse transaction element into a transaction context; when a transaction submitting instruction is received, packaging forward transaction elements cached in a transaction context into a Redis transaction, and submitting and executing the Redis transaction to a main node; when the execution is completed, calling a preset WAIT command to determine whether write-in data corresponding to the Redis transaction is successfully synchronized to the standby node; if the synchronization fails, a transaction rollback is performed based on the cached reverse transaction elements in the transaction context. The method and the device can ensure the consistency of the data in the Redis cluster when the information island occurs.

Description

Method, device, device and computer storage medium for realizing data consistency

technical field

The present invention relates to the technical field of financial technology (Fintech), and in particular, to a method, apparatus, device and computer storage medium for realizing data consistency.

Background technique

With the development of computer technology, more and more technologies are applied in the financial field, and the traditional financial industry is gradually transforming into financial technology (Fintech). requirements.

Various transaction systems of Internet banks are faced with a large number of user requests to be processed. In order to improve processing efficiency, distributed caching technology is usually used to cache products, parameters, users and other information, and Redis (an in-memory database) is a commonly used distributed caching technology. In order to ensure the high availability of the Redis cluster, the master/slaves (master-slave) high-availability cluster deployment mode is usually adopted, that is, one Redis instance is used as the master node Master to support data writing and reading, and then Synchronize the written data to the standby node; use N Redis instances as the standby node Slaves. When the master node crashes due to software and hardware exceptions, Sentinels (sentry, Redis monitoring process) elects a standby node as the master node to continue Serve.

However, when the network of the master node is interrupted due to some events, the Sentinels corresponding to the slave node slaves will think that the master has collapsed, and then elect one of the slaves to become the new master and notify the application instance. So the application instance on the standby node switches the connection to the new Master and continues to read and write; but the application instance on the old Master cannot receive the notification, so the application instance on the old Master is still connected to the old Master , continue reading and writing. In this way, there are two Masters in the Redis cluster, and different application instances read and write to the two Masters, and data inconsistency occurs in the cluster. When the network is restored, the old Master returns to the cluster, but it can only be the Slave identity, and all its data will be overwritten by the new Master data, so all the data written to it after the network is disconnected will be lost. Although the transaction is successful, the data is lost. Therefore, when the master node Master becomes an information island, how to avoid the occurrence of data inconsistency in the Redis cluster is a problem that needs to be solved urgently.

SUMMARY OF THE INVENTION

The main purpose of the present invention is to provide a method, device, device and computer storage medium for realizing data consistency, aiming at ensuring the consistency of data in the Redis cluster when an isolated information island occurs.

In order to achieve the above object, the present invention provides a method for realizing data consistency, which includes:

When receiving the transaction opening instruction, call the preset application program interface API to generate the transaction context;

When receiving a data write command, call the preset API to generate forward transaction elements and reverse transaction elements corresponding to the data write command, and cache them in the transaction context;

When receiving the transaction commit instruction, encapsulate the forward transaction element cached in the transaction context as a Redis transaction, and submit and execute the Redis transaction to the master node;

When the execution is completed, the preset waiting WAIT command is called to determine whether the write data corresponding to the Redis transaction is successfully synchronized to the standby node;

If synchronization fails, the transaction is rolled back based on the reverse transaction element cached in the transaction context.

Optionally, the step of encapsulating the forward transaction element cached in the transaction context into a Redis transaction when receiving a transaction commit instruction, and submitting and executing the Redis transaction to the master node includes:

When receiving the transaction commit instruction, monitor the first key key corresponding to the forward transaction element cached in the transaction context through the WATCH command, and open the first command queue through the MULTI command;

The forward transaction elements are sequentially added to the first command queue according to the cache order, and the Redis transaction is obtained by encapsulation;

Submit and execute the Redis transaction to the master node through the EXEC command.

Optionally, the method for realizing data consistency further includes:

When an abnormal rollback command is received, the rollback command of the preset API is called, the commands in the first command queue are cleared, and execution is abandoned.

Optionally, the step of performing transaction rollback based on the reverse transaction element cached in the transaction context includes:

A lock is applied to the second key corresponding to the reverse transaction element cached in the transaction context, and the second command queue is opened by the MULTI command;

appending the reverse transaction elements to the second command queue in reverse order according to the cache order, and encapsulating to obtain a rollback transaction;

Submit and execute the rollback transaction to the master node through an EXEC command, so as to roll back the transaction.

Optionally, the step of applying a lock to the second key corresponding to the reverse transaction element cached in the transaction context includes:

Apply an exclusive lock to the second key corresponding to the reverse transaction element cached in the transaction context through the SETNX command; or,

The second key corresponding to the reverse transaction element cached in the transaction context is monitored through the WATCH command to apply an optimistic lock.

Optionally, the transaction context includes a forward transaction element stack and a reverse transaction element stack, and the call to the preset API generates a forward transaction element and a reverse transaction element corresponding to the data write command, and The steps of caching into the transaction context include:

Calling the preset API to generate a forward transaction element corresponding to the data write command, and generating a reverse transaction element corresponding to the forward transaction element according to a preset mapping relationship;

The forward transaction element is cached into the stack of forward transaction elements, and the reverse transaction element is cached into the stack of reverse transaction elements.

Optionally, when the execution is completed, the step of invoking a preset waiting WAIT command to determine whether the write data corresponding to the Redis transaction is successfully synchronized to the standby node includes:

When the execution is completed, call the preset waiting WAIT command to obtain the number of standby nodes that return confirmation within the preset time;

determine whether the number is greater than or equal to a preset threshold;

If the number is less than the preset threshold, it is determined that the write data corresponding to the Redis transaction has not been successfully synchronized to the standby node.

In addition, in order to achieve the above object, the present invention also provides a device for realizing data consistency, and the device for realizing data consistency includes:

The generation module is used to call the preset application program interface API to generate the transaction context when receiving the transaction opening instruction;

a cache module, configured to call the preset API to generate forward transaction elements and reverse transaction elements corresponding to the data write command when receiving a data write command, and cache them in the transaction context;

an encapsulation module, configured to encapsulate the forward transaction element cached in the transaction context into a Redis transaction when receiving a transaction commit instruction, and submit and execute the Redis transaction to the master node;

A determination module, configured to call a preset waiting WAIT command when the execution is completed, to determine whether the write data corresponding to the Redis transaction is successfully synchronized to the standby node;

A rollback module, configured to roll back the transaction based on the reverse transaction element cached in the transaction context if the synchronization fails.

In addition, in order to achieve the above object, the present invention also provides a device for realizing data consistency. The device for realizing data consistency includes: a memory, a processor, and a device that is stored on the memory and can run on the processor. A data consistency realization program, when the data consistency realization program is executed by the processor, realizes the steps of the above-mentioned data consistency realization method.

In addition, in order to achieve the above object, the present invention also provides a computer storage medium, on which a program for realizing data consistency is stored, and when the program for realizing data consistency is executed by a processor, the above-mentioned program is realized. The steps of the implementation method of data consistency.

The present invention provides a method, device, device and computer storage medium for realizing data consistency. When a transaction opening instruction is received, a preset API is called to generate a transaction context; when a data write command is received, the preset API is called. Generate corresponding forward transaction elements and reverse transaction elements, and cache them in the transaction context; when receiving a transaction commit instruction, encapsulate the forward transaction elements cached in the transaction context as a Redis transaction, and submit the Redis transaction to the master node for execution Transaction; when the execution is completed, the preset WAIT command is called to determine whether the written data corresponding to the Redis transaction is successfully synchronized to the standby node; if the synchronization fails, the transaction is rolled back based on the reverse transaction element cached in the transaction context. The present invention provides a preset API for business programs to call the preset API to perform Redis writing. Specifically, the preset API does not actually execute Redis writing, but generates corresponding forward transaction elements and reverse transactions. The elements are cached. When the business program submits the transaction, the cached forward transaction element is encapsulated into a real Redis transaction and submitted for execution. By submitting the write command to the master node for execution at one time, the authenticity of the transaction can be greatly reduced. The execution time also largely avoids the problem of data inconsistency caused by the network interruption of the master node in the Redis cluster. When the execution is completed, the WAIT command is called to determine whether the written data is synchronized successfully, and when the synchronization fails, the transaction is rolled back based on the reverse transaction element, which can avoid the situation of "the transaction is successful, but the data is lost". . Therefore, the present invention can ensure the consistency of data in the Redis cluster when the information island occurs in the master node.

Description of drawings

1 is a schematic diagram of a device structure of a hardware operating environment involved in an embodiment of the present invention;

FIG. 2 is a schematic flowchart of a first embodiment of a method for realizing data consistency according to the present invention;

3 is a schematic diagram of the data writing process involved in the implementation method of data consistency of the present invention;

FIG. 4 is a schematic diagram of functional modules of the first embodiment of the apparatus for realizing data consistency according to the present invention.

The realization, functional characteristics and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments.

Detailed ways

It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

Referring to FIG. 1 , FIG. 1 is a schematic diagram of a device structure of a hardware operating environment involved in an embodiment of the present invention.

The device for realizing data consistency in the embodiment of the present invention may be a smart phone, and may also be a terminal device such as a PC (Personal Computer), a tablet computer, and a portable computer.

As shown in FIG. 1 , the device for realizing data consistency may include: a processor 1001 , such as a CPU, a communication bus 1002 , a user interface 1003 , a network interface 1004 , and a memory 1005 . Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. Optionally, the network interface 1004 may include a standard wired interface and a wireless interface (eg, a Wi-Fi interface). The memory 1005 may be high-speed RAM memory, or may be non-volatile memory, such as disk memory. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001 .

Those skilled in the art can understand that the device structure for implementing data consistency shown in FIG. 1 does not constitute a limitation on the device for implementing data consistency, and may include more or less components than those shown in the figure, or combine some components, or a different arrangement of components.

As shown in FIG. 1 , the memory 1005 as a computer storage medium may include an operating system, a network communication module, and a program for realizing data consistency.

In the terminal shown in FIG. 1 , the network interface 1004 is mainly used to connect to the background server and perform data communication with the background server; the user interface 1003 is mainly used to connect to the client and perform data communication with the client; and the processor 1001 can be used for The program for realizing data consistency stored in the memory 1005 is called, and each step of the following method for realizing data consistency is executed.

Based on the above hardware structure, various embodiments of the method for realizing data consistency of the present invention are proposed.

The present invention provides a method for realizing data consistency.

Referring to FIG. 2 , FIG. 2 is a schematic flowchart of a first embodiment of a method for realizing data consistency of the present invention.

In this embodiment, the implementation method of the data consistency includes:

Step S10, when receiving the transaction opening instruction, calling a preset application program interface API to generate a transaction context;

The method for realizing data consistency in this embodiment is realized by a device for realizing data consistency, and the device is described by taking a server as an example. The implementation device of data consistency includes a preset API (Application Programming Interface). The preset API can encapsulate the write operation to Redis as a transaction. As shown in Figure 3, when a business program calls this preset API to perform Redis writing, the preset API does not actually execute Redis writing, but generates the corresponding forward transaction elements and caches them, that is, writes to Redis commands are cached. At the same time, a corresponding reverse transaction element is generated to cache the reverse operation command. When a business program submits a transaction, the preset API encapsulates the cached command as a real Redis transaction and submits it for execution. After the transaction is executed, the preset API calls the WAIT command to determine whether the written data is synchronized successfully. If the synchronization is successful, the information of the successful execution is returned to the business program; if the synchronization fails, the key to be operated corresponding to the reverse transaction element is locked first, and then the reverse command is automatically executed in reverse order, and the information of the execution failure is returned to business procedure.

In this embodiment, when a transaction start instruction sent by a business program is received, that is, when a business program starts a transaction, a preset application program interface API is invoked to generate a transaction context. Specifically, a transaction context (Redis TransactionContext) can be created by a transaction manager (Redis Transaction Manager) in the preset API. The transaction context is created when a business program starts a transaction, and is automatically cleared when the transaction ends.

Step S20, when receiving the data write command, call the preset API to generate forward transaction elements and reverse transaction elements corresponding to the data write command, and cache them in the transaction context;

When the data write command sent by the business program is received, that is, when the business program wants to write to Redis, the preset API is called to generate forward transaction elements and reverse transaction elements corresponding to the data write command, and cache them. into the transaction context. Among them, the forward transaction element and the reverse transaction element can contain the following attributes: 1) Redis command Command, 2) Key (key) to be operated, 3) Value (value) to be written, 4) Others, such as expiration time Wait. The forward transaction element represents the write operation to Redis, and the operation of the reverse transaction element and the forward transaction element is reversed. For example, if the data write command is an overwrite write string set(k,v), The command command of the corresponding forward transaction element is set (write), the key is k, and the value is v. At this time, the command command of the reverse transaction element is delete (delete), and the key is k.

It can be understood that each time a business program sends a data write command, it will call a preset API to generate corresponding forward transaction elements and reverse transaction elements, and cache them in the transaction context.

Wherein, the transaction context includes a forward transaction element stack and a reverse transaction element stack, and step S20 includes:

Step a21, calling the preset API to generate a forward transaction element corresponding to the data write command, and generating a reverse transaction element corresponding to the forward transaction element according to a preset mapping relationship;

Step a22: Cache the forward transaction element in the forward transaction element stack, and cache the reverse transaction element in the reverse transaction element stack.

In order to facilitate caching of forward transaction elements and reverse transaction elements, the transaction context may include a forward transaction element stack and a reverse transaction element stack, where the forward transaction element stack is used to cache forward transaction elements, and reverse transaction elements. The element stack is used to cache reverse transaction elements.

When a data write command is received, the preset API is called to generate a forward transaction element corresponding to the data write command, and a reverse transaction element corresponding to the forward transaction element is generated according to the preset mapping relationship; the specific forward transaction The generation method of elements and reverse transaction elements can refer to the mapping relationship in the following table.

Then, the forward transaction elements are cached into the stack of forward transaction elements, and the reverse transaction elements are cached into the stack of reverse transaction elements.

That is to say, every time a business program sends a data write command and calls the preset API to write to Redis, the data write command can be added to the forward transaction as a forward transaction element through the preset API. When each forward transaction element is cached in the forward transaction stack, there must be a corresponding reverse transaction element cached in the reverse transaction element stack.

Step S30, when receiving the transaction submission instruction, encapsulate the forward transaction element cached in the transaction context as a Redis transaction, and submit and execute the Redis transaction to the master node;

When receiving the transaction commit instruction sent by the business program, the forward transaction element cached in the transaction context can be encapsulated as a Redis transaction through the transaction manager in the preset API, and the Redis transaction can be submitted to the master node for execution. Specifically, the first key key corresponding to the forward transaction element cached in the transaction context is monitored through the WATCH command, and the first command queue is opened through the MULTI command; then, the forward transaction elements are sequentially added to the first key in the cache order. In the command queue, the Redis transaction is encapsulated and obtained; then the Redis transaction is submitted to the master node through the EXEC command for execution. For the specific submission process of the Redis transaction, reference may be made to the following second embodiment, which will not be repeated here.

Step S40, when the execution is completed, call the preset waiting WAIT command to determine whether the write data corresponding to the Redis transaction is successfully synchronized to the standby node;

When the execution is completed, that is, when the data is written to the master node in the Redis cluster, the preset wait WAIT command is called to determine whether the written data corresponding to the Redis transaction is successfully synchronized to the standby node of the Redis cluster.

Specifically, step S40 includes:

Step a41, when the execution is completed, call the preset waiting WAIT command to obtain the number of standby nodes that return confirmation within the preset time;

Step a42, judging whether the quantity is greater than or equal to a preset threshold;

Step a43, if the number is less than the preset threshold, it is determined that the write data corresponding to the Redis transaction has not been successfully synchronized to the standby node.

In this embodiment, the process of judging whether the written data is synchronized successfully is: when the execution is completed, calling a preset waiting WAIT command to obtain the number of standby nodes that return confirmation within a preset time; wherein, the preset WAIT command includes There are two input parameters: numreplicas (number of replicas, the number of data backups) and timeout (timeout), where numreplicas represents the number of slaves (standby nodes), and timeout represents the timeout time in milliseconds. The preset time is the timeout parameter. Then, it is judged whether the number of standby nodes returned for confirmation is greater than or equal to a preset threshold, where the preset threshold is the numreplicas parameter. If the number is less than the preset threshold, it is determined that the write data corresponding to the Redis transaction has not been successfully synchronized to the standby node, that is, the synchronization fails; if the number is greater than or equal to the preset threshold, it is determined that the write data corresponding to the Redis transaction has been successfully synchronized to the standby node, that is, the synchronization is successful.

For example, if the preset WAIT command is WAIT(1,2000), it requires at least one standby node to confirm that the data has been synchronized within 2 seconds (2000 milliseconds). If the result returned by the WAIT command is greater than or equal to 1, the data has been successfully synchronized; if the result returned by the WAIT command is 0, the data synchronization has failed. Of course, it can be understood that the two parameters of the preset WAIT command can be configured in the configuration file in advance according to the actual situation of the Redis cluster.

If the synchronization fails, step S50 is executed to perform transaction rollback based on the reverse transaction element cached in the transaction context.

If synchronization fails, the transaction is rolled back based on the reverse transaction element cached in the transaction context. At the same time, the execution failure information can also be returned to the business program. For the transaction rollback process, first lock the second key corresponding to the reverse transaction element cached in the transaction context, and open the second command queue through the MULTI command; then, append the reverse transaction element to the In the second command queue, encapsulate the rollback transaction; finally submit the rollback transaction to the master node through the EXEC command for transaction rollback. For the specific process, reference may be made to the following third embodiment, which will not be repeated here.

Further, if the synchronization is successful, the information of successful execution is returned to the business program.

It should be noted that if the WAIT command is simply combined on the basis of the existing technology, when the master node Master has an information island due to network reasons, although it can be determined that the synchronization fails, the standby node Slave can be notified to terminate the transaction. Avoid the above situation of "the transaction is successful, but the data is lost". However, due to the occasional momentary network congestion between the active and standby nodes, for example, when multiple systems may transmit large files at the same time, will cause network congestion. This momentary blockage is likely to cause the data synchronization time between the Master and the Slave to exceed the preset time in the WAIT command, but after the blockage, the Master and the Slave will synchronize successfully. This short-term network blockage will not cause the Master to /Slave switch. As a result, there will be a situation of "although the transaction fails, but the data is successfully written to Redis", and there is still a problem of data consistency. However, Redis does not support transaction rollback. In the embodiment of the present invention, automatic data rollback is implemented through a preset API, which avoids the occurrence of data inconsistency caused by instantaneous network congestion.

The embodiment of the present invention provides a method for realizing data consistency. When a transaction opening instruction is received, a preset API is called to generate a transaction context; when a data write command is received, a corresponding forward transaction is generated by calling the preset API. elements and reverse transaction elements, and cache them in the transaction context; when receiving the transaction commit instruction, encapsulate the forward transaction elements cached in the transaction context as a Redis transaction, and submit the Redis transaction to the master node for execution; when the execution is completed , call the preset WAIT command to determine whether the written data corresponding to the Redis transaction is successfully synchronized to the standby node; if the synchronization fails, the transaction is rolled back based on the reverse transaction element cached in the transaction context. In the embodiment of the present invention, a preset API is provided for business programs to call the preset API to perform Redis writing. Specifically, the preset API does not actually execute Redis writing, but generates corresponding forward transaction elements and reverse transaction elements. Cache the transaction element. When the business program submits the transaction, the cached forward transaction element is encapsulated into a real Redis transaction and submitted for execution. By submitting the write command to the master node for execution at one time, the transaction can be greatly reduced. It also largely avoids the problem of data inconsistency caused by the network interruption of the master node in the Redis cluster. When the execution is completed, the WAIT command is called to determine whether the written data is synchronized successfully, and when the synchronization fails, the transaction is rolled back based on the reverse transaction element, which can avoid the situation of "the transaction is successful, but the data is lost". . Therefore, the embodiment of the present invention can ensure the consistency of data in the Redis cluster when the information island occurs on the master node.

Further, based on the above-mentioned first embodiment, a second embodiment of the method for realizing data consistency of the present invention is proposed.

In this embodiment, step S30 includes:

Step a31, when receiving the transaction submission instruction, monitor the first key key corresponding to the forward transaction element cached in the transaction context through the WATCH command, and open the first command queue through the MULTI command;

In this embodiment, when the transaction commit instruction is received, the first key (key) corresponding to the forward transaction element cached in the transaction context is monitored through the WATCH command, and the first command queue is opened through the MULTI command. Among them, the WATCH command is an optimistic locking (optimistic locking), which is used to monitor one or more keys before the EXEC command is executed, and when the EXEC command is executed, check whether at least one of the monitored keys has been modified by other commands. If it has been modified, the server will refuse to execute the transaction and return the result of the business program Redis transaction execution failure. The MULTI command is used to mark the beginning of a transaction block, which can open the command queue.

Step a32, the forward transaction elements are sequentially added to the first command queue according to the cache order, and the Redis transaction is obtained by encapsulation;

Then, the forward transaction elements are sequentially added to the first command queue according to the cache order (ie, the forward order), and the Redis transaction is obtained by encapsulation.

Step a33, submit and execute the Redis transaction to the master node through an EXEC command.

Further, submit and execute the Redis transaction to the master node through the EXEC command, that is, submit all commands in the first command queue to the master node Master for execution at one time. Among them, the EXEC command is used to execute commands within all transaction blocks (that is, in Redis transactions).

Through the above method, the forward transaction elements (corresponding to each Redis write operation) in the forward transaction element stack of the transaction context can be packaged and submitted to the master node in the Redis cluster for execution at one time, which can greatly reduce the actual execution of the transaction. time. It also largely avoids the problem of data inconsistency caused by the network interruption of the master node Master.

Further, after step a33, the realization method of this data consistency also includes:

When an abnormal rollback command is received, the rollback command of the preset API is called, the commands in the first command queue are cleared, and execution is abandoned.

In this embodiment, when the business program encounters an exception during execution, it can send an exception rollback instruction. At this time, when the server receives the exception rollback instruction, it calls the rollback command rollback() command of the preset API, Clears the command in the first command queue and aborts execution. Specifically, the DISCARD command can be used to clear the commands in the first command queue and give up execution. Among them, the DISCARD command is used to cancel the transaction and abandon the execution of all commands in the transaction block (ie, in the Redis transaction).

Further, based on the above-mentioned first embodiment, a third embodiment of the method for realizing data consistency of the present invention is proposed.

In this embodiment, step S50 includes:

Step a51, applying a lock to the second key corresponding to the reverse transaction element cached in the transaction context, and opening the second command queue by the MULTI command;

In this embodiment, a lock is applied to the second key corresponding to the reverse transaction element cached in the transaction context, and the second command queue is opened through the MULTI command. Here, it should be noted that the purpose of applying a lock to the second key is to avoid the occurrence of data inconsistency when other business programs just need to modify the same data key during the transaction rollback process.

Further, the step "locking the second key corresponding to the reverse transaction element cached in the transaction context" includes:

Apply an exclusive lock to the second key corresponding to the reverse transaction element cached in the transaction context through the SETNX command; or,

The second key corresponding to the reverse transaction element cached in the transaction context is monitored through the WATCH command to apply an optimistic lock.

In this embodiment, the locking methods can include two types: 1) apply an exclusive lock to the second key corresponding to the reverse transaction element cached in the transaction context through the SETNX command; 2) use the WATCH command to reverse the cache in the transaction context The second key corresponding to the transaction element is monitored for optimistic locking. Among them, SETNX is the abbreviation of "SET if Not eXists". The SETNX command can be used to implement exclusive locks. The code for locking is as follows: setnx(key, value), when a transaction executes the setnx command and returns 1, it means that the key does not exist originally. The transaction successfully adds an exclusive lock to the key; when a transaction executes setnx and returns 0, it means that the key already exists, and the transaction fails to grab the lock for the key.

If the transaction adds an exclusive lock to the key, the transaction can operate on the key, and other transactions cannot add any locks to the key until the transaction releases the lock on the key, thereby ensuring that other transactions are released in this transaction. Before the lock on the key can no longer operate on the key. Optimistic locking means that before submitting data updates, each transaction will first check whether other transactions have operated on the key after the transaction has operated on the key. If other transactions are updated, the transaction that is being submitted will be rolled back. At this time, due to the different rollback operations of various commands, it will cause a serious burden on each developer, and it is more prone to errors and affects the system. stability. Therefore, in order to avoid the above situation and avoid data inconsistency caused by some special circumstances, preferably, the second key corresponding to the reverse transaction element can be locked by applying an exclusive lock.

Step a52, appending the reverse transaction elements to the second command queue in reverse order according to the cache order, and encapsulating to obtain a rollback transaction;

Step a53, submit and execute the rollback transaction to the master node through an EXEC command, so as to roll back the transaction.

Then, the reverse transaction elements are added to the second command queue in reverse order according to the cache order, and the rollback transaction is obtained by encapsulation; the rollback transaction is submitted to the master node through the EXEC command to perform transaction rollback.

In the above manner, when a transaction rollback is required, the reverse command can be automatically executed in reverse order based on the reverse transaction elements in the reverse transaction element stack through a preset API, so as to realize the transaction rollback.

Further, it should be noted that in the actual application process, when the business program writes data, it usually writes to the relational database first, then writes to the Redis cluster, and then writes to the relational database, then writes to the Redis cluster, and loops in turn , therefore, in order to ensure the consistency of read and write transactions of business programs. The transaction of the preset API can be embedded in the transaction process of the relational database. Specifically, as described above and shown in FIG. 3 , the preset API provides business programs with three interfaces for opening a transaction, submitting a transaction and rolling back a transaction, and these three interfaces can be embedded in the transaction process of the relational database ,details as follows:

1) After opening the relational database transaction, open the default API transaction;

2) Before submitting a relational database transaction, submit this preset API transaction first;

3) Before rolling back the relational database transaction, first roll back the default API transaction;

Specifically, the above integration can be easily accomplished by means of Spring AOP (Aspect-Oriented Programming), or the predefined transaction interceptor Transaction Interceptor and Transaction Synchronization Adpter in Spring. detail.

In the above manner, based on the preset API of the present invention, the integration of Redis and relational database transactions can be realized, thereby further ensuring the consistency of read and write transactions of business programs.

The invention also provides a device for realizing data consistency.

Referring to FIG. 4 , FIG. 4 is a schematic diagram of functional modules of the first embodiment of the apparatus for realizing data consistency of the present invention.

As shown in FIG. 4 , the device for realizing data consistency includes:

The generating module 10 is configured to call a preset application program interface API to generate a transaction context when receiving the transaction opening instruction;

The cache module 20 is configured to, when receiving a data write command, call the preset API to generate forward transaction elements and reverse transaction elements corresponding to the data write command, and cache them in the transaction context;

The encapsulation module 30 is configured to encapsulate the forward transaction element cached in the transaction context into a Redis transaction when receiving the transaction submission instruction, and submit and execute the Redis transaction to the master node;

The determination module 40 is configured to call the preset waiting WAIT command when the execution is completed, to determine whether the write data corresponding to the Redis transaction is successfully synchronized to the standby node;

The rollback module 50 is configured to roll back the transaction based on the reverse transaction element cached in the transaction context if the synchronization fails.

Further, the packaging module 30 includes:

The queue opening unit is used to monitor the first key key corresponding to the forward transaction element cached in the transaction context through the WATCH command when receiving the transaction submission instruction, and open the first command queue through the MULTI command;

a first encapsulation unit, configured to sequentially append the forward transaction elements to the first command queue according to the cache order, and encapsulate to obtain a Redis transaction;

The first submitting unit is configured to submit and execute the Redis transaction to the master node through an EXEC command.

Further, the device for realizing data consistency also includes:

The abandoning execution module is configured to call the rollback command of the preset API when receiving an abnormal rollback instruction, clear the commands in the first command queue and abandon the execution.

Further, the rollback module 50 includes:

A locking unit, for applying a lock to the second key corresponding to the reverse transaction element cached in the transaction context, and opening the second command queue through the MULTI command;

a second encapsulation unit, configured to append the reverse transaction elements to the second command queue in reverse order according to the cache order, and encapsulate to obtain a rollback transaction;

The second submitting unit is configured to submit and execute the rollback transaction to the master node through an EXEC command, so as to perform transaction rollback.

Further, the locking unit is specifically used for:

Apply an exclusive lock to the second key corresponding to the reverse transaction element cached in the transaction context through the SETNX command; or,

The second key corresponding to the reverse transaction element cached in the transaction context is monitored through the WATCH command to apply an optimistic lock.

Further, the transaction context includes a forward transaction element stack and a reverse transaction element stack, and the cache module 20 includes:

a generating unit, configured to call the preset API to generate a forward transaction element corresponding to the data write command, and generate a reverse transaction element corresponding to the forward transaction element according to a preset mapping relationship;

A caching unit, configured to cache the forward transaction element in the forward transaction element stack, and cache the reverse transaction element in the reverse transaction element stack.

Further, the determining module 40 includes:

The calling unit is used to call the preset waiting WAIT command when the execution is completed, so as to obtain the number of standby nodes that return confirmation within the preset time;

a judgment unit for judging whether the quantity is greater than or equal to a preset threshold;

A determining unit, configured to determine that the write data corresponding to the Redis transaction is not successfully synchronized to the standby node if the number is less than a preset threshold.

Wherein, the function implementation of each module in the above-mentioned apparatus for realizing data consistency corresponds to each step in the above-mentioned embodiment of the above-mentioned method for realizing data consistency, and the functions and implementation process thereof will not be repeated here.

The present invention also provides a computer storage medium, where a program for realizing data consistency is stored thereon, and when the program for realizing data consistency is executed by a processor, the data consistency according to any of the above embodiments is realized The steps of the realization method.

The specific embodiments of the computer storage medium of the present invention are basically the same as the above-mentioned embodiments of the data consistency implementation method, and will not be repeated here.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or system comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or system. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article or system that includes the element.

The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

From the description of the above embodiments, those skilled in the art can clearly understand that the method of the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on such understanding, the technical solutions of the present invention can be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products are stored in a storage medium (such as ROM/RAM) as described above. , magnetic disk, optical disk), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) execute the methods described in the various embodiments of the present invention.

The above are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied in other related technical fields , are similarly included in the scope of patent protection of the present invention.

Claims (10)

1. A method for implementing data consistency is characterized in that the method for implementing data consistency comprises the following steps:

when a transaction opening instruction is received, calling a preset Application Program Interface (API) to generate an object context;

when a data write-in command is received, calling the preset API to generate a forward transaction element and a reverse transaction element corresponding to the data write-in command, and caching the forward transaction element and the reverse transaction element into the transaction context;

when a transaction submitting instruction is received, packaging forward transaction elements cached in the transaction context into a Redis transaction, and submitting and executing the Redis transaction to a main node;

when the execution is completed, calling a preset WAIT WAIT command to determine whether write-in data corresponding to the Redis transaction is successfully synchronized to a standby node;

and if the synchronization fails, performing transaction rollback based on the cached reverse transaction elements in the transaction context.

2. An implementation method of data consistency according to claim 1, wherein the step of encapsulating a forward transaction element cached in the transaction context as a Redis transaction and committing execution of the Redis transaction to a master node upon receiving a transaction commit instruction comprises:

when a transaction submission instruction is received, monitoring a first key corresponding to a forward transaction element cached in the transaction context through a WATCH command, and opening a first command queue through an MU L TI command;

sequentially adding the forward transaction elements to the first command queue according to a cache sequence, and packaging to obtain a Redis transaction;

and submitting to the main node through an EXEC command to execute the Redis transaction.

3. The method for implementing data consistency of claim 2, wherein the method for implementing data consistency further comprises:

and when an abnormal rollback instruction is received, calling a rollback command of the preset API, clearing the commands in the first command queue and abandoning execution.

4. The method of claim 1, wherein the step of transaction rollback based on the cached backward transaction elements in the transaction context comprises:

applying a lock to a second key corresponding to a reverse transaction element cached in the transaction context, and opening a second command queue through an MU L TI command;

adding the reverse transaction elements to the second command queue in a reverse order according to a cache sequence, and packaging to obtain a rollback transaction;

submitting the rollback transaction to the master node for transaction rollback through an EXEC command.

5. The method of claim 4, wherein the step of locking the second key corresponding to the cached reverse transaction element in the transaction context comprises:

applying an exclusive lock to a second key corresponding to a cached reverse transaction element in the transaction context through an SETNX command; or the like, or, alternatively,

monitoring a second key corresponding to a cached reverse transaction element in the transaction context through a WATCH command to apply an optimistic lock.

6. The method for implementing data consistency according to claim 1, wherein the transaction context includes a forward transaction element stack and a reverse transaction element stack, and the step of calling the API to generate the forward transaction element and the reverse transaction element corresponding to the data write command and cache the forward transaction element and the reverse transaction element in the transaction context includes:

calling the preset API to generate a forward transaction element corresponding to the data write-in command, and generating a reverse transaction element corresponding to the forward transaction element according to a preset mapping relation;

caching the forward transaction element into the forward transaction element stack and caching the reverse transaction element into the reverse transaction element stack.

7. The method for implementing data consistency according to claim 1, wherein the step of invoking a preset WAIT command to determine whether write data corresponding to the Redis transaction is successfully synchronized to a standby node when the execution is completed comprises:

when the execution is finished, calling a preset WAIT WAIT command to acquire the number of the backup nodes returning confirmation in the preset time;

judging whether the number is greater than or equal to a preset threshold value;

and if the number is smaller than a preset threshold value, determining that the write-in data corresponding to the Redis transaction is not successfully synchronized to the standby node.

8. An apparatus for implementing data consistency, the apparatus for implementing data consistency comprising:

the generating module is used for calling a preset Application Program Interface (API) to generate an object context when receiving an object starting instruction;

the cache module is used for calling the preset API to generate a forward transaction element and a reverse transaction element corresponding to the data write-in command when the data write-in command is received, and caching the forward transaction element and the reverse transaction element into the transaction context;

the encapsulation module is used for encapsulating forward transaction elements cached in the transaction context into Redis transactions and submitting and executing the Redis transactions to the main node when a transaction submitting instruction is received;

the determining module is used for calling a preset WAIT WAIT command when the execution is completed so as to determine whether write-in data corresponding to the Redis transaction is successfully synchronized to the standby node;

and the rollback module is used for rolling back the transaction based on the reverse transaction element cached in the transaction context if the synchronization fails.

9. An implementation device of data consistency, characterized in that the implementation device of data consistency comprises: memory, processor and a data consistency implementation program stored on the memory and executable on the processor, the data consistency implementation program, when executed by the processor, implementing the steps of the data consistency implementation method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that the computer storage medium stores thereon a data consistency implementation program, and the data consistency implementation program, when executed by a processor, implements the steps of the data consistency implementation method according to any one of claims 1 to 7.

Images