CN118885284B

CN118885284B - Distributed queue lock execution method based on key-value database

Info

Publication number: CN118885284B
Application number: CN202411390697.8A
Authority: CN
Inventors: 李南锋
Original assignee: Tianjin Nankai University General Data Technologies Co ltd
Current assignee: Tianjin Nankai University General Data Technologies Co ltd
Priority date: 2024-10-08
Filing date: 2024-10-08
Publication date: 2025-02-18
Anticipated expiration: 2044-10-08
Also published as: CN118885284A

Abstract

The present invention provides a distributed queue lock execution method based on a key-value database, which can be applied to the field of database technology. The method includes: in response to detecting a data access request of a first client for a target resource in a key-value database, reading first value information according to key information of a distributed lock corresponding to the target resource, the first value information including lock queue information of the distributed lock; determining the occupation result of the distributed lock according to the reading result of the first value information; when the occupation result indicates that the distributed lock is occupied by a second client, adding the client identifier of the first client to the lock queue information to obtain the updated first value information; writing the updated first value information in the key-value database to control the first client to access the target resource after the second client accesses the target resource.

Description

Distributed queue lock execution method based on key-value database

Technical Field

The invention relates to the technical field of databases, in particular to a distributed queue lock execution method based on a key-value database.

Background

In a multitasking operating system, multiple tasks running simultaneously may all require the use of the same resource. In order to ensure the rationality of the use of the resources, when a task needs to use the resources, the used resources need to be locked and protected, the lock is also called as a mutual exclusion lock, and after the lock protection, the concurrent access of the systems to the resources can be correct and reasonable.

In the programming mode of a traditional standalone environment, a lock programming interface provided by the database operating system is typically used to lock the resources that need to be protected. However, in a distributed database, since the database task runs on a plurality of different physical machines, the lock control policy in the original stand-alone environment is not available in the distributed scenario, and thus a locking mechanism suitable for the distributed scenario is needed to provide protection for the shared resources in the distributed system. In addition, in the process of realizing a locking mechanism, normal processing of tasks needs to be ensured, and the condition that service starvation occurs due to long-time locking failure is avoided.

Disclosure of Invention

In view of the above problems, the present invention provides a key-value database-based distributed queue lock execution method.

According to a first aspect of the invention, a distributed queue lock execution method based on a key-value database is provided, and the method comprises the steps of responding to a data access request of a first client side for a target resource in the key-value database, reading first value information according to key information of a distributed lock corresponding to the target resource, wherein the first value information comprises lock queue information of the distributed lock, determining an occupation result of the distributed lock according to a reading result of the first value information, opening a queuing transaction when the occupation result represents that the distributed lock is occupied by a second client side, adding a client side identification of the first client side into the lock queue information to obtain updated first value information, and writing the updated first value information into the key-value database so as to control the first client side to access the target resource after the second client side completes access to the target resource.

According to the embodiment of the invention, the method further comprises restarting the queuing transaction after randomly dormancy for a preset time period to add the client identification of the first client to the lock queue information again in response to detecting that the write operation of the updated first value information has a commit conflict.

According to the embodiment of the invention, the method further comprises the steps of responding to receiving lock change notification information aiming at the distributed lock, obtaining second value information of the distributed lock, wherein the lock change notification information represents that the value information of the distributed lock changes, extracting a client identifier positioned at the first position of the lock queue information from lock queue information in the second value information, writing a client address of the first client into the second value information under the condition that the client identifier is matched with the client identifier of the first client, modifying lock state information in the second value information into occupied state, deleting the client identifier of the first client from the lock queue information, and writing updated second value information in a key-value database.

According to the embodiment of the invention, the method further comprises the steps of determining the change type of the distributed lock according to the lock state information in the second value information, and extracting the client identification positioned at the first position of the lock queue information from the lock queue information in the second value information under the condition that the change type is determined to be release of the distributed lock.

According to the embodiment of the invention, the method further comprises the steps of writing the client address of the first client into first value information under the condition that the occupation result is unoccupied, modifying lock state information in the first value information into occupation to obtain updated first value information, and writing the updated first value information into a key-value database.

According to the embodiment of the invention, the method further comprises the steps of responding to the fact that the first client ends the access to the target resource, and acquiring third value information of the distributed lock;

and under the condition that the lock queue information is not empty, modifying the lock state information in the third value information into unoccupied state information, deleting the client address in the third value information to obtain updated third value information, and under the condition that the lock queue information is empty, deleting the third value information.

According to the embodiment of the invention, the occupation result of the distributed lock is determined according to the reading result of the first value information, wherein the determination of the occupation result of the distributed lock is performed when the reading result is that the reading is successful, and the determination of the occupation result of the distributed lock is performed when the reading result is that the reading is failed.

According to the embodiment of the invention, the first value information further comprises lock keep-alive time and lock write time. The method comprises the steps of determining an occupation result of a distributed lock according to a reading result of first value information, obtaining current time when the reading result is successful, calculating time difference between the current time and lock writing time, determining that the occupation result of the distributed lock is occupation timeout when the time difference and lock keep-alive time meet preset conditions, and determining that the occupation result of the distributed lock is occupation when the time difference and the lock keep-alive time do not meet preset conditions.

According to the embodiment of the invention, the method further comprises the steps of modifying the lock state information in the first value information into unoccupied client addresses in the first value information under the condition that the occupation result is the occupation timeout, deleting the client addresses in the first value information to obtain updated first value information, starting a queuing transaction, and adding the client identification of the first client into the lock queue information to obtain updated first value information.

The second aspect of the invention provides a distributed queue lock execution device based on a key-value database, which comprises a reading module and a writing module, wherein the reading module is used for responding to a data access request of a first client side for a target resource in the key-value database, reading the first value information according to key information of a distributed lock corresponding to the target resource, the first value information comprises lock queue information of the distributed lock, the occupying module is used for determining an occupying result of the distributed lock according to a reading result of the first value information, the queuing module is used for opening a queuing transaction when the occupying result represents that the distributed lock is occupied by a second client side, adding a client side identifier of the first client side to the lock queue information to obtain updated first value information, and the writing module is used for writing the updated first value information in the key-value database so as to control the first client side to access the target resource after the first client side completes access to the target resource.

A third aspect of the invention provides an electronic device comprising one or more processors and a memory for storing one or more computer programs which when executed by the one or more processors implement the steps of a method of performing a distributed queue lock based on a key-value database as described above.

A fourth aspect of the invention also provides a computer program product comprising a computer program or instructions which, when executed by a processor, performs the steps of a method of distributed queue lock execution according to the above-described key-value database.

The embodiment of the invention not only realizes a distributed lock mechanism in a distributed scene through a key-value mechanism, but also adds a queuing mechanism under the mechanism, so that the traditional concurrent lock is changed into sequential queuing lock, invalid contention among clients is avoided, and the system overhead caused by lock preemption is greatly reduced. Meanwhile, the embodiment of the invention can also avoid the phenomenon that the business starves because of no locking, and cause an out-of-order database system to become ordered.

Drawings

The foregoing and other objects, features and advantages of the invention will be apparent from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

Fig. 1 shows an application scenario of a key-value database-based distributed queue lock execution method according to an embodiment of the invention.

FIG. 2 illustrates a flow chart of a key-value database based distributed queue lock execution method according to an embodiment of the invention.

FIG. 3 illustrates a timing interaction diagram of occupying transactions and queuing transactions according to an embodiment of the present invention.

Fig. 4 shows a schematic diagram of an interaction for releasing a distributed lock according to a specific embodiment of the invention.

Fig. 5 shows a lock occupancy interaction schematic according to an embodiment of the invention.

FIG. 6 illustrates a flow diagram of a distributed lock queue mechanism according to an embodiment of the invention.

FIG. 7 illustrates a flow diagram of a distributed lock occupancy mechanism according to an embodiment of the present invention.

Fig. 8 shows a block diagram of a distributed queue lock execution apparatus based on a key-value database according to an embodiment of the invention.

Fig. 9 shows a block diagram of an electronic device adapted for a key-value database based distributed queue lock execution method according to an embodiment of the invention.

Detailed Description

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. It should be understood that the description is only illustrative and is not intended to limit the scope of the invention. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the invention. It may be evident, however, that one or more embodiments may be practiced without these specific details. In addition, in the following description, descriptions of well-known structures and techniques are omitted so as not to unnecessarily obscure the present invention.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The terms "comprises," "comprising," and/or the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.

All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be construed to have meanings consistent with the context of the present specification and should not be construed in an idealized or overly formal manner.

Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a convention should be interpreted in accordance with the meaning of one of skill in the art having generally understood the convention (e.g., "a system having at least one of A, B and C" would include, but not be limited to, systems having a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).

In the programming mode of a traditional standalone environment, a lock programming interface provided by the database operating system is typically used to lock the resources that need to be protected. The following will take four interfaces as examples to initialize the mutex lock variable, call the locking method to lock, use the shared resource after the locking is successful, and call the unlocking method to unlock. The method of initializing the exclusive lock variable is typically INT PTHREAD _mutex_init (pthread_mutex_t_mutex, const pthread_ mutexattr _t_attr).

The locking method is called to lock, which is generally INT PTHREAD _mutex_lock (pthread_mutex_t) for locking the exclusive lock. If the mutual exclusion lock is locked, the caller is always blocked in the call interface, so as to achieve the aim of lock protection. When the shared resource is used after the locking is successful, the use of the shared resource is safe due to the successful locking. The unlocking method is generally called INT PTHREAD _mutex_lock (pthread_mutex_t) to unlock. Thus, it can be seen that the mutex lock implemented through a programming interface is generally applicable to stand-alone environments.

In addition, in the existing distributed lock implementation mechanism, the distributed lock is generally simple to form mutual exclusion operation, and shared resource access is realized through concurrent lock operation. For example, the distributed lock function is realized based on a single database, the purpose of locking is mainly achieved by writing a unique index, the distributed lock realized based on redis is mainly realized by writing a key concurrently and ensuring that only one client is successfully written, and the distributed lock based on zookeeper is mainly realized by writing a path concurrently and ensuring that only one writing is successful. Therefore, the existing distributed lock implementation mechanisms are concurrent lock robbing mechanisms, and no queuing mechanism exists. This results in a large amount of overhead in the database system, and may also lead to traffic death due to non-locking.

Therefore, the embodiment of the invention provides a method for realizing the distributed queue lock based on the key-value database, which not only realizes the distributed lock, but also adds a queuing mechanism in a distributed lock realization mechanism. The distributed queue lock execution method based on the key-value database comprises the steps of responding to a data access request of a first client side for a target resource in the key-value database, reading first value information according to key information of a distributed lock corresponding to the target resource, determining an occupation result of the distributed lock according to a reading result of the first value information, opening a queuing transaction when the occupation result represents that the distributed lock is occupied by a second client side, adding a client side identification of the first client side into the lock queue information to obtain updated first value information, and writing the updated first value information into the key-value database so as to control the first client side to access the target resource after the second client side completes access to the target resource.

As shown in fig. 1, an application scenario 100 according to this embodiment may include a first client 101, a second client 102, a third client 103, and a database operating system 104.

The first client 101, the second client 102, and the third client 103 may be servers or database nodes for performing tasks. Database operating system 104 may include a cluster of servers for unified management of multiple databases, servers.

The network is used to provide a medium for communication links between the first client 101, the second client 102, the third client 103, and the database operating system 104. The network may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

It should be noted that, the method for executing the distributed queue lock based on the key-value database according to the embodiment of the present invention may be generally executed by one of the first client 101, the second client 102, and the third client 103. Accordingly, the distributed queue lock execution device based on the key-value database provided by the embodiment of the invention can be generally set in one of the first client 101, the second client 102 and the third client 103.

The method for executing the distributed queue lock based on the key-value database according to the embodiment of the invention will be described in detail with reference to fig. 2 to 7 based on the scenario described in fig. 1.

As shown in FIG. 2, the method 200 includes operations S210-S240.

In operation S210, in response to detecting a data access request of the first client for a target resource in the key-value database, first value information is read according to key information of a distributed lock corresponding to the target resource, where the first value information includes lock queue information of the distributed lock.

In operation S220, an occupation result of the distributed lock is determined according to the read result of the first value information.

In operation S230, in the case that the occupation result represents that the distributed lock is occupied by the second client, the queuing transaction is started, and the client identifier of the first client is added to the lock queue information, so as to obtain updated first value information.

In operation S240, the updated first value information is written in the key-value database, so as to control the first client to access the target resource after the second client completes the access to the target resource.

According to an embodiment of the invention, the first client may be a server, a database node or the like to access the target resource.

A key-value database is a database that stores data in key-value pairs, and can be understood as a distributed hash map (Hashmap), in which keys (primary keys) are used to locate values, i.e., store and retrieve specific values. In the key-value database, one key uniquely corresponds to one value. The value may be used to store any type of data, including integer, character, array, object, etc.

The key-value database adopted by the embodiment of the invention supports multi-node distributed deployment, meets high availability requirements, supports transaction atomicity (atomicity), consistency, isolation and persistence (durability) of data reading and writing, namely ACID characteristics, and supports a key-value change notification mechanism. For example, distributed database foundationdb may be used as a key-value database.

The target resource may be information pre-stored in a key-value database. For example, the target resource may be a shared resource that can be accessed by multiple clients.

Embodiments of the present invention utilize a key-value mechanism to implement distributed locks. For a distributed lock for a target resource, a key may be defined using the lock name of the distributed lock, and a value may be defined by a variety of information, such as lock queue information. The lock queue information is used to characterize whether other clients under the distributed lock are waiting in line to occupy the distributed lock.

When the first client accesses the target resource in the key-value database, the first client can only read the first value information corresponding to the target resource from the key-value database through the key information of the distributed lock corresponding to the target resource. It should be noted that, when accessing the target resource, the first client may construct key information of the distributed lock based on the target resource. For example, the key information may be a lock name, "path access lock".

Because the embodiment of the invention realizes the distributed lock based on the key-value mechanism, whether the distributed lock is occupied or not can be determined according to whether the first value information can be read from the key-value database or not. It should be noted that, due to the unique corresponding properties of the key and the value in the key-value mechanism, if the first value information is read, it indicates that the second client or other clients already occupy the distributed lock, and the occupation information is written into the first value information. If the first value information is not read, it indicates that the distributed lock is temporarily not already occupied by the client.

According to an embodiment of the present invention, operations S210 and S220 may be implemented by occupied transactions of the key-value database, and operations S230 and S240 may be implemented by queued transactions of the key-value database.

In the case that the distributed lock is occupied by the second client in operation S230, the occupied transaction may be ended, the distributed lock occupied operation of the first client is ended, and then the queued transaction is started, and the target resource is sequentially accessed through the queuing mechanism.

According to the embodiment of the invention, since the lock queue information of the distributed lock is included in the first value information, the client identification of the first client can be added to the lock queue information according to time data in queuing transactions. So far, the queuing transaction updates the specific value in the first value information.

For example, while a first client accesses a target resource of a key-value database, a second client is accessing the target resource, and yet a third client waits to access the target resource. Therefore, the lock queue information of the first value information before being updated comprises the client identification of the third client, and after queuing the transaction, the lock queue information in the updated first value information sequentially comprises the client identifications of the third client and the first client so as to realize sequential access of the second client, the third client and the first client to the target resource.

Then, in order to inform the key-value database of the occupation or queuing situation of the distributed lock, the updated first value information needs to be rewritten into the key-value database. Thereby, the queuing mechanism of the first client is completed.

It should be noted that the first value information, the second value information and the third value information in the present invention may be values read in real time for the same target resource under different access scenarios, or values obtained by executing the reading operation at different timings for the same target resource under the same access scenario.

According to embodiments of the invention, the value information of the distributed lock may include lock queue information, client address, and lock status information. The client address is used to characterize the address of the client currently occupying the distributed lock, such as an internet protocol address (Internet Protocol Address, IP address). The lock status information is used to characterize whether the distributed lock is currently occupied.

For the write operation of operation S240, since there may be other clients in the distributed environment, for example, the third client also performs the occupying and queuing operations synchronously, i.e., operations S210 to S240 are performed. At this time, the write operation of the first client with respect to the updated first value information may cause a commit collision.

In the event of a commit collision, the queued transaction of the first client needs to be retried at this time in order to guarantee the reliability of the data in the key-value database. In order to avoid the situation that the commit conflict still occurs when re-queuing, the queuing conflict of the third client is avoided through a random dormancy operation.

According to the embodiment of the invention, the random dormancy preset time length can be dormancy according to a predefined preset time length, and the random dormancy preset time length can also be dormancy by randomly selecting one preset time length from the preset time length interval. For example, dormancy can be randomly performed for 1ms to 10ms, and then the queuing transaction is restarted for queuing, so that the queuing conflict is effectively avoided again.

As shown in FIG. 3, a timing interaction diagram of occupied transactions and queued transactions includes operations S300-S308. Operation S300 characterizes that the distributed lock is currently already occupied by the second client. The occupied transaction comprises operations S301-S303, the queued transaction comprises operations S304-S306, and operations S307 and S308 are re-queuing operations in the case of a commit conflict.

In operation S300, occupancy. The distributed lock is currently already occupied by the second client.

In operation S301, first value information is read.

In operation S302, the reading is successful. Under the condition that the first value information is read, the reading is successful, and the distributed lock is indicated to be added by other clients, the locking fails, and the occupied transaction is ended. In operation S303, the process ends.

In operation S304, the first value information is read. It should be noted that, since there are other operations between operations S301 and S304, the first value information read in operation S304 may be the same as or different from the first value information read in operation S301.

In operation S305, a client identification of the first client is added to the lock queue information.

In operation S306, updated first value information is written.

In operation S307, a conflict is committed.

In operation S308, the sleep is random for a preset period of time.

It should be noted that, before occupying the transaction and queuing the transaction, the method further includes operations of opening the transaction by using a database interface of the key-value database, constructing key information, and inputting the key information by using a read data interface, and then the first client may read the first value information from the key-value database.

According to the embodiment of the present invention, after the random dormancy for a preset period in operation S308 is performed, the queuing transaction is restarted, that is, operations S304 to S306 are performed again, which will not be described herein.

According to the embodiment of the invention, after the first client performs the queuing transaction, the first client can monitor whether the value information of the distributed lock changes in real time. Or the key-value database may send lock change notification information to the first client through a notification mechanism when the value information of the distributed lock changes.

It is contemplated that multiple clients may be simultaneously present for a particular distributed lock for queuing. Since the plurality of clients are queued for locking, in the case that the distributed lock is released, the distributed lock is occupied by the first client. Therefore, considering the system consumption caused by the lock change notification, the key-value database can only send the lock change notification information to the first or the first clients according to the lock queue information, so that the number of the clients to be notified and the notification times are reduced, and the system consumption is reduced.

According to the embodiment of the invention, when the value information of the distributed lock changes, the method further comprises the steps of responding to receiving lock change notification information for the distributed lock, obtaining second value information of the distributed lock, wherein the lock change notification information represents the change of the value information of the distributed lock, extracting a client identifier positioned at the first position of the lock queue information from lock queue information in the second value information, writing a client address of the first client into the second value information when the client identifier is matched with the client identifier of the first client, modifying lock state information in the second value information into occupied state, deleting the client identifier of the first client from the lock queue information, and writing updated second value information in a key-value database.

According to the embodiment of the invention, after the first client receives the lock change notification information for the distributed lock, the first client is awakened, and can open a new transaction to judge whether the distributed lock can be occupied or not, for example, the lock can be opened to awaken the transaction.

The execution logic of the lock wakeup transaction includes queue validation and lock occupancy operations. Specifically, the queue verification includes extracting a client identifier located at the first position of the lock queue information from the lock queue information in the second value information, and judging whether the client identifier is matched with the client identifier of the first client. The lock occupation operation includes updating the second value information if the client identifier matches the client identifier of the first client, and writing the updated second value information.

It should be noted that, whether the lock change notification information is sent to the client located at the first position of the queue or the client located at the first few positions of the queue, the client needs to be verified again when receiving the lock change notification information, so as to avoid the robbery lock caused by the error notification.

For example, when the first client is a client located first in the lock queue information, the first client may receive lock change notification information for the distributed lock. And under the condition that the client identifier is matched with the client identifier of the first client, the first client is characterized as the first position in the lock queue information, namely the first position of the queue, and at the moment, the first client can realize the occupation of the distributed lock by modifying the second value information.

Specifically, when the distributed lock is released, the client address in the second value information acquired by the first client is cleared, and the lock state information is unoccupied, at this time, the first client can modify the client address by writing in the IP address of the first client, and modify the lock state information from unoccupied to occupied. In addition, since the first client is ready to occupy the distributed lock, the client identifier of the first client needs to be deleted in the lock queue information.

According to an embodiment of the present invention, after writing the updated second value information, the method further includes committing the transaction to complete the lock wakeup transaction. At this time, since the queue head information matching mechanism exists in the lock wakeup transaction, no commit collision is caused.

According to embodiments of the present invention, the change in value information of the distributed lock may include a plurality of change types, such as releasing the distributed lock, queuing changes.

Under the condition that the value information of the distributed lock is changed, if the lock state information is unoccupied, determining that the change type is releasing the distributed lock, and if the lock state information is occupied, determining that the change type is queuing change.

And ending the lock wakeup transaction in the case that the change type is determined to be the queuing change.

In operation S400, release is performed.

In operation S401, the value information of the distributed lock is changed.

In operation S402, lock change notification information is issued.

In operation S403, the lock state information is unoccupied.

In operation S404, second value information is requested.

In operation S405, the second value information is read.

In operation S406, a client identifier located at the first position of the lock queue information is extracted from the lock queue information in the second value information.

In operation S407, in case the client identifications match, the IP address and the lock state information are modified.

In operation S408, updated second value information is written.

It should be noted that, after the second client actively releases the distributed lock and writes the value information after release in operation S400, the key-value database may detect that the value information of the distributed lock changes. And then, the key-value database transmits the lock change notification information to the first client positioned at the first position of the lock queue information.

According to the embodiment of the invention, after determining the occupation result of the distributed lock in operation S220, the method further comprises the steps of writing the client address of the first client into first value information, modifying the lock state information in the first value information into occupation to obtain updated first value information under the condition that the occupation result is unoccupied, and writing the updated first value information in a key-value database.

According to the embodiment of the invention, the occupation result is unoccupied, which means that the distributed lock is not occupied currently, so that the first client can directly occupy without queuing, i.e. continue to execute occupied transactions.

According to the embodiment of the invention, since the occupation result is unoccupied and the first value information is empty, the IP address of the first client can be directly written into the first value information, and the lock state information in the first value information is modified into occupation to obtain updated first value information. And then, writing updated first value information in a key-value database to characterize that the distributed lock is occupied by the first client.

According to the embodiment of the invention, after the first client finishes accessing the target resource, the distributed lock can be actively released.

According to the embodiment of the invention, the method further comprises the steps of responding to the fact that the first client ends access to the target resource, obtaining third value information of the distributed lock, determining lock queue information in the third value information under the condition that the client address in the third value information is matched with the client address of the first client, modifying lock state information in the third value information to be unoccupied and deleting the client address in the third value information under the condition that the lock queue information is not empty, obtaining updated third value information, and deleting the third value information under the condition that the lock queue information is empty.

According to the embodiment of the invention, in response to detecting that the first client ends access to the target resource, a lock release transaction can be opened to realize active release of the distributed lock.

Since the operation of changing the value information caused by queuing or the like of the client may occur in the process that the first client occupies the distributed lock, the third value information of the current key information still needs to be read after the lock release transaction is opened.

In the actual application process, when the first client occupies the distributed lock, if the first client fails, other clients release the distributed lock to avoid deadlock, and then if the first client resumes, the distributed lock is released actively, which may result in false release. Therefore, in order to avoid false releases, it is necessary to verify the identity of the client, i.e. determine whether the address of the client in the third value information matches the address of the client performing the lock release transaction.

Under the condition that the first client actively releases the distributed lock, the identity verification of the first client is realized by determining whether the client address in the third value information is matched with the client address of the first client. And determining lock queue information in the third value information under the condition that the client address in the third value information is matched with the client address of the first client. At this time, the lock queue information in the third value information is determined in order to determine whether there are other clients queued. And if the lock queue information is empty, no other queued client exists.

And under the condition that the lock queue information is not empty, modifying the lock state information in the third value information into unoccupied client addresses in the third value information, and deleting the client addresses in the third value information to obtain updated third value information, so that after the first client releases the distributed lock, the distributed lock is occupied by other queued clients. And directly deleting the third value information under the condition that the lock queue information is empty.

According to the embodiment of the invention, a commit conflict can occur when a lock release transaction is committed, which indicates that a queued client request can exist at the moment, lock queue information is written, and the lock release transaction is re-executed.

For commit conflicts, the commit conflict for the occupied transaction may be because client A preempts the distributed lock when executing the occupied transaction by other clients, such as client B. At this time, the client a may re-execute the occupied transaction.

Queued transactions can avoid commit collisions by notifying only the first-located clients of the lock queue information.

When the first client accesses the key-value database, no other client occupies the distributed lock, so that the first client can occupy the distributed lock through operations S501-S504, actively release the distributed lock through operations S505-S510 after the access to the target resource is finished, and then occupy the distributed lock released by the first client through operation S511 by the second client. It should be noted that, at this time, the operation of the second client occupying the distributed lock is similar to that of the first client, and will not be described herein.

In operation S501, the first value information is read.

In operation S502, the reading fails.

In operation S503, the IP address, the current time, and the modified lock state information are written in the first value information.

In operation S504, updated first value information is written.

In operation S505, the third value information is read.

In operation S506, the reading is successful.

In operation S507, the IP address in the third value information coincides with the IP of the first client.

In operation S508, the lock queue information in the third value information is not empty.

In operation S509, the lock status information is modified to be unoccupied, and the IP address in the third value information is deleted.

In operation S510, updated third value information is written.

In operation S511, occupancy.

By the embodiment of the invention, the distributed lock can realize the following conditions of 1 and mutual exclusivity. At any moment, only one client acquires the distributed lock, and after one client acquires and occupies the distributed lock, the other clients can only wait. 2. High availability. In a distributed environment, the functions of the distributed lock need a higher level of availability, i.e. when some machines in the system are abnormal, the lock service can be continuously provided, so that the lock service is not interrupted. 3. Queue mechanism-if the distributed lock is held by another client at this time, then a wait is made for the client that needs the distributed lock, and when the distributed lock is released, the client that is at the head of the lock queue information will be notified.

In the embodiment of the invention, when the occupation result of the distributed lock is directly determined according to the reading result, if the distributed lock is not released by the client due to abnormality, other clients cannot acquire the distributed lock to form deadlock. Thus, lock validity determination may be added to the locking logic to handle out a failed distributed lock.

According to the embodiment of the invention, the first value information further comprises lock keep-alive time and lock write time. The lock keep-alive time is used to determine whether the currently occupied distributed lock is valid, e.g., the lock keep-alive time may be predefined. The lock write time refers to the time when the client occupies the distributed lock and can be determined by a system timestamp.

According to the embodiment of the invention, in the occupied transaction, the lock keep-alive time and the lock write time can be synchronously updated every time the IP address of the value information is updated.

The distributed queue lock execution based on the key-value database further comprises the steps of obtaining the current time when the reading result is that the reading is successful, calculating the time difference between the current time and the lock writing time, determining that the occupied result of the distributed lock is occupied overtime when the time difference and the lock keep-alive time meet preset conditions, and determining that the occupied result of the distributed lock is occupied when the time difference and the lock keep-alive time do not meet preset conditions. Thus, deadlock is avoided in the above manner.

Specifically, the time difference between the current time and the lock write time may be compared to the lock keep-alive time to perform a distributed lock validity determination. And under the condition that the distributed lock fails, determining that the occupation result of the distributed lock is overtime, and then passively releasing the distributed lock.

In an embodiment of the invention, the time difference may be the current time minus the lock keep-alive time.

According to an embodiment of the invention, the predetermined condition may be that the time difference is greater than a predetermined multiple, such as 2 times, of the lock keep-alive time. For example, if a client is abnormal, the lock keep-alive time of the distributed lock occupied by the client is not updated, and at this time, after the time difference between the current time and the lock write time exceeds 2 times of the lock keep-alive time, namely 10 seconds, the lock is invalid.

According to the embodiment of the invention, the lock failure judgment is added in the process of adding the distributed lock, so that the problem of deadlock of the distributed lock can be solved, the lock occupation progress is further promoted, and the resource waste of a database system is reduced.

According to embodiments of the present invention, in addition to adding lock validity determinations to occupied transactions, lock release operations may be added to handle failed distributed locks.

According to the embodiment of the invention, since the modification operation of the first value information, namely the modification of the lock state information and the client address, is newly added in the occupied transaction, the distributed lock which is invalid can be cleaned up when the occupied transaction is ended. Thus, the first client may normally execute the queued transaction.

In another embodiment, if the occupation result is the occupation timeout, whether other queued clients exist or not may be further determined according to the lock queue information in the first value information, and if no queued other clients exist, the IP address of the first client and the lock write time are directly written into the first value information, that is, the distributed lock is directly occupied, without opening the queued transaction. In the event that there are other clients queued, the queued transaction is restarted.

According to embodiments of the invention, lock keep-alive time can be updated by keep-alive processing. Specifically, the first client may periodically interact with the database system to continuously update the lock keep-alive time.

For example, when the first client occupies a distributed lock, the lock keep-alive time is a default of 5 seconds. If the first client needs to continuously occupy the distributed lock, the lock keep-alive time can be prolonged through interactive operation every 5 seconds. For example, through one interactive operation, the lock keep-alive time is prolonged to 10s or 15s, so that when other clients execute occupied transactions, the distributed lock occupied by the first client cannot be cleaned up by mistake due to invalidation.

As shown in fig. 6, after an occupied transaction is started for a client, such as client 1, of a target resource to be accessed, key information of a lock can be constructed for the target resource to be accessed, and the lock is a distributed lock, which is hereinafter referred to as a lock for short. After the key information is constructed, the value information is read according to the key information. After the value information is read, it is determined that the lock is currently occupied by other clients, such as client 2, and then the client identifier of client 1 needs to be written into lock queue information (not shown in the figure) in the value information. And then, writing key and value information in a key-value database, and submitting.

After the commit of the occupied transaction is successful, the client 1 is in a waiting wake-up state. And under the condition of awakening, reading value information according to the key information, and judging whether the client 1 is the first queue. In the case where client 1 is the first queue, the lock information is modified to occupy the distributed lock. And (5) ending after the locking is successful. In case the client 1 is not the first queue, the client 1 resumes the waiting-for-wake state.

And under the condition that the occupied transaction is not successfully submitted, namely the submission fails, the client 1 fails to lock, and retries after random dormancy.

As shown in fig. 7, after an occupied transaction is started for a client, such as client 2, of a target resource to be accessed, the key information of the lock can be constructed by the target resource to be accessed, and the lock is a distributed lock, which is hereinafter referred to as a lock for short. After the key information is constructed, the value information is read according to the key information.

And if the lock keep-alive time is invalid, namely the lock fails, the client 2 releases the current lock, builds the value information of the lock according to the information of the client and builds the lock writing time. For example, value information is constructed according to the IP address of the client 2, and lock write time is constructed according to the system timestamp. And then, writing key and value information in a key-value database, and submitting.

Further, for the commit operation, it is also necessary to determine whether the commit is successful. After the transaction is successfully committed, the client 2 clears the lock which is currently invalid, and the lock is occupied again, and the locking is finished after success. In the case that the occupied transaction is not successful in commit, i.e. commit fails, the client 2 fails to lock and waits for a retry.

Under the condition of failure of reading, the current lock is not occupied, and the value information, the writing time, the writing key and the value information of the lock can be sequentially constructed and submitted.

Compared with the existing distributed lock technology, the distributed queue lock execution method based on the key-value database has the following advantages that 1, the distributed locking requirement in a distributed database system can be met. 2. With fail lock detection. 3. Based on the distributed key-value database system, the logic is simple. 4. The high availability characteristic is satisfied, and the service can be continuously provided even if the node abnormality exists in the database system. 5. The system has a queue mechanism, avoids the system overhead caused by the polling preemption of the lock, and avoids starvation of a service layer caused by locking.

As shown in fig. 8, the key-value database-based distributed queue lock execution apparatus 800 of this embodiment includes a reading module 810, an occupying module 820, a queuing module 830, and a writing module 840.

The reading module 810 is configured to, in response to detecting a data access request of the first client for a target resource in the key-value database, read first value information according to key information of a distributed lock corresponding to the target resource, where the first value information includes lock queue information of the distributed lock.

The occupation module 820 is configured to determine an occupation result of the distributed lock according to the read result of the first value information.

The queuing module 830 is configured to, when the occupation result indicates that the distributed lock is occupied by the second client, open a queuing transaction, and add a client identifier of the first client to the lock queue information, to obtain updated first value information.

The writing module 840 is configured to write the updated first value information in the key-value database, so as to control the first client to access the target resource after the second client completes access to the target resource.

It should be noted that, in the embodiment of the present invention, the device portion corresponds to the method portion in the embodiment of the present invention, and the description of the device portion specifically refers to the method portion and is not described herein again.

Any of the plurality of modules of the reading module 810, the occupying module 820, the queuing module 830, and the writing module 840 may be combined in one module to be implemented, or any of the plurality of modules may be split into a plurality of modules according to an embodiment of the present invention. Or at least some of the functionality of one or more of the modules may be combined with, and implemented in, at least some of the functionality of other modules.

At least one of the read module 810, the occupy module 820, the queue module 830, the write module 840 may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or as hardware or firmware in any other reasonable way of integrating or packaging circuits, or as any one of or a suitable combination of three of software, hardware, and firmware, according to embodiments of the invention. Or at least one of the reading module 810, the occupying module 820, the queuing module 830, the writing module 840 may be at least partially implemented as a computer program module which, when executed, may perform the corresponding functions.

As shown in fig. 9, an electronic device 900 according to an embodiment of the present invention includes a processor 901 that can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 902 or a program loaded from a storage section 908 into a Random Access Memory (RAM) 903. The processor 901 may include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or an associated chipset and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), or the like. Processor 901 may also include on-board memory for caching purposes. Processor 901 may include a single processing unit or multiple processing units for performing the different actions of the method flows according to embodiments of the invention.

In the RAM 903, various programs and data necessary for the operation of the electronic device 900 are stored. The processor 901, the ROM 902, and the RAM 903 are connected to each other by a bus 904. The processor 901 performs various operations of the method flow according to an embodiment of the present invention by executing programs in the ROM 902 and/or the RAM 903. Note that the program may be stored in one or more memories other than the ROM 902 and the RAM 903. The processor 901 may also perform various operations of the method flow according to embodiments of the present invention by executing programs stored in the one or more memories.

According to an embodiment of the invention, the electronic device 900 may also include an input/output (I/O) interface 905, the input/output (I/O) interface 905 also being connected to the bus 904. The electronic device 900 may also include one or more of an input portion 906 including a keyboard, a mouse, etc., an output portion 907 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), etc., and a speaker, etc., a storage portion 908 including a hard disk, etc., and a communication portion 909 including a network interface card such as a LAN card, a modem, etc., connected to the input/output I/O interface 905. The communication section 909 performs communication processing via a network such as the internet. The drive 910 is also connected to the I/O interface 905 as needed. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is installed as needed on the drive 910 so that a computer program read out therefrom is installed into the storage section 908 as needed.

Embodiments of the present invention also include a computer program product comprising a computer program containing program code for performing the method shown in the flowcharts. The program code means for causing a computer system to carry out the methods provided by embodiments of the present invention when the computer program product is run on the computer system. The above-described functions defined in the system/apparatus of the embodiment of the present invention are performed when the computer program is executed by the processor 901. The systems, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the invention.

In one embodiment, the computer program may be based on a tangible storage medium such as an optical storage device, a magnetic storage device, or the like. In another embodiment, the computer program may also be transmitted, distributed, and downloaded and installed in the form of a signal on a network medium, via communication portion 909, and/or installed from removable medium 911. The computer program may comprise program code that is transmitted using any appropriate network medium, including but not limited to wireless, wireline, etc., or any suitable combination of the preceding. In such an embodiment, the computer program may be downloaded and installed from the network via the communication portion 909 and/or installed from the removable medium 911. The above-described functions defined in the system of the embodiment of the present invention are performed when the computer program is executed by the processor 901. The systems, devices, apparatus, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the invention.

According to embodiments of the present invention, program code for carrying out computer programs provided by embodiments of the present invention may be written in any combination of one or more programming languages, and in particular, such computer programs may be implemented in high-level procedural and/or object-oriented programming languages, and/or in assembly/machine languages. Programming languages include, but are not limited to, such as Java, c++, python, "C" or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

Those skilled in the art will appreciate that the features recited in the various embodiments of the invention can be combined and/or combined in a variety of ways, even if such combinations or combinations are not explicitly recited in the present invention. In particular, the features recited in the various embodiments of the invention can be combined and/or combined in various ways without departing from the spirit and teachings of the invention. All such combinations and/or combinations fall within the scope of the invention.

The foregoing description of the embodiments has been provided for the purpose of illustrating the general principles of the invention, and is not meant to limit the invention thereto, but to limit the invention thereto, and any modifications, equivalents, improvements and equivalents thereof may be made without departing from the spirit and principles of the invention.

Claims

1. A key-value database-based distributed queue lock execution method, the method comprising:

Responding to a data access request of a first client for a target resource in a key-value database, and reading first value information according to key information of a distributed lock corresponding to the target resource, wherein the first value information comprises lock queue information of the distributed lock;

determining an occupation result of the distributed lock according to the reading result of the first value information;

opening a queuing transaction under the condition that the occupation result represents that the distributed lock is occupied by a second client, adding the client identification of the first client into the lock queue information to obtain updated first value information, and

Writing the updated first value information in the key-value database so as to control the first client to access the target resource after the second client finishes accessing the target resource;

Further comprises:

The method comprises the steps of receiving lock change notification information for a distributed lock, obtaining second value information of the distributed lock, extracting a client identifier positioned at the first position of lock queue information from lock queue information in the second value information, writing a client address of the first client into the second value information when the client identifier is matched with the client identifier of the first client, modifying lock state information in the second value information into occupied state, deleting the client identifier of the first client from the lock queue information, and writing updated second value information in a key-value database;

The method comprises the steps of obtaining third value information of a distributed lock in response to the fact that a first client ends access to a target resource, determining lock queue information in the third value information under the condition that a client address in the third value information is matched with a client address of the first client, modifying lock state information in the third value information to be unoccupied under the condition that the lock queue information is not empty, deleting the client address in the third value information to obtain updated third value information, and deleting the third value information under the condition that the lock queue information is empty.

2. The method as recited in claim 1, further comprising:

And responding to the detection that the write operation of the updated first value information has a commit conflict, and restarting the queuing transaction after randomly dormancy for a preset time period so as to add the client identification of the first client into the lock queue information again.

3. The method as recited in claim 1, further comprising:

determining the change type of the distributed lock according to the lock state information in the second value information, and

And under the condition that the change type is determined to release the distributed lock, extracting the client identification positioned at the first position of the lock queue information from the lock queue information in the second value information.

4. The method as recited in claim 1, further comprising:

writing the client address of the first client into the first value information under the condition that the occupation result is unoccupied, modifying the lock state information in the first value information into occupation to obtain updated first value information, and

And writing the updated first value information in the key-value database.

5. The method of claim 1, wherein the determining the occupation result of the distributed lock according to the read result of the first value information comprises:

Under the condition that the reading result is that the reading is successful, determining that the occupation result of the distributed lock is in occupation;

and under the condition that the reading result is the reading failure, determining that the occupation result of the distributed lock is unoccupied.

6. The method of claim 1, wherein the first value information further comprises a lock keep-alive time and a lock write time, wherein the determining the occupation result of the distributed lock according to the read result of the first value information further comprises:

acquiring the current time under the condition that the reading result is that the reading is successful;

calculating a time difference between the current time and the lock write time;

under the condition that the time difference and the lock keep-alive time meet the preset conditions, determining that the occupation result of the distributed lock is occupation timeout;

and under the condition that the time difference and the lock keep-alive time do not meet the preset conditions, determining that the occupied result of the distributed lock is occupied.

7. The method as recited in claim 6, further comprising:

Under the condition that the occupation result is the occupation timeout, modifying the lock state information in the first value information into unoccupied state information, deleting the client address in the first value information to obtain updated first value information, and

And starting the queuing transaction, and adding the client identifier of the first client to the lock queue information to obtain updated first value information.

8. An electronic device, the electronic device comprising:

One or more processors;

A memory for storing one or more computer programs,

Characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1-7.