CN118779327A - Data processing method and device - Google Patents

Abstract

The application discloses a data processing method and a device, which relate to the field of cloud computing, the method is applied to a data detection system, the data detection system comprises a storage device, the storage device stores data identifiers and corresponding times of a main node and at least one sub node of a distributed system, and the method can comprise the following steps: acquiring a plurality of data identifications of a main node in a storage device, a plurality of times corresponding to the plurality of data identifications of the main node, and a first time of a target child node and a first data identification corresponding to the first time; determining a second time according to the first data identifier, the plurality of data identifiers of the master node and a plurality of times corresponding to the plurality of data identifiers of the master node; a time difference between the first time and the second time is determined, the time difference being used to characterize traffic data consistency of the master node and the target child node. Data consistency between a master node and a child node in a distributed cluster can be detected.

Description

Data processing method and device

This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on April 6, 2023, with application number 202310361323.2 and application name “A method, device and other equipment for consistency monitoring”, the entire contents of which are incorporated by reference in this application.

Technical Field

The present application relates to the field of cloud computing, and in particular to a data processing method and device.

Background Art

With the rapid development of computer technology, users have an increasing demand for services provided by servers. A single server can no longer meet users' large service needs, so distributed clusters came into being. Distributed clusters refer to the use of high-speed computer networks to connect multiple physically dispersed server nodes to form a logically unified cluster. The basic idea is to store the original centralized data in multiple server nodes connected through the network to obtain larger storage capacity and higher concurrent access. In order to improve the high availability of distributed clusters, distributed clusters can be set to multi-node and multi-copy mode. The so-called multi-node and multi-copy mode refers to synchronizing the data written by the master node server in the distributed cluster to multiple child node servers to generate corresponding data copies in the child node servers, thereby preventing service unavailability due to abnormal situations such as network partitions, server node downtime, and program pauses.

The effective application of multi-node and multi-copy mode requires that the replicas stored in the child nodes have a high degree of data consistency with the data written by the master node. Therefore, it is necessary to detect the data consistency between the master node and the child nodes in the distributed cluster.

Summary of the invention

The present application provides a data processing method and device, which can detect the data consistency between a master node and a sub-node in a distributed cluster.

In a first aspect, a data processing method is provided. The method is applied to a data detection system, the data detection system is used to perform consistency detection on business data of multiple nodes of a distributed system, the multiple nodes of the distributed system include a master node and at least one child node; the data detection system includes a storage device, the storage device stores data identifiers and corresponding times of the master node and the at least one child node, the data identifiers are used to indicate update operations of the business data of the master node and the at least one child node, the method may include:

Acquire multiple data identifiers of a master node in a storage device and multiple times corresponding to the multiple data identifiers of the master node, as well as a first time of a target subnode and a first data identifier corresponding to the first time; determine a second time according to the first data identifier, the multiple data identifiers of the master node, and the multiple times corresponding to the multiple data identifiers of the master node; the second time is a time before the earliest time among the multiple times of the master node when the business data corresponding to the first data identifier is covered by the business data corresponding to the multiple data identifiers of the master node; determine a time difference between the first time and the second time, and the time difference is used to characterize the consistency of the business data of the master node and the target subnode.

In the first aspect, a calculation method for the time difference that characterizes the consistency of business data between the master node and the target sub-node is defined. It does not require the data in the business of the master node and the sub-node to have the characteristics of monotonic transfer order, and is more versatile. Multiple time differences calculated between different sub-nodes and the master node can be compared, and the global data consistency between different sub-nodes and the master node can be measured.

In one possible implementation, determining the second time based on the first data identifier, multiple data identifiers of the master node, and multiple times corresponding to the multiple data identifiers of the master node can include: respectively comparing the first data identifier of the target subnode with the multiple data identifiers of the master node to determine the multiple target data identifiers of the master node whose business data corresponding to the first data identifier is covered by the business data corresponding to the multiple data identifiers of the master node; determining a third data identifier from the multiple target data identifiers, the time corresponding to the third data identifier being the third time; and determining the second time based on the third data identifier and the corresponding third time.

In this implementation, the first data identifier of the target subnode is compared with multiple data identifiers of the main node to determine the third time when the data identifier in the main node earliest covers the first data identifier of the target subnode, and then the second time is determined based on the third time, and the determined second time is more accurate.

In one possible implementation, determining a third data identifier from multiple target data identifiers includes: determining the target data identifier that is in the first order among the multiple target data identifiers as the third data identifier based on the order of multiple target times corresponding to the multiple target data identifiers of the master node in the storage device.

In this implementation, the target data identifier that is in the first order among the multiple target data identifiers is determined as the third data identifier, and the determined third data identifier can be used to determine the second time.

In a possible implementation, determining the second time based on the third data identifier and the corresponding third time includes: determining the time before the third time as the second time based on the order of multiple times corresponding to multiple data identifiers of the master node in the storage device.

In this implementation, the time before the third time is determined to be the second time, and the determined second time is more accurate.

In one possible implementation, the data identifier includes a version identifier; when the data identifier is a version identifier, the second time includes: the time before the first version identifier of the target child node is less than the earliest time among multiple times corresponding to multiple version identifiers of the master node.

In this implementation, a second time when the data identifier is a version identifier is defined.

In one possible implementation, the data identifier includes a location identifier; when the data identifier is a location identifier, the second time includes: the time before the first location identifier of the target child node is less than the earliest time among multiple times corresponding to multiple version identifiers of the master node.

In this implementation, a second time is defined when the data identifier is a location identifier.

In a possible implementation, the method further includes: storing the time difference in a storage device so that a data detection system analyzes the consistency of business data of the master node and the target sub-node.

In this implementation, the time difference is stored in a storage device, which enables the data detection system to analyze the consistency of business data of the master node and the target sub-node based on the stored time difference.

In the second aspect, a data consistency detection device is provided, which is applied to a data detection system and can be a chip or system on chip of the data detection system. The data detection system is used to perform consistency detection on the business data of multiple nodes of a distributed system, and the multiple nodes of the distributed system include a master node and at least one child node; the data detection system includes a storage device, and the storage device stores data identifiers and corresponding times of the master node and at least one child node, and the data identifier is used to indicate the update operation of the business data of the master node and at least one child node; the data processing device can implement the functions performed by the data detection system in the above-mentioned first aspect or the possible design of the first aspect, and the functions can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions. For example, the device may include:

An acquisition module is used to acquire multiple data identifiers of a master node in a storage device and multiple times corresponding to the multiple data identifiers of the master node, as well as a first time of a target subnode and a first data identifier corresponding to the first time; a determination module is used to determine a second time based on the first data identifier, multiple data identifiers of the master node and multiple times corresponding to the multiple data identifiers of the master node; wherein the second time is the time before the earliest time among the multiple times of the master node when the business data corresponding to the first data identifier is covered by the business data corresponding to the multiple data identifiers of the master node; the determination module is also used to determine the time difference between the first time and the second time, and the time difference is used to characterize the consistency of the business data of the master node and the target subnode.

In one possible implementation, the determination module is specifically used to: respectively compare the first data identifier of the target subnode with multiple data identifiers of the main node to determine multiple target data identifiers of the main node whose business data corresponding to the first data identifier is covered by the business data corresponding to the multiple data identifiers of the main node; determine a third data identifier from the multiple target data identifiers; wherein the time corresponding to the third data identifier is the third time; and determine the second time based on the third data identifier and the corresponding third time.

In a possible implementation, the determination module is specifically used to: determine the target data identifier in the first order among the multiple target data identifiers as the third data identifier according to the order of multiple target times corresponding to the multiple target data identifiers of the master node in the storage device.

In a possible implementation, the determination module is specifically used to determine the time before the third time as the second time according to the order of multiple times corresponding to multiple data identifiers of the master node in the storage device.

In one possible implementation, the data identifier includes a version identifier; when the data identifier is a version identifier, the second time includes: the time before the first version identifier of the target child node is less than the earliest time among multiple times corresponding to multiple version identifiers of the master node.

In one possible implementation, the data identifier includes a location identifier; when the data identifier is a location identifier, the second time includes: the time before the first location identifier of the target child node is less than the earliest time among multiple times corresponding to multiple version identifiers of the master node.

In a possible implementation, the apparatus further includes a storage module, which is used to store the time difference in a storage device so that the data detection system analyzes the consistency of business data of the master node and the target sub-node.

In a third aspect, the present application provides a computing device cluster, wherein the computing device cluster includes at least one computing device, each of the at least one computing device includes at least one processor and at least one memory, and the at least one memory stores computer-readable instructions; the at least one processor executes the computer-readable instructions so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect.

In a fourth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computing device cluster, causes the computing device cluster to execute the method described in the first aspect or any possible implementation of the first aspect.

In a fifth aspect, the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect.

Among them, the beneficial effects described in the second to fifth aspects of the present application can refer to the analysis of the beneficial effects of the first aspect, and will not be repeated here.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a schematic diagram of the structure of a distributed cluster provided in an embodiment of the present application;

FIG2 is a schematic diagram of data fields of a master node and a sub-node provided in an embodiment of the present application;

FIG3 is a schematic diagram of location information of a master node and a sub-node provided in an embodiment of the present application;

FIG4 is a schematic diagram of the structure of a data detection system provided in an embodiment of the present application;

FIG5 is a schematic diagram of a flow chart of a data processing method provided in an embodiment of the present application;

FIG6 is a flow chart of another data processing method provided in an embodiment of the present application;

FIG7 is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

FIG8 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

FIG9 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application;

FIG. 10 is a schematic diagram of the structure of another computing device cluster provided in an embodiment of the present application.

DETAILED DESCRIPTION

The network architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. A person of ordinary skill in the art can appreciate that with the evolution of the network architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

It should be understood that in the embodiments of the present application, "at least one (item)" refers to one or more, "multiple" refers to two or more, "at least two (items)" refers to two or three and more than three, and "and/or" is used to describe the association relationship of the associated objects, indicating that there can be three relationships. For example, "A and/or B" can represent: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character "/" generally indicates that the associated objects before and after are in an "or" relationship. "At least one of the following (items)" or similar expressions refers to any combination of these items, including any combination of single items (items) or plural items (items). For example, at least one of a, b or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple. It should be understood that in the embodiments of the present application, "B corresponding to A" means that B is associated with A. For example, B can be determined based on A. It should also be understood that determining B based on A does not mean determining B based solely on A. B can also be determined based on A and/or other information.

It should be noted that the terms "first" and "second" in the specification, claims and drawings of the present application are used to distinguish different objects rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units that are inherent to these processes, methods, products or devices.

Before introducing the embodiments of the present application, some terms involved in the embodiments of the present application are explained.

High availability (HA): The ability of a system to execute logic without interruption. Generally, a higher HA means a more stable and robust system.

Replication: Multiple copies of data storage to ensure that even if one backup fails, other backups can still be used to provide services to users.

Consistency: refers to the fact that in a distributed system, the values of data in multiple nodes are the same.

Availability: The probability that a system can operate normally at a certain time or the expected value of the time occupancy rate.

TP99: refers to the minimum time required to satisfy 99% of network requests. For example, TP99 = 10ms, which means that 99% of network requests during this period are within 10 milliseconds. This means that among 100 users, 99 users have a time limit of 10ms.

Relational database management system (RDBMS): A relational database stores data in different tables instead of putting all data in one large warehouse, which increases speed and flexibility. For example, MySQL (my structured query language), PostgreSQL (postgre structured query language), etc.

Remote dictionary server (Redis): is an open source log-based database written in the American National Standards Institute (ANSI) C language.

Binary log: It is a binary file (also called log) in MySQL, which is used to record the SQL statement information of the user to update the database. For example, SQL statements for changing database tables and changing contents will be recorded in Bin log, but queries on library tables and other contents will not be recorded.

A distributed cluster (also known as a distributed system) can be a group of interconnected computers or servers that work together to perform common tasks. Computers or servers in a distributed cluster (each computer or server can be called a node, and different nodes are divided into master nodes and sub-nodes according to the different tasks performed by the nodes) are usually located in different physical locations and connected to each other through a network. Distributed clusters are often used to provide high availability, scalability, and fault tolerance for applications and services.

In addition to the distributed clusters described above, the distributed clusters in the embodiments of the present application may also be cloud servers, such as public cloud servers, private cloud servers, hybrid cloud servers, etc.; although a cloud server is not a traditional distributed cluster (because a cloud server is not a group of interconnected physical servers), it can be considered a virtual distributed cluster. This is because cloud servers are usually hosted on a hypervisor, which is a virtualization platform that allows multiple virtual machines to run on a single physical server. Each virtual machine is equivalent to a node, and multiple nodes constitute a distributed cluster.

A distributed cluster can be shown as in Figure 1. In this example, the distributed cluster includes a master node and 6 child nodes (child node 11, child node 12, child node 13, child node 14, child node 15, and child node 16). The embodiment of the present application does not limit the number of nodes in the distributed cluster. In the multi-node and multi-copy mode of the distributed cluster, when the data in the master node is updated, it will be synchronized to the child node in a push or pull manner. Among them, push: usually refers to the master node sending new data to the child node that needs to be synchronized. Pull: usually refers to the child node periodically initiating a request to the master node to obtain the latest data, and the master node responds to the request and sends data to the child node. Specifically, after the master node receives the data write request, it will write the data to the disk and record the data update operation in the binlog. When the binlog changes, the changed data will be pushed to the child node; after the child node receives the change notification of the binlog, it pulls the updated data and writes it to the local file to complete the data synchronization.

Since the network conditions of different child nodes and their positional relationships with the master node are different, the progress of data synchronization is also different. For example, one child node may have synchronized all the data, but another child node has only synchronized half of the data. At this time, the data of each node in the distributed cluster is not consistent, which affects the effective application of the multi-node and multi-copy mode of the distributed cluster. Therefore, it is necessary to detect the data consistency between the master node and the child nodes in the distributed cluster.

When testing the data consistency between the master node and the child node in a distributed cluster, if the data in the master node and the child node business has the characteristic of monotonic escape order, the values of the data in the master node and the child node can be queried, and the values of the data between the master node and a child node can be compared through business logic to see if they are consistent. For example, as shown in Figure 2, a field in the data (such as status) has a monotonic escape order (Queue->Pending->Running->Complete, representing four different states of the field respectively). If the state of the data in the child node 21 is Queue, and the state of the same data in the master node 22 is Running, it is considered that the data of the child node 21 and the master node 22 are inconsistent, and the child node 21 lags behind the master node 22 by two states (also known as 2 versions behind).

However, this method requires that the data in the main node and the sub-node business have monotonic escape characteristics, and its versatility is poor. In addition, this method can only reflect the version difference of the data between the sub-node and the main node, and cannot reflect the data delay between the sub-node and the main node. Since the flow time of different fields is not necessarily equal, it is impossible to measure the difference in data consistency of different fields under the same dimension, and thus it is impossible to measure the global data consistency between different sub-nodes and the main node. For example, the escape order of field A is (Queue->Pending), and the transfer order of field B is (α->β). The time for field A to change from Queue to Pending is usually 1 minute, while the time for field B to change from α to β is usually 10 minutes, that is, the flow time of field A and field B is different. Assuming that the data in sub-node A is field A and the data in sub-node B is field B, by comparing the values of the data of the two sub-nodes (sub-node A and sub-node B) with the data of the main node, it is determined that sub-node A lags behind the main node by two versions, and sub-node B lags behind the main node by one version. Since the transfer time of field A and field B is different (the transfer time of field B is longer than that of field A), and the longer the time, the larger the data update, the data delay of sub-node B lagging behind the master node by one version is even greater than the data delay of sub-node A lagging behind the master node by two versions. In other words, it is impossible to compare the data consistency between different sub-nodes based on the version difference relative to the master node.

In order to avoid the situation that the data of the master node and the child node do not have a monotonic escape sequence and cannot be detected for consistency, it is also possible to determine whether the data is consistent by querying the data location information of each node and comparing the data location information of the master node and each child node (used to characterize the amount of data in the node, for example, Bin log in MySQL; pg_current_xlog_location in PostgreSQL; master_repl_offset representing the current replication offset of the node and slave_repl_offset representing the replication offset of a replica in the execution result of the INFO command in Redis, referred to as location information or location identification). According to the difference in location information, the data location gap between the master node and a child node and the amount of data difference can be simply calculated. For example, as shown in Figure 3, the location information of the master node 30 is 1000, while the location information of the child node 31 is 950, and the location information of the child node 32 is 970. Based on the location information, the amount of data difference between subnode 31 and subnode 32 and master node 30 can be estimated. For example, the difference between location information 950 and location information 1000 may be 30GB of data, and the difference between location information 970 and location information 1000 may be 10GB of data.

However, this method is to calculate the data consistency index between the master node and the child nodes by measuring the difference in location information. The update time intervals of the location information of different child nodes are not necessarily equal. For example, one child node is updated every 10 minutes, and another child node is updated every 5 minutes. In this case, due to the inconsistency of the update intervals, the amount of data written corresponding to the location information of different child nodes varies greatly. Comparing the difference in location information cannot reflect the difference in data delay between the master node and different child nodes.

In order to solve the above problems, an embodiment of the present application provides a data processing method, which stores the data identifiers and corresponding times of the master node and the child node through a storage device, and determines the latest time in the data identifier of the master node that can make the child node completely contain the data identifier of the master node based on the data identifier of the child node at an arbitrarily selected time. The time difference between the above arbitrarily selected time and the time corresponding to the data identifier found in the master node is the data delay information of the child node. The method can calculate the data consistency index between each child node and the master node in the distributed system, has strong versatility, and can measure the global data consistency between different child nodes and the master node.

The trigger mechanism of the data processing method provided in the embodiment of the present application can be flexibly set, for example, triggered by user events (user operation data detection system), triggered at time intervals (for example, every 5 minutes/time, users can make corresponding adjustments according to their own needs, faster frequencies provide more accurate data, but relatively consume more resources; lower frequencies provide coarser-grained data, but relatively save more resources), receiving execution instructions from electronic devices to instruct execution of detection, etc., which are not limited by the embodiments of the present application.

The method provided in the embodiment of the present application is described below in conjunction with the accompanying drawings.

The data processing method provided in the embodiment of the present application is applied to a data detection system. As shown in FIG4 , the data detection system is used to perform consistency detection on the business data of multiple nodes of a distributed system. Specifically, the data detection system first stores the data identifiers and corresponding times of the master node and the child node through a storage device, and then performs consistency detection on the business data of multiple nodes of the distributed system based on the stored data. The specific detection process is detailed in the introduction of the data processing method in the embodiment of the present application below.

FIG5 shows a schematic flow chart of a data processing method provided in an embodiment of the present application. As shown in FIG5 , the method may include the following steps:

Step S510, obtaining multiple data identifiers of a master node in a storage device and multiple times corresponding to the multiple data identifiers of the master node, as well as a first time of a target child node and a first data identifier corresponding to the first time.

Among them, the storage device stores the data identifiers and corresponding times of the main node and at least one child node in the distributed cluster. When the storage device stores the data identifiers of each node, it stores them in chronological order. The storage device can use electronic devices with high-flow and low-latency storage functions. For example, cloud storage devices or database servers, etc. When storing the above data, the storage device will store the above data in the same format. The target child node can be any one of the at least one child node. The first time can be any one of the multiple times of the target child node stored in the storage device. The time corresponding to the data identifier can refer to the time when the data identifier is generated. When the main node updates the business data, it generates the corresponding data identifier and time, and synchronizes the updated business data to the child node. When the child node receives the business data of the main node, it also records the data identifier corresponding to the business data of the main node at the current time.

Exemplarily, when the master node updates business data A, it generates data identifier 01 and corresponding time 15. The master node synchronizes business data A to the child node. The corresponding child node receives business data B from the master node (due to factors such as transmission delays, business data A and business data B are generally different at the same time). When the child node receives business data B, the corresponding data identifier 02 and corresponding time 15 are generated.

When updating business data, the master node and the child node can generate data identifiers and corresponding times according to different time granularities, for example, once every 15 seconds. In this case, the mapping relationship between the data identifier and the time can be shown in Table 1.

Table 1

time Data Identification 0 95 15 100 30 110 45 130

The data identifier is a type of quantifiable information that can indicate the update operation of the business data of the main node and the child node. As the degree of completion of the update operation of the business data increases, the scalar representing the data identifier increases accordingly. The update operation of the business data can be to add new business data, delete business data, or change business data. Exemplarily, the data identifier may include a version identifier, location information, etc. of the business data. The embodiments of the present application do not limit the specific composition of the data identifier.

Step S520, determining the second time according to the first data identifier, multiple data identifiers of the master node, and multiple times corresponding to the multiple data identifiers of the master node.

Among them, the first time can be any one of the multiple times of the target subnode stored in the storage device, and the data identifier of the target subnode corresponding to the first time is the first data identifier. The second time is the time before the earliest time among the multiple times of the master node at which the business data corresponding to the first data identifier is covered by the business data corresponding to the multiple data identifiers of the master node. The business data corresponding to the first data identifier is covered by the business data corresponding to the multiple data identifiers of the master node, which means that the degree of completion of the update operation of the business data indicated by the first data identifier is less than the degree of completion of the update operation of the business data indicated by the multiple data identifiers of the master node. For the sake of concise description, "business data corresponding to XX data identifier covers business data corresponding to YY data identifier" is referred to as "XX data identifier covers YY data identifier" in the following.

Exemplarily, when the data identifier is a version identifier, the second time may include: the previous time when the first version identifier of the target child node is smaller than the earliest time among the multiple times corresponding to the multiple version identifiers of the main node; when the data identifier is a position identifier, the second time may include: the previous time when the first position identifier of the target child node is smaller than the earliest time among the multiple times corresponding to the multiple version identifiers of the main node.

In a specific example, the data identifier is a location identifier, and the master node (and the target subnode) generates the data identifier and the corresponding time at 15s/time. As shown in Table 2, Table 2 shows a mapping table of a master node's location identifier-time and a mapping table of a target subnode's location identifier-time.

Table 2

In the example shown in Table 2, when the first time is selected as 45, the corresponding first data identifier is 109. It can be seen that the first data identifier 109 is covered by multiple data identifiers of the master node (position identifier 110, position identifier 130). The multiple times corresponding to the multiple data identifiers (position identifier 110, position identifier 130) are time 30 and time 45. The earliest time 30 between time 30 and time 45 is determined as the third time, and the previous time of position identifier 30 is time 15. According to the above introduction to the second time, it can be determined that the second time in this example is 15.

In another specific example, the data identifier is a version identifier, and the master node (and the target subnode) generates the data identifier and the corresponding time at 15s/time. As shown in Table 3, Table 3 shows a mapping table of version identifier-time of a master node and a mapping table of version identifier-time of a target subnode.

Table 3

In the example shown in Table 3, when the first time is selected as 45, the corresponding first data identifier is 1.09. It can be seen that the first data identifier 1.09 is covered by multiple data identifiers of the master node (version identifier 1.10, version identifier 1.15). The multiple times corresponding to the multiple data identifiers (version identifier 1.10, version identifier 1.15) are time 45 and time 60, among which the earliest time is time 45, which is determined to be the third time. The previous time of the third time 45 is time 30. According to the above introduction to the second time, it can be determined that the second time in this example is 30.

Step S530, determining the time difference between the first time and the second time.

After determining the first time and the second time, the time difference between the two can be determined. The time difference is used to characterize the consistency of the business data of the master node and the target sub-node. The calculation of the time difference can be expressed as:

Consistent＝T _current -T _master

Wherein: Consistent represents the time difference, T _current represents the first time, and T _master represents the second time.

Exemplarily, as shown in Table 2, the first time is 45, the second time is 15, and the time difference between the first time and the second time is 45-15=30.

In one embodiment, the time difference determined in step S530 may be stored in a storage device. After the time difference is stored in the storage device, the business data consistency of the master node and the target subnode may be analyzed. The storage device is capable of storing multiple time differences between the target subnode and the master node determined in multiple time periods, and the multiple time differences may be used in multiple application scenarios, such as historical data backtracking, change trend analysis of the business data consistency between the target subnode and the master node, etc.

In the embodiment of the present application, a method for calculating the time difference that characterizes the consistency of the business data of the master node and the target sub-node is defined. The data in the business of the master node and the sub-node are not required to have the characteristics of monotonic transfer order, and the versatility is strong. The multiple time differences calculated by different sub-nodes and the master node can be compared, and then the global data consistency between different sub-nodes and the master node can be measured.

In one embodiment, as shown in FIG. 6 , when executing step S520, the second time may be determined by specifically performing the following steps S521 to S523:

Step S521 : Compare the first data identifier of the target child node with the multiple data identifiers of the main node respectively to determine multiple target data identifiers.

Among them, the multiple target data identifiers are data identifiers that cover the first data identifier among the multiple data identifiers of the master node. In the example shown in Table 2, the multiple target data identifiers are (data identifier 110 and data identifier 130). When comparing the first data identifier with the multiple data identifiers of the master node to determine the multiple target data identifiers, they can be compared in sequence according to the time sequence corresponding to the multiple data identifiers. Exemplarily, as shown in Table 2, the multiple data identifiers of the master node are in chronological order: data identifier 95, data identifier 100, data identifier 110 and data identifier 130, and the first data identifier is 109. By comparing 109 with data identifier 95, data identifier 100, data identifier 110 and data identifier 130 in sequence, it can be determined that the data identifier of the master node (i.e., multiple target data identifiers) whose business data corresponding to the first data identifier is covered by the business data corresponding to the multiple data identifiers of the master node, including data identifier 110 and data identifier 130.

In another example, the first data identifier of the target child node can also be compared with the multiple data identifiers of the main node according to the reverse time order corresponding to the multiple data identifiers, and the efficiency of the comparison in reverse time order is higher. Exemplarily, as shown in Table 2, the multiple data identifiers of the main node are in reverse time order: data identifier 130, data identifier 110, data identifier 100 and data identifier 95. The first data identifier is 109, and 109 is compared with data identifier 130, data identifier 110 and data identifier 100 in turn. Since data identifier 100 cannot cover the first data identifier 109, the comparison can be stopped when data identifier 100 is compared. It can be determined that the data identifier of the main node whose business data corresponding to the first data identifier is covered by the business data corresponding to the multiple data identifiers of the main node, that is, multiple target data identifiers, including data identifier 110 and data identifier 130.

In another example, there may be a situation where there are a large number of multiple data identifiers of the main node. At this time, it is inefficient to compare one by one in chronological order or in reverse chronological order. The binary search method can be applied to determine multiple target data identifiers to improve the determination efficiency. When applying the binary search method to determine multiple target data identifiers, the data identifier at the middle position of the multiple data identifiers of the main node is compared with the first data identifier. If the two are the same, the data identifier after the middle position of the main node is determined as the target data identifier; otherwise, the multiple data identifiers of the main node are divided into the front and rear parts using the middle position. If the data identifier recorded in the middle position covers the first data identifier, the data identifier in the rear part is determined as the target data identifier, and the target data identifier in the data identifier in the front part is further searched; if the data identifier recorded in the middle position does not cover the first data identifier, the target data identifier is obtained from the difference of the data identifier in the rear part. Repeat the above process until all target data identifiers are determined. This determination method has a high determination efficiency.

The above describes several examples of determining multiple target data identifiers. When determining multiple target data identifiers, other methods may also be used, which are not limited in the embodiments of the present application.

Step S522: determine a third data identifier from multiple target data identifiers.

Wherein, when determining the third data identifier, the target data identifier in the first order among the multiple target data identifiers can be determined as the third data identifier according to the order of the multiple target times corresponding to the multiple target data identifiers of the master node in the storage device, and the time corresponding to the third data identifier is the third time. Exemplarily, as shown in Table 4, the target data identifier includes the target data identifier 110 and the target data identifier 130, and the target data identifier 110 is in the first order, and the third data identifier is the target data identifier 110.

Table 4

Step S523: determine the second time according to the third data identifier and the corresponding third time.

When determining the second time, the time before the third time can be determined as the second time according to the order of multiple times corresponding to multiple data identifiers of the master node in the storage device.

The above describes the process of determining the second time. The principle of determining the second time can be expressed by the formula:

T _master =max[T(Position _master ≤Position _slave )]

Wherein: T _master represents the second time, T represents time, max[……] represents the maximum value, Position _master represents the data identifier of the master node, and Position _slave represents the first data identifier of the target child node.

Exemplarily, as shown in Table 4, the third time is 30, and the time before 30 is time 15, so the second time in this example is time 15.

When introducing the application of binary search method to determine multiple target data identifiers, it is explained that there will be a situation where the data identifier of the main node is the same as the first data identifier of the target child node (the data identifier of the main node in this situation is called the fourth data identifier). Whether there is such a situation depends mainly on the time granularity of the main node and the child node to generate the data identifier. The smaller the time granularity, the greater the probability of the situation where the data identifier of the main node is the same as the first data identifier of the target child node. In this case, refer to the description of "coverage" in step S520 in the embodiment of the present application: the business data corresponding to the first data identifier is covered by the business data corresponding to the multiple data identifiers of the main node, which means that the completion degree of the update operation of the business data indicated by the first data identifier is less than the completion degree of the update operation of the business data indicated by the multiple data identifiers of the main node. It can be seen that since the completion degree of the update operation of the business data indicated by the first data identifier is not less than (but equal to) the completion degree of the update operation of the business data indicated by the fourth data identifier, the fourth data identifier does not cover the first data identifier. Among the multiple data identifiers of the main node, the next data identifier of the fourth data identifier is the third data, and determining the time before the third time as the second time means that the time corresponding to the fourth data identifier is determined as the second time.

In an embodiment of the present application, the first data identifier of the target subnode is compared with multiple data identifiers of the main node to determine the third time when the data identifier in the main node earliest covers the first data identifier of the target subnode, and then the second time is determined based on the third time, and the determined second time is more accurate.

In one embodiment, multiple sub-nodes of a distributed cluster determine the time difference according to the method described in the embodiment of the present application, and store it in a storage device. The data detection system can analyze the time difference of multiple sub-nodes to determine whether the operation of the distributed cluster is normal. For example, the mean of the time difference of multiple sub-nodes can be calculated, and the calculated mean can be compared with the expected value of the normal operation of the distributed cluster to determine whether the operation of the distributed cluster is normal. For example, the expected value of the normal operation of the distributed cluster can be set to 30s. If the mean of the time difference of multiple sub-nodes is less than 30s, the operation of the distributed cluster is considered normal, otherwise it is considered that the operation of the distributed cluster is abnormal. At this time, the distributed cluster can be troubleshooted and optimized.

In another example, the TP99 algorithm can also be applied to analyze the time difference of multiple child nodes. The TP99 algorithm can pre-set a baseline value, which is used to determine whether the operation of the distributed cluster is normal. For example, the baseline value can be set to 30s. Under this setting, the TP99 algorithm requires that the time difference between 99% of the child nodes and the master node is within 30s. In other words, it is required that the time difference of 99 child nodes out of 100 child nodes cannot be greater than 30s.

The above introduces two algorithms for analyzing the operating status of a distributed cluster based on the time difference of multiple sub-nodes. When analyzing the operating status of a distributed cluster, other algorithms can also be used, which is not limited by the embodiments of the present application.

In the embodiment of the present application, the operation status of the distributed cluster is analyzed according to the time difference of multiple sub-nodes, so that the health status of the distributed cluster can be monitored.

In one embodiment, the time difference corresponding to the target sub-node may also be displayed for the user to view when needed.

In one embodiment, when the time difference corresponding to the target sub-node meets the preset alarm condition, alarm information may be generated and displayed.

Exemplarily, the preset alarm condition may be that the time difference exceeding a preset proportion among multiple time differences determined by the target sub-node in different periods exceeds a preset threshold. Alternatively, the preset alarm condition may also be that the average value of multiple time differences corresponding to the target sub-node exceeds the preset threshold. The preset alarm condition may be flexibly set, and the embodiments of the present application are not limited thereto. The alarm information may also be flexibly set according to user needs. Exemplarily, the alarm information may be "network abnormality"; the alarm information may also directly indicate which sub-nodes have network abnormalities, such as "XX sub-node network abnormality".

The embodiment of the present application generates an alarm message when the time difference corresponding to the target sub-node meets the preset alarm condition, and displays the alarm message, so as to promptly remind the user of the problems existing in the network.

In summary, the present application provides a data processing method, which stores the data identifiers and corresponding times of the master node and the child node through a storage device. Based on the data stored in the above storage device, according to the data identifier of the child node corresponding to the arbitrarily selected time, the latest time among the multiple times corresponding to the multiple data identifiers in the data identifier of the master node that can be completely contained in the data identifier of the arbitrarily selected child node is determined, and the time difference between the arbitrarily selected time and the latest time represents the data consistency between the child node and the master node. Based on this, the index of data consistency between each child node and the master node in the distributed system can be calculated, which has strong versatility and can measure the global data consistency between different child nodes and the master node.

The present application also provides a data processing device, as shown in FIG7 , the data processing device 800 includes:

The acquisition module 810 is used to acquire multiple data identifiers of the master node and multiple times corresponding to the multiple data identifiers of the master node in the storage device, as well as the first time of the target child node and the first data identifier corresponding to the first time.

The determination module 820 is used to determine a second time according to the first data identifier, multiple data identifiers of the master node, and multiple times corresponding to the multiple data identifiers of the master node, wherein the second time is a time before the earliest time among the multiple times of the master node at which the business data corresponding to the first data identifier is covered by the business data corresponding to the multiple data identifiers of the master node.

The determination module 820 is further used to determine the time difference between the first time and the second time, where the time difference is used to characterize the consistency of the business data of the master node and the target sub-node.

In one embodiment, the determination module 820 is specifically used to: respectively compare the first data identifier of the target subnode with the multiple data identifiers of the master node to determine the multiple target data identifiers of the master node whose business data corresponding to the first data identifier is covered by the business data corresponding to the multiple data identifiers of the master node. Determine a third data identifier from the multiple target data identifiers. The time corresponding to the third data identifier is the third time. Determine the second time based on the third data identifier and the corresponding third time.

In one embodiment, the determination module 820 is specifically used to: determine the target data identifier in the first order among the multiple target data identifiers as the third data identifier according to the order of multiple target times corresponding to the multiple target data identifiers of the master node in the storage device.

In one embodiment, the determination module 820 is specifically used to determine the previous time of the third time as the second time according to the order of multiple times corresponding to multiple data identifiers of the master node in the storage device.

In one embodiment, the data identifier includes a version identifier. When the data identifier is a version identifier, the second time includes: a time before the first version identifier of the target child node is smaller than the earliest time among multiple times corresponding to multiple version identifiers of the master node.

In one embodiment, the data identifier includes a location identifier. When the data identifier is a location identifier, the second time includes: a time before the first location identifier of the target child node is smaller than the earliest time among multiple times corresponding to multiple version identifiers of the master node.

In one embodiment, the apparatus further includes a storage module 830, and the storage module 830 is used to store the time difference in a storage device so that the data detection system analyzes the consistency of the business data of the master node and the target sub-node.

Among them, each module described above can be implemented by software or by hardware. Exemplarily, the following takes the acquisition module 810 as an example to introduce the implementation of the acquisition module 810. Similarly, the implementation of the determination module 820 and the storage module 830 can refer to the implementation of the acquisition module 810.

As an example of a software functional unit, the acquisition module 810 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the acquisition module 810 may include code running on multiple hosts/virtual machines/containers. It should be noted that the multiple hosts/virtual machines/containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts/virtual machines/containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with close geographical locations. Among them, usually a region may include multiple AZs.

Similarly, multiple hosts/virtual machines/containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

As an example of a hardware functional unit, the acquisition module 810 may include at least one computing device, such as a server, etc. Alternatively, the acquisition module 810 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

The multiple computing devices included in the acquisition module 810 can be distributed in the same region or in different regions. The multiple computing devices included in the acquisition module 810 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the acquisition module 810 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

It should be noted that, in other embodiments, the acquisition module 810 can be used to execute any step in the data processing method, the determination module 820 can be used to execute any step in the data processing method, and the storage module 830 can be used to execute any step in the data processing method. The steps that the acquisition module 810, the determination module 820, and the storage module 830 are responsible for implementing can be specified as needed. The full functions of the data processing device are realized by respectively implementing different steps in the data processing method through the acquisition module 810, the determination module 820, and the storage module 830.

The present application also provides a computing device 100. As shown in FIG8 , the computing device 100 includes: a bus 102, a processor 104, a memory 106, and a communication interface 108. The processor 104, the memory 106, and the communication interface 108 communicate with each other through the bus 102. The computing device 100 may be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 100.

The bus 102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, FIG8 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 104 may include a path for transmitting information between various components of the computing device 100 (e.g., the memory 106, the processor 104, and the communication interface 108).

The processor 104 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

The memory 106 may include a volatile memory, such as a random access memory (RAM). The processor 104 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

The memory 106 stores executable program codes, and the processor 104 executes the executable program codes to respectively implement the functions of the acquisition module 810, the determination module 820 and the storage module 830, thereby implementing the data processing method. That is, the memory 106 stores instructions for executing the data processing method.

Alternatively, the memory 106 stores executable codes, and the processor 104 executes the executable codes to respectively implement the functions of the aforementioned data processing apparatus, thereby implementing the data processing method. That is, the memory 106 stores instructions for executing the data processing method.

The communication interface 103 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

As shown in Fig. 9, the computing device cluster includes at least one computing device 100. The memory 106 in one or more computing devices 100 in the computing device cluster may store the same instructions for executing the data processing method.

In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the data processing method. In other words, the combination of one or more computing devices 100 may jointly execute instructions for executing the data processing method.

It should be noted that the memory 106 in different computing devices 100 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the data processing apparatus. That is, the instructions stored in the memory 106 in different computing devices 100 may implement the functions of one or more of the acquisition module 810, the determination module 820 and the storage module 830.

In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network. The network may be a wide area network or a local area network, etc. FIG. 10 shows a possible implementation. As shown in FIG. 10 , two computing devices 100A and 100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 106 in the computing device 100A stores instructions for executing the functions of the acquisition module 810. At the same time, the memory 106 in the computing device 100B stores instructions for executing the functions of the determination module 820 and the storage module 830.

The connection method between the computing device clusters shown in Figure 10 may be based on the needs of the data processing method provided in this application, so it is considered to hand over the functions implemented by the determination module 820 and the storage module 830 to the computing device 100B for execution.

It should be understood that the functions of the computing device 100A shown in FIG10 may also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B may also be completed by multiple computing devices 100.

The embodiment of the present application also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to the connection mode of the computing device cluster described in Figures 9 and 10. The difference is that the memory 106 in one or more computing devices 100 in the computing device cluster can store the same instructions for executing the data processing method.

In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the data processing method. In other words, the combination of one or more computing devices 100 may jointly execute instructions for executing the data processing method.

It should be noted that the memory 106 in different computing devices 100 in the computing device cluster may store different instructions for executing partial functions of the data detection system. That is, the instructions stored in the memory 106 in different computing devices 100 may implement the functions of the data processing apparatus.

The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes a data processing method or a data processing method.

The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct a computing device to execute a data processing method, or instructs a computing device to execute a data processing method.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims (15)

1. A data processing method, wherein the method is applied to a data detection system, the data detection system including a storage device, the data detection system being configured to perform consistency detection on service data of a plurality of nodes of a distributed system, the plurality of nodes of the distributed system including a master node and at least one child node, the method comprising:

acquiring a plurality of data identifiers of the master node and a plurality of times corresponding to the plurality of data identifiers of the master node in the storage device, and a first time of a target child node and a first data identifier corresponding to the first time; the storage equipment stores data identifiers of the main node and the at least one child node and corresponding time, wherein the data identifiers are used for indicating updating operation of service data of the main node and the at least one child node;

Determining a second time according to the first data identifier, the plurality of data identifiers of the master node and a plurality of times corresponding to the plurality of data identifiers of the master node; wherein the second time is the time before the earliest time in a plurality of times of the master node, in which the service data corresponding to the first data identifier is covered by the service data corresponding to the plurality of data identifiers of the master node;

and determining a time difference between the first time and the second time, wherein the time difference is used for representing the service data consistency of the main node and the target child node.

2. The method of claim 1, wherein the determining the second time based on the first data identifier, the plurality of data identifiers of the master node, and the plurality of times corresponding to the plurality of data identifiers of the master node comprises:

comparing the first data identifier of the target child node with the plurality of data identifiers of the master node respectively to determine a plurality of target data identifiers of the master node, which are covered by the service data corresponding to the first data identifier and the service data corresponding to the plurality of data identifiers of the master node;

determining a third data identifier from the plurality of target data identifiers; the time corresponding to the third data identifier is a third time;

And determining a second time according to the third data identifier and the corresponding third time.

3. The method of claim 2, wherein said determining a third data identity from said plurality of target data identities comprises:

And determining the target data identifier positioned at the first order in the plurality of target data identifiers as a third data identifier according to the ordering of a plurality of target times corresponding to the plurality of target data identifiers of the master node in the storage device.

4. A method according to claim 2 or 3, wherein said determining a second time from said third data identity and corresponding said third time comprises:

And determining the previous time of the third time as the second time according to the ordering of a plurality of times corresponding to a plurality of data identifiers of the master node in the storage device.

5. The method of any one of claims 1 to 4, wherein the data identification comprises a version identification or a location identification;

When the data identifier is the version identifier, the second time includes: the first version identification of the target child node is smaller than the time before the earliest time in a plurality of times corresponding to the version identifications of the master node;

When the data identifier is the location identifier, the second time includes: the first location identifier of the target child node is less than a time immediately before an earliest time of a plurality of times corresponding to the plurality of version identifiers of the master node.

6. The method according to any one of claims 1 to 5, further comprising:

And storing the time difference to the storage device so that the data detection system analyzes the consistency of the business data of the main node and the target child node.

7. A data processing apparatus, the apparatus being applied to a data detection system, the data detection system comprising a storage device, the data detection system configured to perform consistency detection on traffic data of a plurality of nodes of a distributed system, the plurality of nodes of the distributed system comprising a master node and at least one child node, the apparatus comprising:

The acquisition module is used for acquiring a plurality of data identifiers of the master node and a plurality of times corresponding to the plurality of data identifiers of the master node in the storage device, and a first time of a target child node and a first data identifier corresponding to the first time; the storage equipment stores data identifiers of the main node and the at least one child node and corresponding time, wherein the data identifiers are used for indicating updating operation of service data of the main node and the at least one child node;

the determining module is used for determining a second time according to the first data identifier, the plurality of data identifiers of the master node and a plurality of times corresponding to the plurality of data identifiers of the master node; wherein the second time is the time before the earliest time in a plurality of times of the master node, in which the service data corresponding to the first data identifier is covered by the service data corresponding to the plurality of data identifiers of the master node;

the determining module is further configured to determine a time difference between the first time and the second time, where the time difference is used to characterize consistency of service data between the master node and the target child node.

8. The apparatus of claim 7, wherein the determining module is specifically configured to:

comparing the first data identifier of the target child node with the plurality of data identifiers of the master node respectively to determine a plurality of target data identifiers of the master node, which are covered by the service data corresponding to the first data identifier and the service data corresponding to the plurality of data identifiers of the master node;

determining a third data identifier from the plurality of target data identifiers; the time corresponding to the third data identifier is a third time;

And determining a second time according to the third data identifier and the corresponding third time.

9. The apparatus according to claim 8, wherein the determining module is specifically configured to:

And determining the target data identifier positioned at the first order in the plurality of target data identifiers as a third data identifier according to the ordering of a plurality of target times corresponding to the plurality of target data identifiers of the master node in the storage device.

10. The apparatus according to claim 8 or 9, wherein the determining module is specifically configured to:

And determining the previous time of the third time as the second time according to the ordering of a plurality of times corresponding to a plurality of data identifiers of the master node in the storage device.

11. The apparatus according to any of claims 7 to 10, wherein the data identification comprises a version identification or a location identification;

When the data identifier is the version identifier, the second time includes: the first version identification of the target child node is smaller than the time before the earliest time in a plurality of times corresponding to the version identifications of the master node;

When the data identifier is the location identifier, the second time includes: the first location identifier of the target child node is less than a time immediately before an earliest time of a plurality of times corresponding to the plurality of version identifiers of the master node.

12. The apparatus according to any one of claims 7 to 11, further comprising a storage module for storing the time difference to the storage device, such that the data detection system analyzes traffic data consistency of the master node and the target child node.

13. A cluster of computing devices, comprising at least one computing device, each computing device comprising a processor and a memory;

The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method of any of claims 1-6.

14. A computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method of any of claims 1-6.

15. A computer readable storage medium comprising computer program instructions which, when executed by a cluster of computing devices, perform the method of any of claims 1-6.