CN116414908A - Table data synchronization processing method, electronic equipment and medium - Google Patents

Abstract

The application provides a table data synchronization processing method, electronic equipment and media. The table data synchronization processing method includes: obtaining data synchronization log information, the data synchronization log information stores the synchronization status of different batches of table data, different batches The second table data is obtained by batch processing the target source table data. Based on the synchronization status of the table data batches in the data synchronization log information, the corresponding table data batches are synchronized, and the synchronization status is updated to the data synchronization status in real time. in the log information. The table data synchronization processing method of the present application realizes batch synchronization processing of table data, records the table data synchronization processing process through data synchronization log information, prevents the need for full data re-synchronization after table data synchronization is interrupted, and effectively reduces The time required for table data synchronization processing is shortened, and the efficiency of table data synchronization processing is improved.

Description

Table data synchronization processing method, electronic equipment and medium

technical field

The present application relates to the technical field of data processing, in particular to a table data synchronization processing method, electronic equipment and media.

Background technique

The large-scale popularization of electric vehicles has led to a sharp increase in the demand for energy supplements for electric vehicles. In this context, the battery swap mode has been widely promoted and applied. The battery swap mode can not only realize fast energy replenishment like refueling, but also realize the monitoring and management of the battery, and the resulting data is huge. For example: a monitoring mechanism is set up in the process of charging the battery at the power station, and the status data of the battery being charged is reported every once in a while. Up to now, the battery charging process table has accumulated more than 6 billion pieces of data.

In the existing process of extracting data from data sources (data synchronization) into the database, data extraction often fails due to network reasons or data itself. After data extraction fails, the traditional method can only re-extract the full amount of data , Extract the data in the data source from the database intact, which will consume a lot of time and resources to do repetitive work. The existing data synchronization processing method is relatively simple, but the task is huge, resulting in a waste of time and resources, especially for large databases with a data volume of over 100 million, and its work efficiency is relatively low.

Contents of the invention

The main purpose of the present application is to provide a table data synchronization processing method, electronic equipment and media, so as to improve the above-mentioned defects in the prior art.

The application solves the above-mentioned technical problems through the following technical solutions:

As an aspect of the present application, a table data synchronization processing method is provided, including:

Obtaining data synchronization log information, the data synchronization log information stores the synchronization status of different batches of table data, wherein the different batches of table data are obtained by batch processing the target source table data;

Based on the synchronization status of the table data batches in the data synchronization log information, the corresponding table data batches are synchronized, and the synchronization status is updated in the data synchronization log information in real time.

In the table data synchronization processing method in this application, by obtaining data synchronization log information, the synchronization status of different batches of table data is stored in the data synchronization log information, and different batches of table data are processed in batches by target source table data Afterwards, based on the synchronization status of table data batches in the data synchronization log information, the corresponding table data batches are synchronized, and the synchronization status is updated to the data synchronization log information in real time, realizing the batch synchronization processing of table data. The process of table data synchronization processing is recorded through the data synchronization log information to prevent the need for full data re-synchronization after table data synchronization is interrupted. In practical applications, the time required for table data synchronization processing is effectively reduced, and the efficiency of table data synchronization processing is improved.

Optionally, the step of synchronizing corresponding table data batches based on the synchronization status of table data batches in the data synchronization log information, and updating the synchronization status to the data synchronization log information in real time, includes :

Synchronize the table data batches whose synchronization status is unsynchronized in the data synchronization log information;

When the synchronization of the batch of table data is completed, the synchronization status of the batch of table data in the data synchronization log information is updated to be synchronized.

The table data synchronization processing method of the present application synchronizes the table data batches whose synchronization status is unsynchronized in the data synchronization log information. The synchronization status of the data batch is updated to synchronized, which realizes the synchronization processing of unsynchronized table data, provides a recording method for the data table synchronization process, and only synchronizes the unsynchronized table data batches, realizing the continuous transmission of data and improving Improve the efficiency of table data synchronization processing.

Optionally, the step of updating the synchronization status to the data synchronization log information in real time also includes:

When the batch of table data is being synchronized, the synchronization status of the batch of table data in the data synchronization log information is updated as synchronizing.

In the table data synchronization processing method of the present application, when the table data batches are being synchronized, the synchronization status of the table data batches in the data synchronization log information is updated to be synchronizing, which realizes the monitoring of the table data synchronization processing process and facilitates monitoring Personnel are informed of the progress of table data synchronization processing in a timely manner.

Optionally, the step of updating the synchronization status to the data synchronization log information in real time also includes:

When the synchronization of the table data batches is abnormal, the synchronization state of the table data batches in the data synchronization log information is updated to synchronization abnormality.

In the table data synchronization processing method of the present application, when the synchronization of table data batches is abnormal, the synchronization status of the table data batches in the data synchronization log information is updated to synchronization abnormality, and the abnormality in the table data synchronization processing process is realized. Batches are marked, and abnormal status monitoring can be realized at the same time, which is convenient for monitors and developers to locate abnormal table data batches in time, provides a basis for data repair and re-synchronization, and improves the efficiency of table data synchronization processing.

Optionally, the method also includes:

When the synchronization of the batch of table data is abnormal, stop synchronizing the table data of the current batch of abnormal occurrence, and update the synchronization status of the abnormal current batch of table data in the data synchronization log information to synchronization abnormal.

In the table data synchronization processing method of the present application, when an abnormality occurs in the synchronization of table data batches, stop synchronizing the table data of the current batch in which the abnormality occurs, and record the synchronization status of the table data in the current batch in which the abnormality occurs in the data synchronization log information It is updated to Synchronization Abnormal, which realizes the monitoring of the abnormal state of the table data synchronization process, which is convenient for the monitoring personnel to know the abnormal process of the table data synchronization process in time, and stops the data synchronization in time when the table data synchronization process is abnormal, so as to prevent abnormal data Enter the server (or data warehouse), and optimize the process of synchronous processing of table data.

Optionally, the method also includes:

In response to stopping the table data synchronization of the abnormal current batch, continue to synchronize the remaining table data batches whose synchronization status of the table data batches in the data synchronization log information is unsynchronized.

In the table data synchronization processing method of the present application, when an abnormality occurs in the synchronization of table data batches, stop synchronizing the table data of the current batch in which the abnormality occurs, and record the synchronization status of the table data in the current batch in which the abnormality occurs in the data synchronization log information Update to synchronization exception, in response to stop synchronizing the table data of the current batch where the exception occurs, continue to synchronize the data synchronization log information, the synchronization state of the table data batch is the rest of the table data batches that are not synchronized, and provide skip exception table data The batch continues to complete the method of remaining batch table data, avoiding the interruption of the synchronization process by abnormal data, improving the automation of table data synchronization processing, and improving the efficiency of table data synchronization processing.

Optionally, also include:

In response to stopping the table data synchronization of the abnormal current batch, performing abnormal repair on the abnormal table data batch;

Update the synchronization status of the batches of table data after exception repair to unsynchronized, and synchronize the batches of table data after exception repair.

In the table data synchronization processing method of the present application, when an abnormality occurs in the synchronization of table data batches, stop synchronizing the table data of the current batch in which the abnormality occurs, and record the synchronization status of the table data in the current batch in which the abnormality occurs in the data synchronization log information Update to a synchronization exception; in response to stopping the synchronization of the table data of the current batch with the exception, perform abnormal repair on the abnormal table data batch; update the synchronization status of the abnormally repaired table data batch to unsynchronized, and synchronize Batches of table data after the exception is repaired; thus, the continued transfer of table data after the exception is eliminated, the resumed table data does not include the batches that have been synchronized, optimizes the process of table data synchronization processing, and reduces the number of table data The time required for synchronous processing improves the efficiency of table data synchronous processing.

Optionally, before the step of obtaining data synchronization log information, the method further includes:

According to the preset field information, the target source table data is processed in batches to generate different batches of table data.

The table data synchronous processing method of the present application performs batch processing on the target source table data according to the preset field information to generate different batches of table data, and provides batching of the target source table data according to the characteristics of the target source table data The basis for processing ensures that the target source data will not be missed or confused during batch processing, which improves the reliability of synchronous processing of table data.

Optionally, the field information includes any one or more of time information, classification information, and sequence number information.

The table data synchronous processing method of the present application processes the target source table data in batches according to preset field information to generate different batches of table data, and the field information includes any one or more of time information, classification information, and serial number information kind. According to the characteristics of the target source table data, the basis for batch processing of the target source table data is selected to ensure that the target source data will not be missed and confused during batch processing, and the reliability of table data synchronization processing is improved. According to the characteristics of the battery monitoring data of the power station, the basis for selecting table data to divide batches can not only prevent data omission and confusion, but also ensure that the data volume of table data in different batches is similar. According to the meaning of field information, it can also help maintenance And developers clearly understand the content of the data contained in the table data batches; it can be seen that it is not only beneficial to data synchronization, but also beneficial to system maintenance and development.

Optionally, the synchronization order of table data batches is also stored in the data synchronization log information;

The step of synchronizing corresponding table data batches based on the synchronization status of table data batches in the data synchronization log information includes:

Based on the synchronization state and synchronization order of the table data batches in the data synchronization log information, the corresponding table data batches are synchronized.

The table data synchronization processing method of the present application, based on the synchronization state and synchronization order of the table data batches in the data synchronization log information, synchronizes the corresponding table data batches; thus, the orderly table data batches are realized. Synchronization, on the one hand, can prevent the omission of data in synchronization, and on the other hand, it also provides developers with the possibility to control the table data synchronization process.

Optionally, in the step of obtaining data synchronization log information,

The data synchronization log information stores synchronization states of different batches of table data of multiple target source table data classified based on table names.

In the table data synchronization processing method of the present application, the synchronization status of different batches of table data of multiple target source table data can be saved in the same data synchronization log information at the same time, which makes the method have commonality and scalability. For newly added The synchronization requirements of the target source table data can also achieve data synchronization that can be resumed from breakpoints.

As another aspect of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored on the memory and operable on the processor. When the processor executes the computer program, any of the above-mentioned Table data synchronization processing method.

As another aspect of the present application, computer instructions are stored thereon, and when the computer instructions are executed by a processor, any one of the table data synchronization processing methods described above can be realized.

According to the contents of the present application, those skilled in the art can understand other aspects of the contents of the present application.

The positive progress effect of this application is:

In the table data synchronization processing method of the present application, by obtaining data synchronization log information, the synchronization status of different batches of table data is stored in the data synchronization log information, and different batches of table data are processed in batches by target source table data Obtained, based on the synchronization status of the table data batches in the data synchronization log information, the corresponding table data batches are synchronized, and the synchronization status is updated to the data synchronization log information in real time, realizing the batch synchronization processing of the table data, through The data synchronization log information records the process of table data synchronization processing to prevent the need for full data re-synchronization after table data synchronization is interrupted. In practical applications, the time required for table data synchronization processing is effectively reduced and the efficiency of table data synchronization processing is improved.

Description of drawings

The features and advantages of the present application can be better understood after reading the detailed description of the embodiments of the present application in conjunction with the following drawings. In the drawings, components are not necessarily drawn to scale, and components with similar related properties or characteristics may have the same or similar reference numerals.

FIG. 1 is a schematic flow diagram of a method for synchronizing table data provided by Embodiment 1 of the present application;

FIG. 2 is a schematic flowchart of a table data synchronization processing method in step 102 provided in Embodiment 1 of the present application;

FIG. 3 is a schematic flowchart of a table data synchronization processing method provided in Embodiment 1 of the present application;

FIG. 4 is a schematic flowchart of a table data synchronization processing method provided in Embodiment 1 of the present application;

FIG. 5 is a schematic flowchart of a table data synchronization processing method provided in Embodiment 1 of the present application;

FIG. 6 is a schematic flowchart of a table data synchronization processing method provided in Embodiment 1 of the present application;

FIG. 7 is a schematic flowchart of a table data synchronization processing method provided in Embodiment 1 of the present application;

FIG. 8 is a schematic structural diagram of an electronic device provided in Embodiment 2 of the present application.

Detailed ways

The present application will be further described below by means of examples, but the present application is not limited to the scope of the examples.

It should be noted that references in the specification to "an embodiment," "an alternative embodiment," "another embodiment," etc. indicate that the described embodiments may include a particular feature, structure, or characteristic, but each embodiment The specific feature, structure or characteristic may not necessarily be included. Moreover, such phrases are not necessarily referring to the same embodiment. Furthermore, when a particular feature, structure or characteristic is described in conjunction with an embodiment, it is within the purview of those skilled in the relevant arts to implement such feature, structure or characteristic in conjunction with other embodiments, whether or not explicitly described.

In the description of the content of the present application, it should be understood that the terms "central", "lateral", "upper", "lower", "left", "right", "vertical", "horizontal", "top" , "bottom", "inner", "outer" and other indicated orientations or positional relationships are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the content of the application and simplifying the description, rather than indicating or implying Any device or element must have a specific orientation, be constructed and operate in a specific orientation, and therefore should not be construed as limiting the content of the application. In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, a feature defined as "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the contents of this application, unless otherwise specified, "plurality" means two or more. Additionally, the term "comprise" and any variations thereof, are intended to cover a non-exclusive inclusion.

In the description of the content of this application, it should be noted that, unless otherwise clearly specified and limited, the terms "installation", "connection", and "connection" should be understood in a broad sense, for example, it can be a fixed connection or a flexible connection. Detachable connection, or integral connection; it can be mechanical connection or electrical connection; it can be direct connection or indirect connection through an intermediary, and it can be the internal communication of two components. Those of ordinary skill in the art can understand the specific meanings of the above terms in the content of this application in specific situations.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of example embodiments. As used herein, the singular forms "a", "an" and "an" are intended to include the plural unless the context clearly dictates otherwise. It should also be understood that the terms "comprising" and/or "comprising" as used herein specify the presence of stated features, integers, steps, operations, units and/or components, but do not exclude the presence or addition of one or more Other features, integers, steps, operations, units, components and/or combinations thereof.

Example 1

This embodiment provides a table data synchronization processing method. FIG. 1 is a schematic flowchart of a table data synchronization processing method provided by this embodiment. The data synchronization processing method includes:

Step 101, acquiring data synchronization log information.

The data synchronization log information in this step stores the synchronization status of different batches of table data, wherein the table data of different batches is obtained by batch processing the target source table data.

That is, different batches of table data are obtained after the target source table data is processed in batches, and the synchronization status information of different batches of table data is stored in the data synchronization log information, and each entry in the data synchronization log information is the corresponding batch The synchronization status of the table data.

Step 102, based on the synchronization state of the table data batches in the data synchronization log information, synchronize the corresponding table data batches, and update the synchronization status to the data synchronization log information in real time.

The synchronization state in this step includes multiple states, such as an unsynchronized state, a synchronizing state, a synchronous abnormal state, and a synchronized state. According to the synchronization status of the table data batches in the data synchronization log information, the corresponding table data batches are synchronized, and the synchronization status is updated to the data synchronization log information in real time.

In the table data synchronization processing method in this embodiment, by obtaining data synchronization log information, the synchronization status of different batches of table data is stored in the data synchronization log information, and different batches of table data are obtained by batching the target source table data After processing, based on the synchronization status of table data batches in the data synchronization log information, the corresponding table data batches are synchronized, and the synchronization status is updated to the data synchronization log information in real time, realizing batch synchronization processing of table data , the process of table data synchronization processing is recorded through data synchronization log information to prevent the need for full data re-synchronization after table data synchronization is interrupted. In practical applications, the time required for table data synchronization processing is effectively reduced, and the efficiency of table data synchronization processing is improved. .

In an optional implementation manner, FIG. 2 is a schematic flowchart of a method for synchronizing table data in step 102 provided in this implementation manner. As shown in FIG. 2 , the above step 102 specifically includes:

Step 1021 , synchronize the table data batches whose synchronization status is unsynchronized in the data synchronization log information.

In this step, the synchronization state of the table data batch in the data synchronization log information is an unsynchronized state, and the table data batches corresponding to the unsynchronized state are synchronized. For example, to encode the synchronization state of the table data in the data synchronization log information, the code corresponding to the unsynchronized state is 0, the code corresponding to the synchronization state is 1, the code corresponding to the synchronization abnormal state is 2, and the code corresponding to the synchronized state is 3, the code corresponding to the unsynchronized state of the table data batch in this step is 0.

Step 1022, when the synchronization of the table data batches is completed, update the synchronization status of the table data batches in the data synchronization log information to be synchronized.

In this step, when the synchronization of the unsynchronized state of the table data batch is completed, the synchronization state of the table data batch is updated to the synchronized state, that is, the code corresponding to the unsynchronized state of the table data batch is 0 After the table data batch is updated synchronously, the code corresponding to the synchronization status of the table data batch is updated to 3.

For example, the target source meter data replacement station is the relevant battery status data during the battery charging process, monitors the battery charging process, and reports the battery status data that is being charged every 10 seconds. When the accumulated data reaches more than 6 billion, when the data is connected to the server or data warehouse, the historical data needs to be initialized to the server. If the traditional data synchronization method is used to initialize the data, when the data synchronization (abnormal extraction) fails , it is necessary to extract the full amount of charging process data again, which consumes time and resources.

If the table data synchronization processing method of this embodiment is adopted, the target source table data includes relevant historical data in the battery charging process, and different batches of table data are obtained after the target source table data is processed in batches according to the preset batching rules. data, and store different batches of table data in data synchronization log information, and the data synchronization log information stores the synchronization status of different batches of table data. When a batch of table data encounters a data synchronization failure, the synchronization status of the batch of table data can be updated to a synchronization exception status for subsequent processing.

The table data synchronization processing method in this embodiment synchronizes the table data batches whose synchronization state is unsynchronized in the data synchronization log information. When the synchronization of the table data batches is completed, the data synchronization log information The synchronization status of the table data batch is updated to synchronized, which realizes the synchronization processing of unsynchronized table data, provides a recording method for the data table synchronization process, only synchronizes the unsynchronized table data batches, and realizes the continuous transmission of data. The efficiency of synchronous processing of table data has been improved.

In an optional implementation manner, FIG. 3 is a schematic flowchart of a table data synchronization processing method provided in this embodiment. As shown in FIG. 3 , the table data synchronization processing method includes:

Step 201: Process target source table data in batches according to preset field information to generate different batches of table data.

Wherein, the field information includes any one or more of time information, classification information, and serial number information.

Step 202, acquiring data synchronization log information.

This step corresponds to step 101 above.

The synchronization status of different batches of table data is stored in the data synchronization log information, wherein the different batches of table data are obtained by batch processing the target source table data.

The data synchronization log information also stores the synchronization order of the table data batches. Based on the synchronization status and synchronization order of the table data batches in the data synchronization log information, the corresponding table data batches are synchronized, and the synchronization status is updated in real time to Data synchronization log information.

Step 203: Synchronize the table data batches whose synchronization status is unsynchronized in the data synchronization log information.

This step corresponds to step 1021 above.

Step 209 , when it is detected that the table data batches have been synchronized, update the synchronization status of the table data batches in the data synchronization log information to be synchronized.

This step corresponds to step 1022 above.

In the table data synchronization processing method of this embodiment, the target source table data is processed in batches according to the preset field information to generate different batches of table data, the data synchronization log information is obtained, and the table data batches in the data synchronization log information The synchronization status is synchronized for unsynchronized table data batches. When it is detected that the table data batches have been synchronized, the synchronization status of the table data batches in the data synchronization log information is updated to synchronized, realizing the synchronization of unsynchronized table data. Synchronization processing provides a recording method for the synchronization process of the data table, and only synchronizes unsynchronized table data batches, which realizes data continuation and improves the efficiency of table data synchronization processing.

In an optional embodiment, the step of updating the synchronization status to the data synchronization log information in the above step 102 in real time further includes: when the table data batches are being synchronized, updating the table data batches in the data synchronization log information The synchronization status is updated to Synchronizing.

FIG. 4 is a schematic flowchart of a table data synchronization processing method provided in this embodiment. As shown in FIG. 4 , after the above step 203, it also includes:

Step 204, when it is detected that the table data batches are being synchronized, update the synchronization status of the table data batches in the data synchronization log information to be synchronizing.

In this embodiment, the code corresponding to the unsynchronized state of the table data batch is 0, and the table data batch is synchronized. When the table data batch is being synchronized, the table data batch in the data synchronization log information The synchronization state of is the synchronization state, and the code corresponding to the synchronization state is 1, that is, the code corresponding to the synchronization state of the table data batch is updated to 1.

In the table data synchronization processing method in this embodiment, when the table data batches are being synchronized, the synchronization status of the table data batches in the data synchronization log information is updated to be synchronizing, so as to realize the monitoring of the table data synchronization processing process, It is convenient for monitoring personnel to know the progress of synchronous processing of table data in time.

In an optional embodiment, the step of updating the synchronization status to the data synchronization log information in the above step 102 in real time further includes: when the synchronization of the table data batches is abnormal, updating the table data batches in the data synchronization log information The sync status for is updated to sync exceptions. In this embodiment, the code corresponding to the unsynchronized state of the table data batch is 0, and the table data batch is synchronized. When the synchronization of the table data batch is abnormal, the table data batch in the data synchronization log information The second synchronization state is the synchronization abnormal state, and the code corresponding to the synchronization abnormal state is 2, that is, the code corresponding to the synchronization state of the table data batch is updated to 2.

In the table data synchronization processing method in this embodiment, when an abnormality occurs in the synchronization of table data batches, the synchronization status of the table data batches in the data synchronization log information is updated to a synchronization exception, which realizes the synchronization of table data that occurs during the table data synchronization process. Abnormal batches are marked, and at the same time, abnormal state monitoring can be realized, which is convenient for monitors and developers to locate abnormal table data batches in time, provides a basis for data repair and re-synchronization, and improves the efficiency of table data synchronization processing.

In an optional embodiment, the step of updating the synchronization status to the data synchronization log information in the above step 102 in real time also includes: when the synchronization of the table data batches is abnormal, stop processing the table data of the abnormal current batch. Synchronize, and update the synchronization status of the current batch of table data where an exception occurs in the data synchronization log information to synchronization exception.

FIG. 5 is a schematic flowchart of a table data synchronization processing method provided in this embodiment. As shown in FIG. 5 , after the above step 204, it also includes:

Step 205, when it is detected that the synchronization of the batch of table data is abnormal, stop synchronizing the table data of the current batch with the abnormality, and update the synchronization status of the table data of the current batch with the abnormality in the data synchronization log information to Synchronization Abnormal .

In this embodiment, in this embodiment, the code corresponding to the unsynchronized state of the table data batch is 0, and the table data batch is synchronized. When the synchronization of the table data batch is abnormal, the synchronization of the table data batch is stopped. The table data synchronization of the current batch is abnormal. At this time, the synchronization state of the table data batch in the data synchronization log information is the synchronization abnormal state, and the code corresponding to the synchronization abnormal state is 2, that is, the synchronization state of the table data batch corresponds to Coding updated to 2. For example, the situation where the synchronization abnormal state occurs may be that the network is disconnected during the data synchronization process or data abnormality occurs in the synchronization data itself.

In the table data synchronization processing method in this embodiment, when an abnormality occurs in the synchronization of the table data batches, stop synchronizing the table data of the current batch in which the abnormality occurs, and record the table data of the current batch in which the abnormality occurs in the data synchronization log information. The synchronization status is updated to Synchronization Abnormal, which realizes the monitoring of the abnormal status of the table data synchronization process, which is convenient for the monitoring personnel to know the abnormal process of the table data synchronization process in time, and stops the data synchronization in time when the table data synchronization process is abnormal, so as to prevent Abnormal data enters the server, optimizing the process of table data synchronization.

In an optional embodiment, the step of updating the synchronization status to the data synchronization log information in the above step 102 in real time also includes: when the synchronization of the table data batches is abnormal, stop processing the table data of the abnormal current batch. Synchronize, and update the synchronization status of the table data in the current batch with exceptions in the data synchronization log information to synchronization exceptions; in response to stopping the synchronization of the table data in the current batch with exceptions, continue to synchronize the table data batches in the data synchronization log information The rest of the table data batches whose synchronization status is not synchronized.

FIG. 6 is a schematic flowchart of a table data synchronization processing method provided in this embodiment. As shown in FIG. 6 , after the above step 205, it also includes:

Step 206 , in response to stopping the synchronization of the table data of the current batch where the abnormality occurs, continue to synchronize the rest of the table data batches whose synchronization status of the table data batches in the data synchronization log information is unsynchronized.

In this embodiment, the code corresponding to the unsynchronized state of the table data batch is 0, and the table data batch is synchronized. When the synchronization of the table data batch is abnormal, the synchronization of the abnormal current batch is stopped. Table data synchronization, at this time, the synchronization state of the table data batch in the data synchronization log information is the synchronization abnormal state, and the code corresponding to the synchronization abnormal state is 2, that is, the code corresponding to the synchronization state of the table data batch is updated to 2; when When the synchronization state of the table data batch is synchronization exception state 2, stop synchronizing the current batch of table data (that is, the table data batch in the synchronization abnormal state), and continue to synchronize the next batch of table data batches in the data synchronization log information The rest of the table data batches whose synchronization status is not synchronized.

In this embodiment, when the synchronization status of multiple table data batches is unsynchronized, the synchronization tasks are executed sequentially. If the synchronization of the current table data batches is abnormal, the synchronization status of the table data batches is If the synchronization exception status is 2, then the synchronization status of the next table data batch is executed for the rest of the table data batches that are not synchronized.

In the table data synchronization processing method in this embodiment, when an abnormality occurs in the synchronization of the table data batches, stop synchronizing the table data of the current batch in which the abnormality occurs, and record the table data of the current batch in which the abnormality occurs in the data synchronization log information. Synchronization status is updated to synchronization exception, in response to stop synchronizing the table data of the current batch where the exception occurs, continue to synchronize the data synchronization log information, the synchronization status of the table data batch is the rest of the table data batches that are not synchronized, and skip exception is provided Table data batches continue to complete the remaining batches of table data, avoiding the interruption of the synchronization process by abnormal data, improving the automation of table data synchronization processing, and improving the efficiency of table data synchronization processing.

In an optional embodiment, the step of updating the synchronization status to the data synchronization log information in the above step 102 in real time also includes: when the synchronization of the table data batches is abnormal, stop processing the table data of the abnormal current batch. Synchronize, and update the synchronization status of the abnormal current batch of table data in the data synchronization log information to synchronization exception; in response to stopping the synchronization of the abnormal current batch of table data, perform abnormal repair on the abnormal table data batch ;Update the synchronization status of the batches of table data after exception repair to unsynchronized, and synchronize the batches of table data after exception repair.

FIG. 7 is a schematic flowchart of a table data synchronization processing method provided in this embodiment. As shown in FIG. 7 , after the above step 205, it also includes:

Step 207 , in response to stopping the synchronization of the table data of the current batch where the exception occurs, and repairing the exception of the batch of table data where the exception occurs.

Step 208 , updating the synchronization status of the batches of table data after abnormality repair to unsynchronized, and synchronizing the batches of table data after abnormality repairing.

In this embodiment, the code corresponding to the unsynchronized state of the table data batch is 0, and the table data batch is synchronized. When the synchronization of the table data batch is abnormal, the synchronization of the abnormal current batch is stopped. Table data synchronization, at this time, the synchronization state of the table data batch in the data synchronization log information is the synchronization abnormal state, and the code corresponding to the synchronization abnormal state is 2, that is, the code corresponding to the synchronization state of the table data batch is updated to 2; when When the synchronization state of the table data batch is synchronization exception state 2, stop synchronizing the table data of the current batch with the exception, and repair the exception of the table data batch with the exception, and synchronize the table data batch after the exception repair The status is updated to unsynchronized, that is, the synchronization status code of the table data batch is updated to the code 0 corresponding to the unsynchronized state, and the table data batch after the abnormality is repaired is synchronized, that is, after a certain batch of table data batch synchronization is completed Afterwards, if there is a batch of unsynchronized table data that has been repaired before its sequence, the batch of unsynchronized table data that has been repaired will be synchronized.

In the table data synchronization processing method in this embodiment, when an abnormality occurs in the synchronization of the table data batches, stop synchronizing the table data of the current batch in which the abnormality occurs, and record the table data of the current batch in which the abnormality occurs in the data synchronization log information. The synchronization status is updated to synchronization exception; in response to stopping the synchronization of the table data of the current batch with the exception, the exception repair is performed on the abnormal table data batch; the synchronization status of the table data batch after the exception repair is updated to unsynchronized, And synchronize the batches of table data after the abnormality is repaired; thus, the table data after the abnormality is eliminated can be resumed. The time required for table data synchronization processing improves the efficiency of table data synchronization processing.

In an optional implementation manner, before step 101, it further includes: performing batch processing on the target source table data according to preset field information to generate different batches of table data.

The table data synchronous processing method in this embodiment performs batch processing on the target source table data according to the preset field information to generate different batches of table data, and provides the synchronization of the target source table data according to the characteristics of the target source table data. The basis for batch processing ensures that the target source data will not be missed or confused during batch processing, which improves the efficiency of synchronous processing of table data.

In an optional implementation manner, the field information includes any one or more of time information, classification information, and serial number information.

Table 1 is a schematic table of target source table data in a table data synchronization processing method provided in this embodiment.

As shown in Table 1, the time information is the month (month) in Table 1, the classification information is the province (province) and city (city) in Table 1, and the serial number information is the id (Identity document) in Table 1. ), the field information includes any one or more of time information, classification information, and serial number information. According to the preset field information, the target source table data is processed in batches to generate different batches of table data. Either process the data in the target source table in batches based only on time information or classification information, or based on sequence number information and time information, or sequence number information and classification information, or time information and classification information, or time information, Classification information and serial number information perform batch processing on the target source table data.

Schematic diagram of target source table data in table 1 table data synchronization processing method

In the table data synchronization processing method in this embodiment, the target source table data is processed in batches according to the preset field information to generate different batches of table data, and the field information includes any one of time information, classification information, and serial number information or more; according to the characteristics of the target source table data, it provides the basis for the batch processing of the target source table data, ensuring that the target source data will not be missed and confused during batch processing, and improves the reliability of table data synchronization processing sex. According to the characteristics of the battery monitoring data of the power station, the basis for selecting table data to divide batches can not only prevent data omission and confusion, but also ensure that the data volume of table data in different batches is similar. According to the meaning of field information, it can also help maintenance And developers clearly understand the content of data contained in table data batches; it can be seen that it is not only beneficial to data synchronization, but also beneficial to system maintenance and development.

In an optional implementation, the synchronization sequence of the table data batches is also stored in the data synchronization log information; the above step 102 specifically includes: based on the synchronization status and synchronization sequence of the table data batches in the data synchronization log information, the The corresponding table data batches are synchronized.

Table 2 is a schematic table of data synchronization log information in a table data synchronization processing method provided in this embodiment.

ID batch status 1 2021-01 0 2 2021-02 0 3 2021-03 0 4 2021-04 0 5 2021-05 0 6 2021-06 0 7 2021-07 0 8 2021-08 0 9 2021-09 0 10 2021-10 0

Table 2 shows the schematic diagram of data synchronization log information in the data synchronization processing method

As shown in Table 2, the synchronization sequence is the ID in Table 2, and the synchronization status is the status (synchronization status) in Table 2. A status of 0 indicates that the synchronization status of the batch of data in this table is not synchronized.

The batch in Table 2 is the batch of table data. Based on the synchronization status and synchronization order of the table data batches in the data synchronization log information, that is, based on the ID and status, the table data batches in the data synchronization log information are synchronized.

The table data synchronization processing method in this embodiment, based on the synchronization state and synchronization order of the table data batches in the data synchronization log information, synchronizes the corresponding table data batches; On the one hand, it can prevent the omission of data in the synchronization, and on the other hand, it also provides developers with the possibility to control the table data synchronization process.

In an optional implementation manner, in the step of obtaining data synchronization log information in the above step 101, the synchronization status of different batches of table data of multiple target source table data classified based on table names is stored in the data synchronization log information .

In this embodiment, the synchronization status of different batches of table data of multiple source table data is stored in the data synchronization log information, and table data of different batches of multiple target source table data are classified based on the table name. For table data, there is no need to create new data synchronization log information, just add different batches of table data and the synchronization status of the different batches of table data based on the table name in the original data synchronization log information.

In the table data synchronization processing method in this embodiment, the synchronization status of different batches of table data of multiple target source table data classified based on the table name is stored in the data synchronization log information. Batches of table data are integrated into the same data synchronization log information, and the corresponding table data batches are synchronized to achieve batch processing of target source table data to obtain data synchronization log information, which makes the method common and Scalability, for the synchronization requirements of the newly added target source table data, it can also realize the data synchronization that can resume the breakpoint. Example 2

This embodiment provides an electronic device, and FIG. 8 is a schematic structural diagram of the electronic device provided by this embodiment. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the table data synchronization processing method in the above embodiment is implemented. The electronic device 80 shown in FIG. 8 is only an example, and should not limit the functions and scope of use of this embodiment of the present application.

As shown in FIG. 8, the electronic device 80 may be in the form of a general-purpose computing device, for example, it may be a server device. Components of the electronic device 80 may include, but are not limited to: at least one processor 81 , at least one memory 82 , and a bus 83 connecting different system components (including the memory 82 and the processor 81 ).

The bus 83 includes a data bus, an address bus and a control bus.

The memory 82 may include a volatile memory, such as a random access memory (RAM) 821 and/or a cache memory 822 , and may further include a read only memory (ROM) 823 .

Memory 82 may also include programs/utilities 825 having a set (at least one) of program modules 824 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, which Each or some combination of the examples may include the implementation of a network environment.

The processor 81 executes various functional applications and data processing by executing the computer program stored in the memory 82, such as the table data synchronization processing method in the above embodiment of the present application.

Electronic device 80 may also communicate with one or more external devices 84 (eg, keyboards, pointing devices, etc.). Such communication may occur through input/output (I/O) interface 85 . Also, the model generation device 80 can also communicate with one or more networks (eg, a local area network (LAN), a wide area network (WAN) and/or a public network, such as the Internet) via a network adapter 86 . As shown in FIG. 8 , network adapter 86 communicates with other modules of model generation device 80 via bus 83 . It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the model generating device 80, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems.

It should be noted that although several units/modules or subunits/modules of an electronic device are mentioned in the above detailed description, such division is only exemplary and not mandatory. Actually, according to the embodiment of the present application, the features and functions of two or more units/modules described above may be embodied in one unit/module. Conversely, the features and functions of one unit/module described above can be further divided to be embodied by a plurality of units/modules.

Example 3

This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps in the method for synchronizing table data in the above embodiment are implemented.

Wherein, the readable storage medium may more specifically include but not limited to: portable disk, hard disk, random access memory, read-only memory, erasable programmable read-only memory, optical storage device, magnetic storage device or any of the above-mentioned the right combination.

In a possible implementation, the present application can also be implemented in the form of a program product, which includes program code. When the program product is executed on the terminal device, the program code is used to make the terminal device execute the table in the above embodiment. Steps in a data synchronization process method.

Wherein, the program code for executing the application can be written in any combination of one or more programming languages, and the program code can be completely executed on the user equipment, partially executed on the user equipment, as an independent software Package execution, partly on the user device and partly on the remote device, or entirely on the remote device.

Although the specific implementation manners of the present application have been described above, those skilled in the art should understand that this is only an example, and the protection scope of the present application is defined by the appended claims. Those skilled in the art can make various changes or modifications to these embodiments without departing from the principle and essence of the present application, but these changes and modifications all fall within the protection scope of the present application.

Claims (13)

1. A table data synchronous processing method, is characterized in that, comprises: Obtaining data synchronization log information, the data synchronization log information stores the synchronization status of different batches of table data, wherein the different batches of table data are obtained by batch processing the target source table data; Based on the synchronization status of the table data batches in the data synchronization log information, the corresponding table data batches are synchronized, and the synchronization status is updated in the data synchronization log information in real time. 2. The method according to claim 1, wherein, based on the synchronization state of the table data batches in the data synchronization log information, the corresponding table data batches are synchronized, and the synchronization state is updated to The steps in the data synchronization log information include: Synchronize the table data batches whose synchronization status is unsynchronized in the data synchronization log information; When the synchronization of the batch of table data is completed, the synchronization status of the batch of table data in the data synchronization log information is updated to be synchronized. 3. The method according to claim 2, wherein the step of updating the synchronization status to the data synchronization log information in real time further comprises: When the batch of table data is being synchronized, the synchronization status of the batch of table data in the data synchronization log information is updated as synchronizing. 4. The method according to claim 2, wherein the step of updating the synchronization status to the data synchronization log information in real time further comprises: When the synchronization of the table data batches is abnormal, the synchronization state of the table data batches in the data synchronization log information is updated to synchronization abnormality. 5. The method of claim 4, further comprising: When the synchronization of the batch of table data is abnormal, stop synchronizing the table data of the current batch of abnormal occurrence, and update the synchronization status of the abnormal current batch of table data in the data synchronization log information to synchronization abnormal. 6. The method of claim 5, further comprising: In response to stopping the table data synchronization of the abnormal current batch, continue to synchronize the remaining table data batches whose synchronization status of the table data batches in the data synchronization log information is unsynchronized. 7. The method of claim 5, further comprising: In response to stopping the table data synchronization of the abnormal current batch, performing abnormal repair on the abnormal table data batch; Update the synchronization status of the batches of table data after exception repair to unsynchronized, and synchronize the batches of table data after exception repair. 8. The method according to claim 1, wherein before the step of obtaining data synchronization log information, the method further comprises: According to the preset field information, the target source table data is processed in batches to generate different batches of table data. 9. The method according to claim 8, wherein the field information includes any one or more of time information, classification information, and serial number information. 10. The method according to claim 1, wherein the synchronization order of table data batches is also stored in the data synchronization log information; The step of synchronizing corresponding table data batches based on the synchronization status of table data batches in the data synchronization log information includes: Based on the synchronization state and synchronization order of the table data batches in the data synchronization log information, the corresponding table data batches are synchronized. 11. The method according to claim 1, wherein in the step of obtaining data synchronization log information, The data synchronization log information stores synchronization states of different batches of table data of multiple target source table data classified based on table names. 12. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and operable on the processor, characterized in that, when the processor executes the computer program, any one of claims 1 to 11 can be realized. Table data synchronization processing method described in item. 13. A computer-readable medium, on which computer instructions are stored, wherein when the computer instructions are executed by a processor, the table data synchronization processing method according to any one of claims 1-11 is implemented.

Images