CN118427011A

CN118427011A - Data maintenance method, electronic device and computer readable medium

Info

Publication number: CN118427011A
Application number: CN202410520795.2A
Authority: CN
Inventors: 陈焱
Original assignee: Yici Netlink Hangzhou Technology Co ltd
Current assignee: Yici Netlink Hangzhou Technology Co ltd
Priority date: 2024-04-28
Filing date: 2024-04-28
Publication date: 2024-08-02

Abstract

The application discloses a data maintenance method, which comprises the following steps: reading check data from a check file, reading real data from a real file, and calculating a check value of the real data; under the condition that the occurrence of data damage is determined according to the check data and the check value, reading backup data from a backup file; overlaying the real data in the real file with the backup data; the verification file and the backup file are located in a backup storage area, the real file is located in a real storage area, the backup storage area and the real storage area are located on different storage media, the real file and the backup file are used for storing direct-write written same data, and the verification file is used for storing verification values of the direct-write written same data. The application also discloses an electronic device and a computer readable medium. The reliability of the data is guaranteed.

Description

Data maintenance method, electronic device and computer readable medium

Technical Field

The present application relates to the field of data maintenance, and in particular, to a data maintenance method, an electronic device, and a computer readable medium.

Background

Because the power supply system is not as stable as that of the enterprise-level machine room, the power input loss situation of the household-level embedded equipment often occurs, and most of the household-level embedded equipment is configured with a large capacitor or a spare battery to ensure that the equipment can still work for a period of time after power failure, and then, in order to prolong the available time of the equipment after power failure as much as possible, peripherals with large power consumption, such as Solid state disks (Solid STATE DRIVES, SSD), are preferentially closed after a relatively short delay time (such as 1S).

Nevertheless, there is still the problem of damage to the data caused by the electrical drop, and some important data such as configuration data may cause abnormal traffic after being damaged.

Disclosure of Invention

The present application aims to solve one of the technical problems in the related art to a certain extent. To this end, the present application provides a data maintenance method, an electronic device, and a computer-readable medium.

As a first aspect of the present application, there is provided a data maintenance method, wherein the method includes:

Reading check data from a check file, reading real data from a real file, and calculating a check value of the real data;

under the condition that the occurrence of data damage is determined according to the check data and the check value, reading backup data from a backup file;

overlaying the real data in the real file with the backup data;

The verification file and the backup file are located in a backup storage area, the real file is located in a real storage area, the backup storage area and the real storage area are located on different storage media, the real file and the backup file are used for storing direct-write written same data, and the verification file is used for storing verification values of the direct-write written same data.

Optionally, the determining that the data corruption occurs according to the check data and the check value includes:

And under the condition that the check data is not matched with the check value, determining that data damage occurs.

Optionally, after the backup data is read from the backup file in the case that the data corruption is determined to occur according to the check data and the check value, the method further includes:

responding to a data reading request according to the backup data.

Optionally, the method further comprises:

and under the condition that the data damage does not occur according to the check data and the check value, responding to a data reading request according to the real data.

Optionally, after said overlaying said real data in said real file with said backup data, said method further comprises:

Writing the data to be stored corresponding to the data storage request into the backup file in a direct-write manner;

calculating a check value of the data to be stored, and directly writing the check value of the data to be stored into the check file as check data;

and writing the data to be stored into the real file by direct writing.

Optionally, the method further comprises:

Under the condition that the data damage does not occur according to the check data and the check value, writing the data to be stored corresponding to the data storage request into the backup file in a direct writing way;

and writing the data to be stored into the real file by direct writing.

Optionally, the calculating the check value of the real data includes:

And calculating a check value of the real data according to a preset hash algorithm.

Optionally, the preset hash algorithm includes a fifth version of the message digest algorithm MD5.

Optionally, the backup storage area is located on one of a solid state disk SSD and a standard embedded memory eMMC, and the real storage area is located on the other of the SSD and the eMMC.

As a second aspect of the present application, there is provided an electronic apparatus, wherein the electronic apparatus includes:

One or more processors;

And a memory having one or more computer programs stored thereon, which when executed by the one or more processors cause the one or more processors to implement the data maintenance method according to the first aspect of the present application.

As a third aspect of the present application, there is provided a computer-readable medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the data maintenance method according to the first aspect of the present application.

In the data maintenance method provided by the embodiment of the application, the real file and the backup file are used for storing the same data written by direct writing, the check file is used for storing the check value of the same data written by direct writing, the whole writing process is split into three writing stages, and the integrity of each writing stage is ensured; the backup storage area and the real storage area are located on different storage media, ensuring the integrity of the whole writing process. This makes the process of writing data to a file atomic. The backup storage area and the real storage area being located on different storage media also ensures that at least one of the backup data and the real data is not corrupted. Under the condition that the data damage occurs according to the check data and the check value, the fact that the backup data in the backup file is undamaged can be indicated, the backup data is read from the backup file to cover the real data which is possibly damaged in the real file, and the real data in the real file is maintained to be consistent with the undamaged backup data in the backup file, so that the reliability of the data is effectively guaranteed.

Drawings

The application is further described below with reference to the accompanying drawings:

FIG. 1 is a flow chart of an embodiment of a method for maintaining data according to an embodiment of the present application;

FIG. 2 is a flow chart of a further implementation of a data maintenance method according to an embodiment of the present application;

FIG. 3 is a flow chart of a further implementation of the data maintenance method according to an embodiment of the present application;

FIG. 4 is a flow chart of another method for maintaining data according to an embodiment of the present application;

FIG. 5 is a flow chart of a further implementation of a data maintenance method according to an embodiment of the present application;

FIG. 6 is a schematic flow chart of an embodiment of the present application when there is a data storage requirement;

FIG. 7 is a schematic flow chart of an embodiment of the present application when there is a data reading requirement;

FIG. 8 is a block diagram of an embodiment of an electronic device provided by an embodiment of the present application;

fig. 9 is a block diagram of a computer readable medium according to an embodiment of the present application.

Description of the reference numerals

101: Processor 102: memory device

103: I/O interface 104: bus line

Detailed Description

Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to like or similar elements or elements having like or similar functions throughout. The examples in the embodiments are intended to illustrate the present application and are not to be construed as limiting the present application.

Reference in the specification to "one embodiment" or "an example" means that a particular feature, structure, or characteristic described in connection with the embodiment itself can be included in at least one embodiment of the disclosure. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

At present, the problem of data corruption caused by power loss still exists in the household-level embedded device. For example, a home-level embedded device may be configured with a large capacitor or a spare battery to ensure that the device can still operate for a period of time after power failure, and then, in order to prolong the usable time of the device after power failure as much as possible, the SSD with large power consumption is turned off preferentially after a relatively short delay (for example, 1S), where if there is data writing into the SSD, the SSD is likely to be damaged by power failure.

The inventors of the present application have found that if a transaction is considered to be a program, which is either completely executed or not executed at all, this property is called atomicity, but the process of writing data into a file by the operating system is not substantially atomistic, so that when data is written in half, if the Input/Output Interface (I/O Interface) of the storage device is abnormal, there is a high probability that the data in the file will be corrupted.

In view of this, the inventors of the present application have proposed that, by making the process of writing data into a file atomic, the occurrence of data corruption can be effectively avoided.

As a first aspect of the embodiment of the present application, there is provided a data maintenance method, as shown in fig. 1, the method may include the steps of:

in step S110, reading verification data from a verification file, reading real data from a real file, and calculating a verification value of the real data;

in step S120, in the case that it is determined that data damage occurs according to the check data and the check value, the backup data is read from the backup file;

in step S130, the real data in the real file is overlaid with the backup data;

In the embodiment of the application, the data corruption can include, but is not limited to, various abnormal situations such as data deviation from an actual value, data loss and the like. For distinction, the data read from the check file, the real file, and the backup file are sequentially referred to as check data, real data, and backup data, respectively.

In the embodiment of the application, the same data is directly written into the real file and the backup file respectively, and the check value of the same data is directly written into the check file, namely, the whole writing process is split into three writing stages. Because of the special property of write-through, when each write-through stage completes write-through, the description data must also be written into the corresponding file, thereby guaranteeing the integrity of each write-through stage. Furthermore, the backup storage area and the real storage area are positioned on different storage media, so that the completion degree of three writing stages is ensured, namely the integrity of the whole writing process is ensured. Finally, the backup storage area and the real storage area are located on different storage media, and because the different storage media cannot be powered down in the writing stage, the situation that the real data and the backup data are damaged at the same time even though the writing stage is completed is avoided.

The writing operation in the writing process is expanded from one time to three times, and each time, a direct writing mode is adopted, so that two or more writing anomalies are unlikely to exist at the same time, and any one writing anomaly only causes the damage of data in one file. That is, after the data writing is completed, at least one file must be stored in the real file and the backup file, and the data stored in the at least one file is not damaged.

And determining that data damage occurs according to the check data and the check value, wherein the occurrence of data damage indicates that one of the check data and the real data is abnormal, and the possibility of damage to the real data is high, namely, one of a writing stage of directly writing the check value into the check file and a writing stage of directly writing the real data into the real file is abnormal, and the writing stage of directly writing the backup data into the backup file is necessarily normal.

That is, in the case where it is determined that data corruption occurs based on the check data and the check value, it can be explained that the writing phase of writing the backup data directly into the backup file is normal, and at this time, the backup data in the backup file is normal, that is, no corruption occurs. In this way, the backup data is read from the backup file, the backup data is used for covering the real data which is possibly damaged in the real file, and the real data in the real file is maintained, so that the real data is consistent with the backup data which is not damaged in the backup file, and the reliability of the data is effectively ensured.

As described above, when writing data, the check value of the data is written as the check data directly into the check file, and the data itself is written directly into the real file, then when the check data read from the check file does not match (is inconsistent with) the check value of the real data, it is indicated that the check data or the real data is damaged. Accordingly, in some embodiments, the determining that the data corruption (i.e. the data corruption is involved in the step S120) occurs according to the check data and the check value may include: and under the condition that the check data is not matched with the check value, determining that data damage occurs.

In the embodiment of the present application, the time for data maintenance is not particularly limited. For example, data maintenance may be performed when a predetermined period has arrived and/or when there is a data reading need. Accordingly, in some embodiments, as shown in fig. 2, after the backup data is read from the backup file (i.e. step S120) in the case that it is determined that the data corruption occurs according to the check data and the check value, the method may further include the steps of:

in step S140, a data read request is responded to according to the backup data.

The data reading request may be received before step S110, where data needs to be read, but it is not determined whether the previous writing process has a write abnormality, that is, it is not determined which of the backup data and the real data has no damage, and by executing step S110 and step S120, it is determined that the backup data has no damage, and the data reading request is responded according to the read backup data. And the read backup data is further used for maintaining the real data which may be damaged.

Of course, in addition to the above, there may be a data reading requirement when the data maintenance is performed according to the preset period, and the data reading request may be received after step S110. After the data maintenance is started, when the situation that the real data is possibly damaged is identified and the fact that the backup data is not damaged is confirmed, the read backup data can be used for maintaining the real data possibly damaged, and the read backup data can be further used for responding to the data reading request.

And determining that the data is not damaged according to the check data and the check value, determining that the real data is not damaged, and responding to the data reading request according to the real data. Accordingly, in some embodiments, as shown in fig. 3, the method may further include the steps of:

in step S150, in the case where it is determined that no data corruption has occurred based on the check data and the check value, a data read request is responded to based on the real data.

The data reading request may be received before step S110, where data needs to be read, but it is not determined whether the previous writing process has a write abnormality, that is, it is not determined which of the backup data and the real data has no damage, and by executing step S110 and step S150, it is determined that the real data has no damage, and the data reading request is responded according to the read real data.

Of course, the data read request may also be received after step S110. After the data maintenance is started, when the fact that the real data is not damaged is confirmed, the read backup data can be used for responding to the data reading request.

In summary, whenever a data read request is received, it is only necessary to ensure that the data in response to the data read request is uncorrupted.

Analysis may be performed for the case of a write exception at any of three stages of the overall write process:

① . If an abnormality occurs in the write phase of writing backup data directly to the backup file: at this time, checking values of the check data in the check file, wherein the check values of the check data in the check file are the values of the check data in the check file;

② . If an abnormality occurs in the write phase of writing the verification data directly to the verification file: at this time check data ≡ in the check file! Check value of real data in real file;

③ . If an abnormality occurs in the writing phase of writing the real data directly to the real file: at this time check data ≡ in the check file! Check value of real data in real file.

It can be seen that, in the case that the check data does not match the check value, the above condition ② or condition ③ must exist, and the check data or the real data is damaged, where it can be clearly understood that the backup data is not damaged; in the case where the check data matches the check value, the condition ① may or may not exist, and it is clear that the real data is not corrupted.

In summary, for a data read request, the read model can be summarized in pseudo code as follows:

Check data in if check file= check value of real file

Taking the real data of the real file

else

And taking the backup data of the backup file and covering the real data of the real file.

The inventors of the present application have also proposed that, in addition to the case where data corruption occurs, it is also possible that verification files and/or backup files are not present, in which case the backup areas are most likely to be entirely corrupted, for which the real files can be read directly. Accordingly, in some embodiments, the method further comprises: and responding to a data reading request according to the real data under the condition that the check file and/or the backup file are determined to be absent.

In addition to the possible data reading requirements, there may be data storage requirements, at which time it must be ensured that both the backup data in the backup file and the real data in the real file are not damaged in order to perform the new writing process. Accordingly, in some embodiments, as shown in fig. 4, after the overlaying of the real data in the real file with the backup data (i.e. step S130), and in the case that it is determined that no data corruption occurs according to the check data and the check value, the method may further include the steps of:

in step S210, writing the data to be stored corresponding to the data storage request into the backup file;

In step S220, a check value of the data to be stored is calculated, and the check value of the data to be stored is used as check data to be written in the check file;

in step S230, the data to be stored is written in the real file.

The inventor of the present application found that a writing process updates a file in a previous writing process, if writing anomalies occur in both the previous and subsequent writing processes, which respectively result in damage to real data and backup data, then damage to both backup data and real data will eventually occur, since whether data damage occurs according to the check data and the check value or data damage does not occur according to the check data and the check value, the read data is actually damaged (only falsely considered as not damaged). To avoid this, it is necessary to avoid the occurrence of a double write anomaly, which results in the simultaneous corruption of the backup data and the real data.

The inventor of the present application proposes that, as an effective solution, data maintenance may be performed first when there is a data storage requirement, and then data storage is performed after the data maintenance is completed, so as to ensure that the real data is kept as normal data without damage before the current writing process is performed.

Under the condition that the data damage is determined according to the check data and the check value, the backup data is used for covering the real data in the real file and then storing the data, so that the problem of write-over abnormality in the write-in stage of directly writing the real data into the real file in the last write-in process is solved.

And under the condition that the data damage does not occur according to the check data and the check value, the data can be directly written in. In this regard, the data may be stored after the backup data in the backup file is overlaid with the real data. Accordingly, in some embodiments, as shown in fig. 5, the method may further include the steps of:

In step S310, under the condition that it is determined that no data damage occurs according to the check data and the check value, writing the data to be stored corresponding to the data storage request into the backup file;

in step S320, a check value of the data to be stored is calculated, and the check value of the data to be stored is used as check data to be written in the check file;

in step S330, the data to be stored is written in the real file.

The data storage request may be received after step S110, similar to the data reading request. And data storage is performed after data maintenance is completed.

In the embodiment of the present application, how to calculate the check value of the real data is not particularly limited. For example, the verification value of the real data may be calculated according to a preset hash algorithm. Accordingly, in some embodiments, the calculating the check value of the real data (i.e. referred to in step S110) may include: and calculating a check value of the real data according to a preset hash algorithm.

The same principle as that of calculating the check value of the data to be stored (i.e., referred to in step S220 and step S320) is only required to ensure that the same hash algorithm is adopted when calculating the check value of the data to be stored and when calculating the check value of the real data.

In the embodiment of the application, the preset hash algorithm is not particularly limited. For example, the pre-set hash Algorithm may include, but is not limited to, any of secure hash Algorithm-1 (Secure Hash Algorithm, SHA-1), secure hash Algorithm-2 (Secure Hash Algorithm, SHA-2), fifth edition Message Digest Algorithm (Message-Digest Algorithm 5, MD5), RACE original integrity check Message Digest Algorithm (RACE INTEGRITY PRIMITIVES Evaluation MESSAGE DIGEST, RIPEMD-160) hash encryption Algorithm.

As a specific implementation manner with a relatively high calculation speed, in some embodiments, the preset hash algorithm includes a fifth version of the message digest algorithm MD5.

As a specific implementation, in some embodiments, the backup storage area is located on one of an SSD and a standard Embedded Multi MEDIA CARD, EMMC, and the real storage area is located on the other of the SSD and the eMMC.

The case of a data storage requirement is described in detail below with reference to fig. 6 in conjunction with a most specific embodiment. As shown in fig. 6, a flow chart of an embodiment of the present application when there is a data storage requirement is provided, where the flow chart may include the following steps: in response to the need to write configuration data, reading the content of the real file (namely real data), calculating an MD5 value (namely check value) of the real data, and reading the content of the MD5 file (namely check file) (namely check data); judging whether the MD5 value is consistent with the MD5 file content or not; if the data are consistent, the real data are not damaged, and then the writing process is executed: writing the configuration data into a new file (namely a backup file) in a direct-write manner, calculating an MD5 value of the configuration data, using the MD5 value as verification data to write into the MD5 file in a direct-write manner, and writing the configuration data into a real file in a direct-write manner; if the file contents are inconsistent, the real file is possibly damaged, and the content (namely the backup data) of the new file (namely the backup file) is read to cover the content of the real file, so that the real file is ensured to be available.

The case of a data read request is described in detail below with reference to fig. 7 in conjunction with a most specific embodiment. FIG. 7 is a schematic flow chart of an embodiment of the present application when there is a data reading requirement, the flow may include the following steps: in response to the need to read the configuration data, reading the real file content (namely real data), calculating an MD5 value (namely check value) of the real data, and reading the MD5 file content (namely check file) and the check data; judging whether the MD5 value is consistent with the MD5 file content or not; if the file contents are consistent, the contents (namely, the real data) of the real file are read; if the new file (i.e. the backup file) is inconsistent, the content (i.e. the backup data) of the new file (i.e. the backup file) is read; the new file content is written directly over the new file (i.e., the backup file) content.

As a third aspect of the embodiment of the present application, there is provided an electronic device, wherein, as shown in fig. 8, the electronic device includes:

One or more processors 101;

a memory 102 having one or more computer programs stored thereon, which when executed by the one or more processors 101, cause the one or more processors 101 to implement the data maintenance method provided by the first aspect of the embodiment of the present application.

The electronic device may also include one or more I/O interfaces 103 coupled between the processor 101 and the memory 102 configured to enable information interaction of the processor 101 with the memory 102.

Wherein the processor 101 is a device having data processing capabilities, including but not limited to a Central Processing Unit (CPU) or the like; memory 102 is a device with data storage capability including, but not limited to, random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically charged erasable programmable read-only memory (EEPROM), FLASH memory (FLASH); an I/O interface (read/write interface) is connected between the processor and the memory, and can implement information interaction between the processor and the memory, which includes, but is not limited to, a data Bus (Bus), and the like.

In some embodiments, processor 101, memory 102, and I/O interface 103 are connected to each other via bus 104, and thus to other components of the computing device.

As a third aspect of the embodiment of the present application, as shown in fig. 9, there is provided a computer readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the data maintenance method provided in the first aspect of the embodiment of the present application.

Those skilled in the art will appreciate that implementing all or part of the processes in the methods of the embodiments described above may be accomplished by computer programs to instruct related hardware. Accordingly, the computer program may be stored in a non-volatile computer readable storage medium, which when executed, performs the method of any of the above embodiments. Any reference to memory, storage, database, or other medium used in embodiments of the application may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.

The above is only a specific embodiment of the present application, but the scope of the present application is not limited thereto, and it should be understood by those skilled in the art that the present application includes but is not limited to the accompanying drawings and the description of the above specific embodiment. Any modifications which do not depart from the functional and structural principles of the present application are intended to be included within the scope of the appended claims.

Claims

1. A method of data maintenance, the method comprising:

overlaying the real data in the real file with the backup data;

2. The method of claim 1, wherein said determining that a data corruption has occurred based on said check data and said check value comprises:

3. The method of claim 1, wherein after the reading of the backup data from the backup file in the event of a data corruption determined to occur based on the verification data and the verification value, the method further comprises:

responding to a data reading request according to the backup data.

4. The method according to claim 1, wherein the method further comprises:

5. The method of claim 1, wherein after said overwriting said real data in said real file with said backup data, said method further comprises:

and writing the data to be stored into the real file by direct writing.

6. The method according to claim 1, wherein the method further comprises:

and writing the data to be stored into the real file by direct writing.

7. The method according to any one of claims 1-6, wherein said calculating a check value of said real data comprises:

8. The method of any one of claims 1-6, wherein the backup storage area is located on one of a solid state drive, SSD, and a standard embedded memory, eMMC, and the real storage area is located on the other of the SSD and eMMC.

9. An electronic device, the electronic device comprising:

One or more processors;

A memory having one or more computer programs stored thereon, which when executed by the one or more processors cause the one or more processors to implement the data maintenance method of any of claims 1-8.

10. A computer readable medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the data maintenance method according to any one of claims 1-8.