CN119127406A - Residual block device cleaning method, device, computer device, readable storage medium and program product - Google Patents
Residual block device cleaning method, device, computer device, readable storage medium and program product Download PDFInfo
- Publication number
- CN119127406A CN119127406A CN202411360990.XA CN202411360990A CN119127406A CN 119127406 A CN119127406 A CN 119127406A CN 202411360990 A CN202411360990 A CN 202411360990A CN 119127406 A CN119127406 A CN 119127406A
- Authority
- CN
- China
- Prior art keywords
- residual block
- cleaning
- block equipment
- path information
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
- G06F2009/45562—Creating, deleting, cloning virtual machine instances
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The application relates to a residual block device cleaning method, a residual block device cleaning device, computer equipment, a computer readable storage medium and a computer program product, which relate to a cloud platform backup technology and can improve the cleaning efficiency of the residual block device. The method comprises the steps of obtaining block equipment identifiers and path information of residual block equipment in a cloud platform backup system, wherein the path information comprises a data transmission path between the residual block equipment and the cloud platform backup system, storing the block equipment identifiers and the path information into preset residual block equipment information record files, detecting the residual block equipment information record files to obtain detection results when the cloud platform backup system is started, and cleaning the residual block equipment according to the block equipment identifiers and the path information if the detection results indicate that the residual block equipment information record files have the block equipment identifiers and the path information.
Description
Technical Field
The present application relates to the field of cloud platform backup technology, and in particular, to a method, an apparatus, a computer device, a computer readable storage medium, and a computer program product for a residual block device.
Background
With the rapid development of cloud computing and virtualization technologies, cloud platforms are becoming a core component in enterprise IT architecture. In order to meet the requirement of efficient operation of Virtual Machines (VMs), storage area network (Storage Area Network, SAN) technology is widely used. The SAN connects the server with the remote storage device through the network, provides storage service with high performance and centralized management, and supports large-scale virtual resource management in the cloud platform. The block device based on the small Computer system interface (INTERNET SMALL Computer SYSTEM INTERFACE, ISCSI) allows the virtual machine to access the remote storage like a local hard disk, thereby greatly improving the utilization rate of storage resources.
In the conventional backup technology, a cloud platform uses a agentless backup system, exports block devices of a virtual machine to a backup host through a SAN network, and manually cleans the block devices after the backup is completed. However, as the scale of the cloud platform increases, the backup system faces the challenge of managing thousands of SAN block devices, and the traditional cleaning mode appears to be inefficient and easily generates block device residues.
Disclosure of Invention
In view of the foregoing, it is desirable to provide a residual block device cleaning method, apparatus, computer device, computer readable storage medium, and computer program product.
In a first aspect, the present application provides a method for cleaning a residual block device, including:
acquiring block equipment identifiers and path information of residual block equipment in a cloud platform backup system, wherein the path information comprises a data transmission path between the residual block equipment and the cloud platform backup system;
storing the block equipment identifier and the path information into a preset residual block equipment information record file;
When the cloud platform backup system is started, detecting the residual block equipment information record file to obtain a detection result;
And if the detection result indicates that the block equipment identifier and the path information exist in the residual block equipment information record file, cleaning the residual block equipment according to the block equipment identifier and the path information.
In one embodiment, the obtaining the block device identifier and the path information of the residual block device in the cloud platform backup system includes:
After the cloud platform backup system finishes backup, acquiring data transmission records of the residual block equipment and the cloud platform backup system through a block equipment management interface of the cloud platform backup system;
And determining the block equipment identifier of the residual block equipment and a plurality of path information corresponding to the block equipment identifier based on the data transmission record.
In one embodiment, the cleaning the residual block device according to the block device identifier and the path information includes:
determining the number of the path information corresponding to each block equipment identifier;
sorting all the block equipment identifiers in a descending order according to the number of the path information to obtain a descending order;
Determining the cleaning priority of the residual block equipment corresponding to the block equipment identification according to the descending order;
And cleaning the residual block equipment according to the cleaning priority of the residual block equipment.
In one embodiment, the cleaning the residual block device according to the cleaning priority of the residual block device includes:
Grouping according to the cleaning priority of the residual block devices, and planning the residual block devices with the same cleaning priority into the same group to obtain a grouping result;
And according to the grouping result, the residual block devices in the same group are cleaned in parallel.
In one embodiment, after the cleaning of the residual block device according to the block device identifier and the path information, the method further includes:
cleaning and detecting the residual block equipment to obtain a cleaning and detecting result;
if the cleaning detection result indicates that the cleaning is successful, deleting the block equipment identifier and the path information corresponding to the cleaned residual block equipment in the residual block equipment information record file;
If the cleaning detection result indicates that the cleaning is failed, adding a mark for the path information of the residual block equipment which is currently cleaned and skipping, and continuing to clean the residual block equipment according to the path information which has the same block equipment identification and is not marked, wherein the mark is used for prompting that the cleaning of the corresponding residual block equipment is failed.
In one embodiment, the detecting the residual block device information record file when the cloud platform backup system is started includes:
If the cloud platform backup system is in an idle state, detecting the residual block equipment information record file according to a preset period;
And if the cloud platform backup system is in a load state, detecting the residual block equipment information record file, and detecting the residual block equipment information record file again according to the preset period when the current load state is switched to enter the next idle state.
In a second aspect, the present application further provides a residual block device cleaning apparatus, including:
The device comprises a residual block device identification module, a data transmission path information acquisition module and a data transmission module, wherein the residual block device identification module is used for acquiring block device identification and path information of residual block devices in a cloud platform backup system;
The residual block equipment information recording module is used for storing the block equipment identifier and the path information into a preset residual block equipment information recording file;
The residual block equipment detection module is used for detecting the residual block equipment information record file when the cloud platform backup system is started to obtain a detection result;
And the residual block equipment cleaning module is used for cleaning the residual block equipment according to the block equipment identifier and the path information if the detection result indicates that the block equipment identifier and the path information exist in the residual block equipment information record file.
In a third aspect, the present application also provides a computer device comprising a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring block equipment identifiers and path information of residual block equipment in a cloud platform backup system, wherein the path information comprises a data transmission path between the residual block equipment and the cloud platform backup system;
storing the block equipment identifier and the path information into a preset residual block equipment information record file;
When the cloud platform backup system is started, detecting the residual block equipment information record file to obtain a detection result;
And if the detection result indicates that the block equipment identifier and the path information exist in the residual block equipment information record file, cleaning the residual block equipment according to the block equipment identifier and the path information.
In a fourth aspect, the present application also provides a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
acquiring block equipment identifiers and path information of residual block equipment in a cloud platform backup system, wherein the path information comprises a data transmission path between the residual block equipment and the cloud platform backup system;
storing the block equipment identifier and the path information into a preset residual block equipment information record file;
When the cloud platform backup system is started, detecting the residual block equipment information record file to obtain a detection result;
And if the detection result indicates that the block equipment identifier and the path information exist in the residual block equipment information record file, cleaning the residual block equipment according to the block equipment identifier and the path information.
In a fifth aspect, the application also provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of:
acquiring block equipment identifiers and path information of residual block equipment in a cloud platform backup system, wherein the path information comprises a data transmission path between the residual block equipment and the cloud platform backup system;
storing the block equipment identifier and the path information into a preset residual block equipment information record file;
When the cloud platform backup system is started, detecting the residual block equipment information record file to obtain a detection result;
And if the detection result indicates that the block equipment identifier and the path information exist in the residual block equipment information record file, cleaning the residual block equipment according to the block equipment identifier and the path information.
The method, the device, the computer equipment, the computer readable storage medium and the computer program product for cleaning the residual block equipment are used for acquiring the block equipment identification and the path information of the residual block equipment in the cloud platform backup system, the path information comprises a data transmission path between the residual block equipment and the cloud platform backup system, the block equipment identification and the path information are stored in a preset residual block equipment information record file, the residual block equipment information record file is detected when the cloud platform backup system is started to obtain a detection result, and the residual block equipment is cleaned according to the block equipment identification and the path information if the detection result indicates that the block equipment identification and the path information exist in the residual block equipment information record file. According to the application, the block equipment which is not cleaned normally in the system can be accurately identified by acquiring the block equipment identification and the path information of the residual block equipment. The block device identifiers and the path information are stored in a preset residual block device information record file, so that a reliable storage mechanism is provided, and the information can be kept even if the system is restarted or is operated for a long time, and omission is avoided. When the cloud platform backup system is started, the residual block equipment is accurately positioned by automatically detecting the residual block equipment information record file, and if the detection result shows that the block equipment identification and the path information exist in the file, the system can call an operating system interface to execute cleaning operation according to the stored identifier and the path information. Thereby improving the cleaning efficiency of the residual block device.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related art, the drawings that are needed in the description of the embodiments of the present application or the related technologies will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other related drawings may be obtained according to these drawings without inventive effort to those of ordinary skill in the art.
FIG. 1 is a flow diagram of a method of cleaning residual block devices in one embodiment;
FIG. 2 is a flow chart of a method of cleaning residual block devices in another embodiment;
FIG. 3 is a block diagram of a residual block device cleaning apparatus in one embodiment;
fig. 4 is an internal structural diagram of a computer device in one embodiment.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
In order that those skilled in the art may better understand the present application, the related art will be described.
The cloud computing platform adopts a virtualization technology to abstract physical computing resources into virtual resources, so that a plurality of Virtual Machines (VM) can run in parallel on the same physical server. In order to support these virtual machines effectively, the cloud platform needs to have a high-performance and high-availability storage system. Storage Area Networks (SANs) are widely used in such scenarios as a core component of such storage systems.
SAN is a dedicated high-performance network that aims to interconnect computing servers with storage devices (e.g., disk arrays, tape libraries, solid state drives, etc.), ensuring that multiple servers can share and access storage resources through a high-speed network. SAN provides an efficient, reliable and scalable solution for centralized management of storage devices, particularly suited to the storage resource requirements of large-scale cloud platforms.
The block device (BlockDevice) is a storage device that performs read and write operations in a unit of data block. Block devices based on iSCSI protocols are one of the common implementations of SANs that allow block-level data to be transferred over a network, enabling a virtual machine to access a remote storage device as if it were a local hard disk. With the remote storage management capability provided by the SAN, the backup system of the cloud platform adopts a agentless architecture design. By means of the SAN, block devices used by the virtual machine are mapped to the backup host, so that backup software can take the block devices as local disks to execute backup operation, and efficient data backup is achieved.
In one embodiment, as shown in fig. 1, a method for cleaning a residual block device is provided, where the method is applied to a server for illustrating, it is understood that the method may also be applied to a terminal, and may also be applied to a system including the terminal and the server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the steps of:
Step S102, block equipment identification and path information of residual block equipment in the cloud platform backup system are obtained, wherein the path information comprises a data transmission path between the residual block equipment and the cloud platform backup system.
The residual block device refers to a block device which is not normally released by the system or is not cleaned after the backup operation is completed. The residual block device may continue to occupy system resources and impact subsequent system operation or backup performance. The block device identification refers to a unique identification code of the block device in the storage system for uniquely identifying each block device. The path information refers to data transmission path information between the system and the block device, and may indicate a plurality of data transmission paths of physical or logical connection, for locating a specific location of the device in the network or the storage system.
Alternatively, the server may read the unique identifier of the residual block device directly from the storage device via a standard storage management protocol. After acquiring the block device identifiers, the server further collects path information between each block device and the cloud platform backup system, and the server may acquire the Multipath information of each block device through a Multipath I/O (Multipath I/O) technique.
Alternatively, the server may obtain path information of the block device of the device, including details of the IP address, network interface information, and other related data transmission channels, by accessing a network storage interface in the operating system or virtualization platform.
And step S104, saving the block equipment identification and the path information to a preset residual block equipment information record file.
The residual block device information record file is used for recording residual block devices and path information thereof in the cloud platform backup system, and can be a file or a data structure of persistent storage device information. The type and manner of storage of the files may be configured according to system requirements, including but not limited to text files, database tables, log files, or the like.
Specifically, after the server obtains the unique identifier and the path information of the block device, the information is written into the residual block device information record file according to a predetermined format. In one embodiment, the server may also add other information to each record, such as time stamps, device status, to facilitate management and ordering during subsequent operations.
Alternatively, the server may write the information in lines into a text file through a standard file system interface or insert the information into a specified table through a database operating program interface.
Alternatively, the server may set a log mechanism to record each write operation. The log mechanism can ensure that the system can recover lost records through log backtracking when the server fails accidentally.
And step S106, detecting the residual block equipment information record file when the cloud platform backup system is started, and obtaining a detection result.
Specifically, when the cloud platform backup system is started, the server will first check the preset configuration in the system to determine whether a residual block device information record file exists, and if the server detects that the file exists, the server will further open the file and read the content in the file. After reading and analyzing the file, the server generates a detection result. The results may include whether the file exists, whether the file contents are valid, and whether there is residual block device information in the file that has not been processed.
Optionally, the server may set a data verification mechanism to ensure the integrity and correctness of the file. Upon detection, the server will verify the file content. In some embodiments, the server may ensure that the recorded data is not corrupted by a file hash value check or timestamp check.
Alternatively, if a residual block device is detected to be present in the log file, the server may generate a warning notification based on the status of the device, alerting the system administrator of further processing tasks. If the file does not exist or the content is empty, the server prompts that the current system has no residual block equipment, and the detection result is no operation.
Step S108, if the detection result indicates that the residual block equipment information record file has block equipment identification and path information, cleaning the residual block equipment according to the block equipment identification and the path information.
Specifically, the server confirms whether the file contains the block device identifier and the path information through the detection result. If the detection result shows that unprocessed residual block equipment exists, the server automatically enters a cleaning flow, and a cleaning instruction is sent to a storage system through a system program interface or a command line tool according to the block equipment identification and path information in the record file. The cleaning instruction is used for unloading, deleting or releasing the residual block equipment, so that the equipment is no longer connected with the cloud platform backup system.
The method for cleaning the residual block equipment comprises the steps of obtaining block equipment identifiers and path information of residual block equipment in a cloud platform backup system, wherein the path information comprises a data transmission path between the residual block equipment and the cloud platform backup system, storing the block equipment identifiers and the path information into preset residual block equipment information record files, detecting the residual block equipment information record files to obtain detection results when the cloud platform backup system is started, and cleaning the residual block equipment according to the block equipment identifiers and the path information if the detection results indicate that the residual block equipment information record files have the block equipment identifiers and the path information. In the embodiment, the block equipment which is not cleaned normally in the system can be accurately identified by acquiring the block equipment identification and the path information of the residual block equipment. The block device identifiers and the path information are stored in a preset residual block device information record file, so that a reliable storage mechanism is provided, and the information can be kept even if the system is restarted or is operated for a long time, and omission is avoided. When the cloud platform backup system is started, the residual block equipment is accurately positioned by automatically detecting the residual block equipment information record file, and if the detection result shows that the block equipment identification and the path information exist in the file, the system can call an operating system interface to execute cleaning operation according to the stored identifier and the path information. Thereby improving the cleaning efficiency of the residual block device.
In one exemplary embodiment, obtaining block device identification and path information of a residual block device in a cloud platform backup system includes:
And determining the block equipment identification of the residual block equipment and a plurality of path information corresponding to the block equipment identification based on the data transmission record.
Specifically, after the cloud platform backup system completes the backup task, the server enters a data management stage. At this stage, the backup task has been successfully ended, and the server starts a checking program of the residual block device to acquire block device information which is not normally released.
The server directly acquires the data transmission record related to the block device through a block device management interface provided by the system. The data transmission record details the data transmission situation between the block device and the server in the cloud platform backup system, including but not limited to the network identification of the block device, the block device type, the block device provider information, the unique identifier of the block device, and the block device path.
The server identifies those residual block devices that are not cleaned normally by analyzing the acquired data transfer records. At this time, the server may determine according to two conditions of whether the block device still has an active transmission path and whether there is incomplete transmission activity or abnormal connection state on the transmission path.
After identifying the residual block device, the server may further extract the unique identifier of the device, which may include WWN (World Wide Name) or UUID (Universally Unique Identifier). On the other hand, the server acquires path information related to the block equipment through the block equipment management interface, wherein the path information comprises detailed information such as a physical path, a transmission channel, a network interface and the like between the equipment and the cloud platform backup system. After the server acquires the block equipment identification and the path information thereof, the information is stored in a residual block equipment information record file in the system.
In one embodiment, in a system where the operating system is LINUX, the server may list details of the block devices connected to the backup system, including "[3:0:0:1], DISK, IET, VIRTUAL-DISK, 0x86034a11400000000048f,/dev/sdd", SCSI ID of the corresponding residual block device, device type, device provider information, unique identifier WWN of the block device, and device path, by executing lsscsi- -WWN operating system commands.
In some embodiments, in a SAN environment, a block device is provided with multi-path I/O (Input/Output) functionality, which means that there are multiple data transfer paths between the device and the server to ensure reliability of data transfer, load balancing, and failover capability. When cleaning the residual block device, the path corresponding to the WWN of the block device needs to be removed to ensure that the device is completely cleaned from the system.
In this embodiment, the server can accurately identify and manage each residual block device through the block device identifier and the path information, so as to provide complete information support for subsequent cleaning operation, and can also monitor the connection state of the block device based on the data transmission record, thereby effectively preventing device resource waste and improving the overall operation efficiency of the cloud platform backup system.
In an exemplary embodiment, cleaning the residual block device according to the block device identification and the path information comprises determining the number of path information corresponding to each block device identification, sorting the block device identifications in a descending order according to the number of path information to obtain a descending order, determining the cleaning priority of the residual block device corresponding to the block device identification according to the descending order, and cleaning the residual block device according to the cleaning priority of the residual block device.
Specifically, the server obtains path information of each residual block device through the block device management interface. For each residual block device, the server counts the number of connection paths between the server and the cloud platform backup system. Each residual block device may be connected to the server via a plurality of data transmission paths, the path information being extracted via a multi-path I/O management tool or API interface (Application Programming interface). The server executes the command to determine the number of paths of each device one by one and records the information for subsequent sequencing and priority determination.
The server orders the block devices in descending order according to the number of paths owned by each block device. For the residual block devices with more paths, the system resources occupied by the residual block devices are relatively more, in other words, the residual block devices with more paths have higher priority in a storage architecture, and the residual block devices with more paths are subjected to priority treatment so as to release more resources or ensure the smooth operation of the system.
And the server sequentially cleans the residual block equipment according to the determined cleaning priority. The server performs device unloading, releasing or deleting operations one by one from the device with the highest priority, ensuring that each device and its associated path are completely cleaned.
In this embodiment, the purposes of improving the utilization efficiency of system resources and optimizing storage management are achieved by performing descending order sorting and determination of cleaning priority according to the number of path information of the block devices. The system reasonably distributes cleaning tasks according to the number of paths of the equipment, thereby achieving the effect of optimizing storage management and ensuring the stable operation of the cloud platform backup system.
In an exemplary embodiment, cleaning the residual block devices according to the cleaning priority of the residual block devices comprises grouping the residual block devices with the same cleaning priority into the same group according to the cleaning priority of the residual block devices to obtain a grouping result, and cleaning the residual block devices of the same group in parallel according to the grouping result.
Specifically, the server firstly plans the residual block devices with the same priority into the same group according to the residual block device cleaning priority determined in the previous stage. Each priority represents the same class of equipment, and the same class of block equipment has similar resource occupation conditions in the system and can be processed simultaneously. After the grouping result is generated, the server creates a task queue for each group, wherein the queue contains the identification of the block equipment to be cleaned in the group and the corresponding transmission path information. The server starts to clean up devices in the same group at the same time based on the grouping result, instead of sequentially processing individual devices.
In the embodiment, the equipment with the same priority is grouped, and the grouped residual block equipment is cleaned in parallel, so that the system cleaning time is shortened, the equipment resource release speed is improved, and the efficiency of the residual block equipment with the system processing capacity is obviously improved.
In an exemplary embodiment, after cleaning the residual block device according to the block device identification and the path information, the method further includes:
The method comprises the steps of cleaning residual block equipment, detecting the cleaning of the residual block equipment to obtain a cleaning detection result, deleting block equipment identifiers and path information corresponding to the cleaned residual block equipment in a residual block equipment information record file if the cleaning detection result indicates that the cleaning is successful, adding marks for the path information of the currently cleaned residual block equipment and skipping over if the cleaning detection result indicates that the cleaning is failed, continuing cleaning the residual block equipment according to the path information which has the same block equipment identifier and is not marked, and marking the marks for prompting that the cleaning of the corresponding residual block equipment fails.
Specifically, when the server performs cleaning operation on a certain residual block device and path information thereof, the detection of the cleaning result is started immediately. The server checks the clean state of the device through the block device management interface or system log, including confirming whether the device has been unloaded from the system, whether the path has been deleted correctly.
If the cleaning detection result shows that the equipment is successfully cleaned, the server deletes the residual block equipment information from the residual block equipment information record file, and the equipment is ensured not to be subjected to subsequent processing. If the cleaning detection result shows that the cleaning of the equipment fails, the server adds a mark for the path information of the equipment, and marks that the path of the equipment fails to be cleaned. This flag is used to prompt the system or administrator to continue processing the remaining path of the device in a subsequent step.
For those paths marked as failed cleaning, the server will continue to process other path information not marked according to the same block device identification, try to clean the residual block device through the remaining paths, and finally clean the block device identification and path information in the residual block device record file.
In this embodiment, by performing the cleaning detection step after cleaning the residual block device, the effect of confirming whether the device and its path have been successfully cleaned is achieved. According to the detection result, the cleaning process can be effectively managed for the residual block equipment which is successfully cleaned, the equipment which is successfully cleaned is not repeatedly processed, and the marking mechanism helps the system to record the path information which is not successfully cleaned for the residual block equipment which is not successfully cleaned, so that subsequent operation is facilitated to continuously try to clean, and meanwhile equipment or paths are prevented from being missed in the cleaning process. Therefore, the effects of improving the resource management and fault processing capacity of the cloud platform backup system are achieved.
In an exemplary embodiment, detecting a residual block device information record file at a start-up of a cloud platform backup system includes:
If the cloud platform backup system is in an idle state, detecting a residual block device information record file according to a preset period; and if the cloud platform backup system is in a load state, detecting the residual block equipment information record file, and detecting the residual block equipment information record file again according to a preset period when the current load state is switched to enter the next idle state.
Specifically, when the cloud platform backup system is started, the server starts a detection task of the residual block device information record file. This detection task determines when to perform based on the current operating state of the system. At the first moment of system start-up, the server will determine the system load status to determine whether to immediately perform file detection.
And if the cloud platform backup system is in an idle state currently, the server detects the residual block equipment information record file according to a preset detection period. The detection period may be a preset time interval (e.g., every 15 minutes or 1 hour) for periodic inspection and cleaning of the residual block devices that have not been processed.
If the cloud platform backup system is in a load state, the cloud platform backup system can be used for carrying out backup tasks or processing a large amount of data, and the server immediately executes the detection operation of the residual block device information record file. The system waits for the completion of the current task, and detects the current task according to a preset period after the load state is switched to the idle state.
Optionally, the server may detect the residual block device in an idle state according to a preset period, and the time interval of detection may be dynamically adjusted according to the system requirement. For example, the server may shorten the detection period during periods of lighter duty, and may lengthen the detection period during periods of higher duty.
In the embodiment, when the cloud platform backup system is started, whether the system is in a load state or an idle state, the detection of the residual block equipment information record file can be effectively performed, and the effect of ensuring the residual block equipment to be detected and processed in time is achieved. Particularly, under the load state, the system can immediately detect, so that the rapid processing of the residual block equipment is realized, the equipment is prevented from continuously occupying system resources under the high load, and the system performance is prevented from being reduced.
In order that those skilled in the art may better understand the above steps, an embodiment of the present application will be described below by way of an example, but it should be understood that the embodiment of the present application is not limited thereto.
In one embodiment, as shown in fig. 2, after the cloud platform backup system completes the backup task, the server obtains the residual block device currently connected to the system and its path information by executing lsscsi-wwn commands. lsscsi-WWN commands list the residual block devices connected to the system and display their unique identifiers (WWNs) and corresponding device paths.
The server records the acquired block device identification and path information into a preset recording file residual_block_devices. The file stores the WWN of each device and a corresponding plurality of transmission paths. This step ensures that when the cloud platform backup system is started, the server can quickly determine that the block device is a residual block device based on the file.
And the server judges whether the residual block devices are detected according to the acquired device information. If no residual block device is detected, the system continues to perform a detection procedure to ensure that device information that may become a residual block device is completely recorded in the residual block device txt file. If all the residual block devices are detected, the server enters the next step, and a cloud platform backup system is started.
After the cloud platform backup system is started, the server firstly accesses the residual_block_devices. By reading the file, the server can judge residual block equipment still existing in the system, and further start subsequent cleaning operation. And in the running process of the system, the server is combined with a periodic detection mechanism to periodically check the residual block device txt file to judge whether residual block devices which are not cleaned exist. The server decides the execution time of the detection and cleaning operation according to the load state of the current system. If the system is in an idle state, the cleaning task is executed according to a preset period, and if the system is in a load state, the detection is still immediately carried out, but the cleaning task is executed after the system is in an idle state, so that the influence on the performance of the system is reduced.
After identifying the device that needs to be cleaned, the server performs a cleaning operation through the path management tool, which may be using the Linux command "echo 1 >/sys/block/sdX/device/delete". This command is used to unload the residual block device and ensure that its transport path is completely removed. After the device is cleaned, the server performs a detection operation to confirm whether the device and its path have been successfully cleaned. Through the device management interface, the server can judge the state of each device in real time. If the cleaning is successful, the server deletes the corresponding device information from the residual_block_devices. If the cleaning fails, the system marks the path information for the device and continues to attempt to clean the unmarked paths.
In this embodiment, by acquiring the device identifier and the path information, periodically detecting, grouping, and parallel cleaning, and a failure processing mechanism, it is ensured that the residual block device can be cleaned efficiently, and system resources are prevented from being occupied for a long time. The path number ordering mechanism further optimizes the cleaning sequence, and preferentially processes the equipment occupying more resources, thereby improving the resource utilization efficiency of the system. The periodic detection mechanism enables the system to keep track of unclean devices, avoiding the impact of device accumulation on system performance. The stability and the high efficiency of the system operation are obviously improved.
It should be understood that, although the steps in the flowcharts related to the embodiments described above are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.
Based on the same inventive concept, the embodiment of the application also provides a residual block device cleaning device for realizing the above-mentioned residual block device cleaning method. The implementation of the solution provided by the apparatus is similar to that described in the above method, so the specific limitation in the embodiments of the apparatus for cleaning a residual block device provided below may be referred to the limitation of the method for cleaning a residual block device hereinabove, and will not be repeated herein.
In one exemplary embodiment, as shown in FIG. 3, there is provided a residual block device cleaning apparatus, comprising a residual block device identification module 310, a residual block device information recording module 320, a residual block device detection module 330, and a residual block device cleaning module 340, wherein:
The residual block device identification module 310 is configured to obtain a block device identifier of a residual block device in the cloud platform backup system and path information, where the path information includes a data transmission path between the residual block device and the cloud platform backup system.
A residual block device information recording module 320, configured to store the block device identifier and the path information in a preset residual block device information recording file;
The residual block device detection module 330 is configured to detect the residual block device information record file when the cloud platform backup system is started, so as to obtain a detection result;
And a residual block device cleaning module 340, configured to clean the residual block device according to the block device identifier and the path information if the detection result indicates that the block device identifier and the path information exist in the residual block device information record file.
In one embodiment, the residual block device identification module 310 is configured to obtain, through a block device management interface of the cloud platform backup system, a data transmission record of the residual block device and the cloud platform backup system after the cloud platform backup system completes backup, and determine, based on the data transmission record, the block device identifier of the residual block device and a plurality of path information corresponding to the block device identifier.
In one embodiment, the residual block device cleaning module 340 is configured to determine a number of the path information corresponding to each block device identifier, sort the block device identifiers in descending order according to the number of the path information to obtain a descending order, determine a cleaning priority of the residual block device corresponding to the block device identifier according to the descending order, and clean the residual block device according to the cleaning priority of the residual block device.
In one embodiment, the residual block device cleaning module 340 is configured to group the residual block devices according to the cleaning priorities of the residual block devices, plan the residual block devices with the same cleaning priority into the same subgroup to obtain a grouping result, and clean the residual block devices of the same subgroup in parallel according to the grouping result.
In one embodiment, the cleaning module 340 of the residual block device is configured to perform cleaning detection on the cleaning of the residual block device to obtain a cleaning detection result, delete the block device identifier and the path information corresponding to the cleaned residual block device in the residual block device information record file if the cleaning detection result indicates that the cleaning is successful, add a flag to the path information of the residual block device currently cleaned and skip the path information if the cleaning detection result indicates that the cleaning is failed, and continue cleaning the residual block device according to the path information having the same block device identifier and not marked, where the flag is used to prompt that the cleaning of the corresponding residual block device fails.
In one embodiment, the residual block device detection module 330 is configured to detect the residual block device information record file according to a preset period if the cloud platform backup system is in an idle state, detect the residual block device information record file if the cloud platform backup system is in a load state, and detect the residual block device information record file again according to the preset period when the current load state is switched to a next idle state.
The above-described modules in the residual block device cleaning apparatus may be implemented in whole or in part by software, hardware, or a combination thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one exemplary embodiment, a computer device is provided, which may be a server, the internal structure of which may be as shown in fig. 4. The computer device includes a processor, a memory, an Input/Output interface (I/O) and a communication interface. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface is connected to the system bus through the input/output interface. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database of the computer device is used for storing the residual block device information record file. The input/output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a residual block device cleaning method.
It will be appreciated by persons skilled in the art that the architecture shown in fig. 4 is merely a block diagram of some of the architecture relevant to the present inventive arrangements and is not limiting as to the computer device to which the present inventive arrangements are applicable, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.
In one embodiment, a computer device is provided, comprising a memory and a processor, the memory having stored therein a computer program, the processor implementing the steps of the method embodiments described above when the computer program is executed.
In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored which, when executed by a processor, implements the steps of the method embodiments described above.
In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the method embodiments described above.
It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are both information and data authorized by the user or sufficiently authorized by each party, and the collection, use and processing of the related data are required to meet the related regulations.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in embodiments provided herein may include at least one of non-volatile memory and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (RESISTIVE RANDOM ACCESS MEMORY, reRAM), magneto-resistive Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (PHASE CHANGE Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in various forms such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The databases referred to in the embodiments provided herein may include at least one of a relational database and a non-relational database. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processor referred to in the embodiments provided in the present application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computation, an artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) processor, or the like, but is not limited thereto.
The technical features of the above embodiments may be arbitrarily combined, and for brevity, all of the possible combinations of the technical features of the above embodiments are not described, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the present application.
The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of the application should be assessed as that of the appended claims.
Claims (10)
1. A method of cleaning a residual block device, the method comprising:
acquiring block equipment identifiers and path information of residual block equipment in a cloud platform backup system, wherein the path information comprises a data transmission path between the residual block equipment and the cloud platform backup system;
storing the block equipment identifier and the path information into a preset residual block equipment information record file;
When the cloud platform backup system is started, detecting the residual block equipment information record file to obtain a detection result;
And if the detection result indicates that the block equipment identifier and the path information exist in the residual block equipment information record file, cleaning the residual block equipment according to the block equipment identifier and the path information.
2. The method of claim 1, wherein the obtaining block device identification and path information of the residual block device in the cloud platform backup system comprises:
After the cloud platform backup system finishes backup, acquiring data transmission records of the residual block equipment and the cloud platform backup system through a block equipment management interface of the cloud platform backup system;
And determining the block equipment identifier of the residual block equipment and a plurality of path information corresponding to the block equipment identifier based on the data transmission record.
3. The method of claim 1, wherein the cleaning the residual block device based on the block device identification and the path information comprises:
determining the number of the path information corresponding to each block equipment identifier;
sorting all the block equipment identifiers in a descending order according to the number of the path information to obtain a descending order;
Determining the cleaning priority of the residual block equipment corresponding to the block equipment identification according to the descending order;
And cleaning the residual block equipment according to the cleaning priority of the residual block equipment.
4. A method according to claim 3, wherein said cleaning of said residual block device according to said cleaning priority of said residual block device comprises:
Grouping according to the cleaning priority of the residual block devices, and planning the residual block devices with the same cleaning priority into the same group to obtain a grouping result;
And according to the grouping result, the residual block devices in the same group are cleaned in parallel.
5. The method of claim 1, wherein after cleaning the residual block device based on the block device identification and the path information, further comprising:
cleaning and detecting the residual block equipment to obtain a cleaning and detecting result;
if the cleaning detection result indicates that the cleaning is successful, deleting the block equipment identifier and the path information corresponding to the cleaned residual block equipment in the residual block equipment information record file;
If the cleaning detection result indicates that the cleaning is failed, adding a mark for the path information of the residual block equipment which is currently cleaned and skipping, and continuing to clean the residual block equipment according to the path information which has the same block equipment identification and is not marked, wherein the mark is used for prompting that the cleaning of the corresponding residual block equipment is failed.
6. The method according to any one of claims 1 to 5, wherein detecting the residual block device information record file at a start-up of the cloud platform backup system comprises:
If the cloud platform backup system is in an idle state, detecting the residual block equipment information record file according to a preset period;
And if the cloud platform backup system is in a load state, detecting the residual block equipment information record file, and detecting the residual block equipment information record file again according to the preset period when the current load state is switched to enter the next idle state.
7. A residual block device cleaning apparatus, the apparatus comprising:
The device comprises a residual block device identification module, a data transmission path information acquisition module and a data transmission module, wherein the residual block device identification module is used for acquiring block device identification and path information of residual block devices in a cloud platform backup system;
The residual block equipment information recording module is used for storing the block equipment identifier and the path information into a preset residual block equipment information recording file;
The residual block equipment detection module is used for detecting the residual block equipment information record file when the cloud platform backup system is started to obtain a detection result;
And the residual block equipment cleaning module is used for cleaning the residual block equipment according to the block equipment identifier and the path information if the detection result indicates that the block equipment identifier and the path information exist in the residual block equipment information record file.
8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 6 when the computer program is executed.
9. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
10. A computer program product comprising a computer program, characterized in that the computer program, when being executed by a processor, implements the steps of the method of any of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411360990.XA CN119127406B (en) | 2024-09-27 | 2024-09-27 | Residual block device cleaning method, apparatus, computer device, readable storage medium and program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411360990.XA CN119127406B (en) | 2024-09-27 | 2024-09-27 | Residual block device cleaning method, apparatus, computer device, readable storage medium and program product |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119127406A true CN119127406A (en) | 2024-12-13 |
| CN119127406B CN119127406B (en) | 2025-07-25 |
Family
ID=93760149
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411360990.XA Active CN119127406B (en) | 2024-09-27 | 2024-09-27 | Residual block device cleaning method, apparatus, computer device, readable storage medium and program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119127406B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050216788A1 (en) * | 2002-11-20 | 2005-09-29 | Filesx Ltd. | Fast backup storage and fast recovery of data (FBSRD) |
| CN108388658A (en) * | 2018-03-07 | 2018-08-10 | 成都创信特电子技术有限公司 | Data file reliable storage method |
| CN111988169A (en) * | 2020-07-29 | 2020-11-24 | 苏州浪潮智能科技有限公司 | Method, system, equipment and medium for cleaning and repairing abnormal disk of cloud platform |
| CN113238719A (en) * | 2021-07-12 | 2021-08-10 | 苏州浪潮智能科技有限公司 | Method, device and equipment for cleaning residual path of host end |
| CN115374065A (en) * | 2022-10-25 | 2022-11-22 | 山东捷瑞数字科技股份有限公司 | File cleaning method and system based on cloud platform log record monitoring |
| CN116962441A (en) * | 2023-06-27 | 2023-10-27 | 腾讯科技（深圳）有限公司 | Cloud storage control method and device and cloud storage system |
| CN117406730A (en) * | 2023-10-20 | 2024-01-16 | 深圳汉阳科技有限公司 | Breakpoint recovery method, device, mobile device and storage medium |
-
2024
- 2024-09-27 CN CN202411360990.XA patent/CN119127406B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050216788A1 (en) * | 2002-11-20 | 2005-09-29 | Filesx Ltd. | Fast backup storage and fast recovery of data (FBSRD) |
| CN108388658A (en) * | 2018-03-07 | 2018-08-10 | 成都创信特电子技术有限公司 | Data file reliable storage method |
| CN111988169A (en) * | 2020-07-29 | 2020-11-24 | 苏州浪潮智能科技有限公司 | Method, system, equipment and medium for cleaning and repairing abnormal disk of cloud platform |
| CN113238719A (en) * | 2021-07-12 | 2021-08-10 | 苏州浪潮智能科技有限公司 | Method, device and equipment for cleaning residual path of host end |
| CN115374065A (en) * | 2022-10-25 | 2022-11-22 | 山东捷瑞数字科技股份有限公司 | File cleaning method and system based on cloud platform log record monitoring |
| CN116962441A (en) * | 2023-06-27 | 2023-10-27 | 腾讯科技（深圳）有限公司 | Cloud storage control method and device and cloud storage system |
| CN117406730A (en) * | 2023-10-20 | 2024-01-16 | 深圳汉阳科技有限公司 | Breakpoint recovery method, device, mobile device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119127406B (en) | 2025-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11169706B2 (en) | Rebalancing storage I/O workloads by storage controller selection and redirection | |
| US10261853B1 (en) | Dynamic replication error retry and recovery | |
| US9372767B2 (en) | Recovery consumer framework | |
| US11221785B2 (en) | Managing replication state for deleted objects | |
| US11409711B2 (en) | Barriers for dependent operations among sharded data stores | |
| WO2021112907A1 (en) | Replication barriers for dependent data transfers between data stores | |
| CN106406758A (en) | Data processing method based on distributed storage system, and storage equipment | |
| US9984139B1 (en) | Publish session framework for datastore operation records | |
| CN110807064A (en) | Data Recovery Device in RAC Distributed Database Cluster System | |
| US20210165760A1 (en) | Managing Dependent Delete Operations among Data Stores | |
| WO2018121456A1 (en) | Data storage method, server and storage system | |
| CN113946471B (en) | Distributed file-level backup method and system based on object storage | |
| CN106034137A (en) | Intelligent scheduling method for distributed system, and distributed service system | |
| US11861165B2 (en) | Object tiering in a distributed storage system | |
| CN105975358A (en) | Fault-tolerant method and system based on SCSI equipment | |
| US9195528B1 (en) | Systems and methods for managing failover clusters | |
| CN111176888A (en) | Cloud storage disaster recovery method, device and system | |
| CN112905480B (en) | Test script generation method, device, storage medium and electronic device | |
| CN115454717B (en) | Database real-time backup method and device, computer equipment and storage medium | |
| CN119127406B (en) | Residual block device cleaning method, apparatus, computer device, readable storage medium and program product | |
| US9952951B2 (en) | Preserving coredump data during switchover operation | |
| CN118656244A (en) | Anomaly detection method, electronic device and storage medium for big data tasks | |
| CN119341899A (en) | Fault detection method, device, storage medium and electronic device | |
| US8954780B1 (en) | Systems and methods for transferring input/output operations within computer clusters | |
| CN108694067A (en) | Method and apparatus for memory space management for multiple virtual machines |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |