CN115357254B - A container image update method and system for cloud-edge application collaboration - Google Patents
A container image update method and system for cloud-edge application collaboration Download PDFInfo
- Publication number
- CN115357254B CN115357254B CN202210797456.XA CN202210797456A CN115357254B CN 115357254 B CN115357254 B CN 115357254B CN 202210797456 A CN202210797456 A CN 202210797456A CN 115357254 B CN115357254 B CN 115357254B
- Authority
- CN
- China
- Prior art keywords
- mirror image
- container mirror
- version container
- file
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/61—Installation
- G06F8/63—Image based installation; Cloning; Build to order
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/60—Software deployment
- G06F8/65—Updates
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F8/00—Arrangements for software engineering
- G06F8/70—Software maintenance or management
- G06F8/71—Version control; Configuration management
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/455—Emulation; Interpretation; Software simulation, e.g. virtualisation or emulation of application or operating system execution engines
- G06F9/45533—Hypervisors; Virtual machine monitors
- G06F9/45558—Hypervisor-specific management and integration aspects
Landscapes
- Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明涉及一种面向云边应用协同的容器镜像更新方法及系统，方法包括：用户发布容器镜像，对容器镜像发布版本与历史版本的重复数据，首先在文件层次上基于容器镜像文件的哈希值进行文件去重，并记录在历史版本容器镜像文件索引中；进而在字节流层次上通过分块算法对文件去重未命中的容器镜像文件进行字节流数据分块，基于数据分块的哈希值进行数据块去重，并记录在历史版本容器镜像数据块索引中；边缘宿主机将镜像部署到容器运行环境，并更新历史版本容器镜像库，同时重建发布版本容器镜像。该方法极大程度的缓解了多版本容器镜像重复数据对网络资源、边缘宿主机资源造成的浪费，尤其适用于容器镜像版本迭代频繁以及边缘设备资源受限的场景。
The present invention relates to a container image update method and system for cloud-edge application collaboration, the method comprising: a user publishes a container image, and for duplicate data of a release version and a historical version of the container image, firstly deduplicates the file based on the hash value of the container image file at the file level, and records it in the historical version container image file index; then, at the byte stream level, byte stream data blocks are performed on the container image file that does not hit the file deduplication through a block algorithm, and data blocks are deduplicated based on the hash value of the data blocks, and recorded in the historical version container image data block index; the edge host deploys the image to the container operating environment, and updates the historical version container image library, and rebuilds the release version container image at the same time. This method greatly alleviates the waste of network resources and edge host resources caused by duplicate data of multi-version container images, and is particularly suitable for scenarios where container image versions are frequently iterated and edge device resources are limited.
Description
Technical Field
The invention relates to the technical field of computer application systems, in particular to a cloud edge application collaboration oriented container mirror image updating method and system.
Background
The container technology is a process-level virtualization technology, and the container mirror image packages applications and library files, running environments and the like depending on the applications. By means of the container technology, lightweight release and iteration of the application can be achieved through deployment and instantiation of container mirror images. Along with the rapid development of cloud technology, typical container products such as Docker, rocket and the like are widely applied to application practices of enterprises, governments, banks and other institutions, meanwhile, due to the promotion of edge computing and the technology of the Internet of things, the container technology gradually sinks from the original ecology of the cloud to the edge, and the container is extremely suitable for application requirements of massive heterogeneous edge equipment due to the light weight and platform independence of the container.
However, in the iteration process of containerization application, the network environment of the edge equipment is complex, and the transmission of the mirror image has higher network requirements, the existing method is even unavailable in a weak network and a narrow-band environment, meanwhile, the quantity of the edge equipment is huge, the increment coefficient of the mass equipment for the quantity of transmitted data not only increases the quantity of network data transmission in a multiplied way, but also generates a great load for the edge mirror image deployment equipment of the cloud.
Because the container mirror images are packed with application dependencies such as operating system library files and middleware, a large amount of repeated data exist in the container mirror images of different applications and the container mirror images of different versions of the same application, the industry and academia propose a large amount of schemes for the repeated data in the container mirror images in the fields of storage, transmission and the like, for example, in the container mirror image library, the repeated files are eliminated through image file hash values, the storage space is saved, in the edge container deployment process, the image files are replaced by transmitting edge container mirror image indexes, and the image is reconstructed through a shared file library at the edge equipment side, so that the data transmission quantity is reduced.
The existing method for updating the multi-version container mirror image of the edge equipment can have the problems that (1) the granularity of repeated data screening is too large, a large amount of repeated data still exists in the multi-version container mirror image based on mirror image layer duplication removal and file duplication removal, the total amount of network transmission data of the repeated data cannot be ignored under the multiplication of the increment coefficient of the mass edge equipment, huge waste is caused to network resources of the edge hosts, (2) the file mirror image is rebuilt through transmission indexes, under the condition that the calculation and storage resources of the edge hosts are limited, if a local metafile library is small, files required by the index are difficult to ensure to exist, the deletion needs to be reacquired from the mirror image library, the starting efficiency of the container is influenced, if the local metafile library is large, the consumption of the storage resources is large, and the searching process generates large calculation cost, and (3) under the condition that the container mirror image data transmission amount of the container mirror image is not reduced to the maximum extent, and the concurrent pressure of the container mirror image management server is large due to the huge number of the edge equipment.
In summary, in order to solve the problem of the load of cloud edge equipment caused by huge quantity of edge equipment, the cloud edge application collaborative mirror image updating method is introduced, the operation and the duplication are removed at the file level, the duplication is removed at the data block level, the concurrency pressure of a server is relieved, meanwhile, a real file is not required to be acquired through multiple interactions with a mirror image management server, and the starting efficiency of a container is improved.
Disclosure of Invention
The invention aims to overcome the defects and shortcomings of the prior art and provides a cloud-edge application-oriented collaborative container mirror image updating method and system, which relieve the concurrency pressure of a cloud end.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the cloud edge application collaboration-oriented container mirror image updating method is characterized by comprising the following steps of:
mounting a release version container mirror image through a container mirror image management server, mounting a history version container mirror image of the release version container mirror image, and acquiring a file index of the history version container mirror image;
Performing hash operation on all files of the release version container mirror image to obtain a first hash value, constructing a release version container mirror image file index based on the obtained first hash value, performing file deduplication operation on the release version container mirror image by combining the history version container mirror image file index, and replacing the release version container mirror image duplicate file with a file identifier;
Obtaining byte stream data blocks of missed files in the historical version container mirror image through a blocking algorithm, carrying out hash operation on the byte stream data blocks to obtain a second hash value, constructing a historical version container mirror image data block index based on the obtained second hash value, obtaining byte stream data blocks of files which are not replaced by the file identifiers in the release version container mirror image through the blocking algorithm, carrying out hash operation on the byte stream data blocks to obtain a third hash value, and carrying out data block deduplication operation on the release version container mirror image based on the obtained third hash value;
The container mirror image management server updates a history version container mirror image library and updates a history version container mirror image file index library;
Performing redundancy elimination operation on the historical version container mirror image file index and the historical version container mirror image data block index, packaging the historical version container mirror image file index and the published version container mirror image, and transmitting the packaged historical version container mirror image file index and the published version container mirror image data block index to an edge host;
The edge host receives the release version container mirror image, mounts the release version container mirror image, and mounts the history version container mirror image of the release version container mirror image;
The edge host checks whether the file identifier exists in the published version container mirror image, if so, the file corresponding to the historical version container mirror image is used for replacing, and checks whether the data block identifier exists in the published version container mirror image, if so, the data block corresponding to the historical version container mirror image is used for replacing;
and verifying the published version container mirror image, deploying the mirror image into a container running environment, deleting the local historical version container mirror image, and updating a local historical version container mirror image library.
Preferably, the cloud-edge application-oriented collaborative container image updating method is characterized in that a historical version container image of the published version container image is mounted, and a file index of the historical version container image is obtained, specifically:
And inquiring and mounting a corresponding historical version container mirror image in a historical version container mirror image library based on the published version container mirror image element identification, and extracting a corresponding historical version container mirror image file index in a historical version container mirror image file index library.
Preferably, the cloud-edge application-oriented collaborative container image updating method is characterized in that the file deduplication operation is performed on the published version container image by combining the historical version container image file index, and specifically comprises the following steps:
And inquiring the first hash value in the file index of the historical version container image, if the first hash value is hit, replacing the file corresponding to the release version container image with a file identifier, and marking the file identifier as hit in the file index of the historical version container image.
Preferably, the cloud-edge application-oriented collaborative container image updating method is characterized in that the data block deduplication operation is performed on the release version container image based on the obtained third hash value, and specifically comprises the following steps:
And inquiring the third hash value in the historical version container mirror image data block index, if the third hash value is hit, replacing the byte stream data block corresponding to the issuing version container mirror image with a data block identifier, and marking the data block identifier as hit in the historical version container mirror image data block index.
Preferably, the cloud-edge application-oriented collaborative container mirror image updating method is characterized in that a historical version container mirror image library is updated, and a historical version container mirror image file index library is updated, specifically:
And replacing the historical version container mirror image with the release version container mirror image in the historical version container mirror image library, and replacing the historical version container mirror image file index with the release version container mirror image file index in the historical version container mirror image file index library.
Preferably, the cloud-edge application-oriented collaborative container image updating method is characterized in that performing redundancy elimination operation on the historical version container image file index and the historical version container image data block index comprises the following steps:
Deleting the missed file index in the historical version container mirror image file index, and deleting the missed data block index in the historical version container mirror image data block index.
Preferably, the cloud-edge application-oriented collaborative container image updating method is characterized in that a historical version container image of the release version container image is locally mounted, specifically:
and the edge host queries and mounts the corresponding historical version container mirror image in a local historical version container mirror image library based on the meta identification of the release version container mirror image.
Preferably, the cloud-edge application-oriented collaborative container mirror image updating method is characterized in that if the cloud-edge application-oriented collaborative container mirror image updating method exists, file replacement corresponding to the historical version container mirror image is used, specifically, the file index of the historical version container mirror image is queried based on the file identification, the file corresponding to the file identification is obtained from the historical version container mirror image, and the file identification is replaced;
And if so, replacing the data block corresponding to the historical version container mirror image by using the data block corresponding to the data block identifier, namely inquiring the historical version container mirror image data block index based on the data block identifier, and acquiring the data block corresponding to the data block identifier from the historical version container mirror image to replace the data block identifier.
Preferably, the cloud-edge application-oriented collaborative container mirror image updating method is characterized by updating a local historical version container mirror image library, and particularly replacing the historical version container mirror image with the release version container mirror image in the local historical version container mirror image library.
The cloud edge application collaboration-oriented container mirror image updating system is characterized by comprising the following modules:
The server container mirror image processing module is used for mounting the release version container mirror image through the container mirror image management server, mounting the history version container mirror image of the release version container mirror image, and acquiring the file index of the history version container mirror image;
the server file processing module is used for carrying out hash operation on all files of the release version container mirror image to obtain a first hash value, constructing a release version container mirror image file index based on the obtained first hash value, carrying out file deduplication operation on the release version container mirror image by combining the history version container mirror image file index, and replacing the release version container mirror image duplicate file with a file identifier;
The server data block processing module is used for obtaining byte stream data blocks of missed files in the historical version container mirror image through a blocking algorithm, carrying out hash operation on the byte stream data blocks to obtain a second hash value, constructing a historical version container mirror image data block index based on the obtained second hash value, obtaining byte stream data blocks of files which are not replaced by the file identification in the release version container mirror image through the blocking algorithm, carrying out hash operation on the byte stream data blocks to obtain a third hash value, and carrying out data block deduplication operation on the release version container mirror image based on the obtained third hash value;
The server history version management module is used for updating the history version container mirror image library and updating the history version container mirror image file index library;
The edge host machine container mirror image processing module is used for receiving the release version container mirror image, mounting the release version container mirror image, checking the release version container mirror image and deploying the mirror image into a container running environment;
the file processing module of the edge host is used for judging whether the file identifier exists in the published version container mirror image or not, and if so, replacing the file corresponding to the historical version container mirror image;
the edge host data management module is used for checking whether the published version container mirror image has the data block identifier or not, and if so, replacing the data block corresponding to the historical version container mirror image;
And the edge host history version management module is used for extracting a corresponding history version container mirror image from the local history version container mirror image library according to the issued version container mirror image element identifier, deleting the local history version container mirror image and updating the local history version container mirror image library.
Compared with the prior art, the invention has the following advantages and beneficial effects:
(1) According to the invention, repeated data are repeated aiming at the multi-version edge container mirror image, the repeated operation is performed not only on the file level, but also on the byte stream data block level, so that the data quantity required to be transmitted by the release container mirror image can be compressed to a great extent, cloud edge network resources are saved, and the concurrency pressure of an edge container management server in the cloud is relieved;
(2) According to the invention, the edge host can reconstruct the published version container mirror image by combining the published version container mirror image obtained by file index, data block index and file deduplication and data block deduplication with the local historical version container mirror image, and the missing file is not required to be acquired through multiple interactions with the mirror image management server, so that the container starting efficiency is high;
(3) According to the invention, the edge device does not need to maintain a local metafile library, so that the storage resource of the edge device is saved to a great extent, and the historical version container mirror image is searched only through an index in the process of reconstructing the release version mirror image, so that the computing resource of the edge device is saved to a great extent.
In summary, the method only exchanges the computing, storage and network resource costs of the mass edge devices for one-time computing costs through the container mirror image management server, and relieves the concurrency pressure of the cloud.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 is a flowchart of a cloud-edge application collaboration oriented container image updating method according to an embodiment of the present invention.
Fig. 2 is a cloud-edge application-oriented collaborative container mirror update system model diagram according to an embodiment of the present invention.
Fig. 3 is a diagram illustrating an embodiment of updating a container image for cloud-edge application collaboration according to an embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the technical solutions of the present application, the following description will further describe the technical solutions of the embodiments of the present application in detail with reference to the attached drawings and figures of the embodiments, but the embodiments of the present application are not limited thereto. All other embodiments, based on the embodiments of the application, which are obtained by a person skilled in the art without the innovative teachings are intended to be within the scope of the application.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearances of such phrases in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of skill in the art will explicitly and implicitly appreciate that the described embodiments of the application may be combined with other embodiments.
As shown in fig. 1, the method for updating a container image for cloud edge application collaboration in this embodiment includes the following steps:
S1, a user issues a container mirror image through a container mirror image management server, the container mirror image management server mounts the container mirror image of the issued version, mounts the container mirror image of the history version of the container mirror image of the issued version, and acquires a file index of the container mirror image of the history version.
Further, the historical version container mirror image of the release version container mirror image is mounted, and a file index of the historical version container mirror image is obtained, specifically:
And inquiring and mounting a corresponding historical version container mirror image in a historical version container mirror image library based on the published version container mirror image element identification, and extracting a corresponding historical version container mirror image file index in a historical version container mirror image file index library.
It can be understood that a user can issue a Docker mirror image to a plurality of raspberry pies, an edge gateway and an intelligent terminal through a container mirror image management server, after the container mirror image management server receives a request for updating the container mirror image issued by the user, a container mirror image processing module in the container mirror image management server can locally mount a release version Docker mirror image sent by the user, and call a server history version management module to extract a history version Docker mirror image corresponding to the local mount in a history version container mirror image library through a meta identifier, and extract a history version Docker mirror image file index in a history version container mirror image file index library.
S2, carrying out hash operation on all files of the release version container mirror image to obtain a first hash value, constructing a release version container mirror image file index based on the obtained first hash value, carrying out file deduplication operation on the release version container mirror image by combining the history version container mirror image file index, and replacing the release version container mirror image duplicate file with a file identifier.
Further, the file deduplication operation is performed on the published version container image by combining the historical version container image file index, specifically:
And inquiring the first hash value in the file index of the historical version container image, if the first hash value is hit, replacing the file corresponding to the release version container image with a file identifier, and marking the file identifier as hit in the file index of the historical version container image.
Furthermore, the container image management service file processing module calculates a first hash value of the published version Docker image file through an SHA-1 algorithm, namely, the first hash value is used for providing the integrity protection of the information, and constructs a published version container image file index based on the obtained first hash value, the purpose of constructing the index is that hash operation is not needed to be carried out on the version container image file again when the next container version is published, after the file index is constructed, the first hash value is queried in a historical version container image file index, if the file index is hit, the corresponding file of the published version container image is replaced by a file identifier, the file identifier comprises an SHA-1 abstract, and meanwhile, the file is marked as hit in the historical version container image file index.
S3, obtaining byte stream data blocks of missed files in the historical version container mirror image through a blocking algorithm, carrying out hash operation on the byte stream data blocks to obtain a second hash value, constructing a historical version container mirror image data block index based on the obtained second hash value, obtaining byte stream data blocks of files which are not replaced by the file identification in the release version container mirror image through the blocking algorithm, carrying out hash operation on the byte stream data blocks to obtain a third hash value, and carrying out data block deduplication operation on the release version container mirror image based on the obtained third hash value.
Further, performing data block deduplication operation on the release version container image based on the obtained third hash value, specifically:
And inquiring the third hash value in the historical version container mirror image data block index, if the third hash value is hit, replacing the byte stream data block corresponding to the issuing version container mirror image with a data block identifier, and marking the data block identifier as hit in the historical version container mirror image data block index.
Specifically, the server data block processing module performs blocking on a missed file in the historical version Docker image through an AE blocking algorithm, the blocking algorithm generates a variable-length content block based on a byte stream data blocking extremum, byte drift problems caused by adding, deleting and modifying the file content can be effectively solved, the data block is subjected to SHA-1 operation to obtain a second hash value, a historical version container image data block index is constructed based on the obtained second hash value, byte stream data blocks are obtained on a file which is not replaced by the file identifier in the published version Docker image through an AE blocking algorithm, the byte stream data blocks are subjected to hash operation to obtain a third hash value, the data block deduplication operation is performed on the published version container image based on the obtained third hash value, the hit data block is replaced by the data block identifier in the published version container image, and the historical version container image data block index is marked.
And S4, the container mirror image management server updates a history version container mirror image library and updates a history version container mirror image file index library.
Further, the historical version container mirror image library is updated, and the historical version container mirror image file index library is updated, specifically:
And replacing the historical version container mirror image with the release version container mirror image in the historical version container mirror image library, and replacing the historical version container mirror image file index with the release version container mirror image file index in the historical version container mirror image file index library.
Further, the server history version management module updates a history version container mirror image library and a history version container mirror image file index library, replaces the history version container mirror image with a release version container mirror image in the history version container mirror image library, and replaces the history version container mirror image file index with a release version container mirror image file index in the history version container mirror image file index library.
S5, performing redundancy elimination operation on the historical version container image file index and the historical version container image data block index, packaging the historical version container image file index and the published version container image, and transmitting the packaged historical version container image file index and the published version container image data block index to an edge host.
Further, performing redundancy elimination operation on the historical version container image file index and the historical version container image data block index, including:
Deleting the missed file index in the historical version container mirror image file index, and deleting the missed data block index in the historical version container mirror image data block index.
Further, the server container mirror image processing module performs redundancy elimination operation on the historical version container mirror image file index and the historical version container mirror image data block index, and includes deleting the missed file index in the historical version container mirror image file index, deleting the missed data block index in the historical version container mirror image data block index, and packaging and publishing the version mirror image to transmit to each raspberry group, edge gateway and intelligent terminal, as shown in fig. 3.
S6, the edge host receives the release version container mirror image, mounts the release version container mirror image, and mounts the history version container mirror image of the release version container mirror image.
Further, the history version container mirror image of the release version container mirror image is locally mounted, specifically:
and the edge host queries and mounts the corresponding historical version container mirror image in a local historical version container mirror image library based on the meta identification of the release version container mirror image.
Furthermore, in each raspberry group, the edge gateway and the intelligent terminal, the edge host machine container mirror image processing module receives and locally mounts the historical version container mirror image of the release version container mirror image, and the edge host machine queries and mounts the corresponding historical version container mirror image in the local historical version container mirror image library based on the release version container mirror image element identification.
S7, the edge host checks whether the file identification exists in the published version container mirror image, if so, the file corresponding to the historical version container mirror image is used for replacing, and checks whether the data block identification exists in the published version container mirror image, and if so, the data block corresponding to the historical version container mirror image is used for replacing.
Further, if the file exists, replacing the file corresponding to the historical version container mirror image, specifically, inquiring the file index of the historical version container mirror image based on the file identification, and acquiring the file corresponding to the file identification from the historical version container mirror image to replace the file identification.
And if so, replacing the data block corresponding to the historical version container mirror image by using the data block corresponding to the data block identifier, namely inquiring the historical version container mirror image data block index based on the data block identifier, and acquiring the data block corresponding to the data block identifier from the historical version container mirror image to replace the data block identifier.
Further, the file processing module of the edge host queries the file index of the historical version container image based on the file identification, acquires the file corresponding to the file identification from the historical version container image, and replaces the file identification; and if so, using the data block corresponding to the historical version container mirror image to replace, specifically, inquiring the data block index of the historical version container mirror image based on the data block identification, acquiring the data block corresponding to the data block identification from the historical version container mirror image, replacing the data block identification, and checking whether the data block identification exists in the published version container mirror image by using the data block processing module of the edge host machine, and if so, using the data block corresponding to the historical version container mirror image to replace.
S8, verifying the published version container mirror image, deploying the mirror image into a container running environment, deleting the local historical version container mirror image, and updating a local historical version container mirror image library.
Further, the local history version container mirror library is updated, specifically:
and replacing the historical version container image with the release version container image in the local historical version container image library.
Furthermore, the edge host machine mirror image processing module checks the mirror image of the release version container to ensure the integrity and the accuracy of the mirror image of the release version container, deploys the mirror image to the container running environment, deletes the mirror image of the local history version container, and finally replaces the mirror image of the history version container with the mirror image of the release version container in the mirror image library of the local history version container and updates the mirror image library of the local history version container.
Based on the analysis, the updating of the container mirror image not only carries out the duplication removal on the file level, but also carries out the duplication removal operation on the byte stream data block, thereby compressing the data quantity required to be transmitted for releasing the container mirror image, saving cloud network resources, and simultaneously, only needing to exchange the once calculation cost of the container mirror image management server for the calculation, storage and network resource cost of mass edge equipment, and relieving the concurrency pressure of the cloud.
As shown in fig. 2, in another embodiment of the present invention, a cloud-edge application-oriented collaborative container image update system is further provided, where the system may be configured to execute the above-mentioned cloud-edge application-oriented collaborative container image update method, and the system includes the following modules:
The server container mirror image processing module is used for mounting the release version container mirror image through the container mirror image management server, mounting the history version container mirror image of the release version container mirror image, and acquiring the file index of the history version container mirror image;
the server file processing module is used for carrying out hash operation on all files of the release version container mirror image to obtain a first hash value, constructing a release version container mirror image file index based on the obtained first hash value, carrying out file deduplication operation on the release version container mirror image by combining the history version container mirror image file index, and replacing the release version container mirror image duplicate file with a file identifier;
The server data block processing module is used for obtaining byte stream data blocks of missed files in the historical version container mirror image through a blocking algorithm, carrying out hash operation on the byte stream data blocks to obtain a second hash value, constructing a historical version container mirror image data block index based on the obtained second hash value, obtaining byte stream data blocks of files which are not replaced by the file identification in the release version container mirror image through the blocking algorithm, carrying out hash operation on the byte stream data blocks to obtain a third hash value, and carrying out data block deduplication operation on the release version container mirror image based on the obtained third hash value;
The server history version management module is used for updating the history version container mirror image library and updating the history version container mirror image file index library;
The edge host machine container mirror image processing module is used for receiving the release version container mirror image, mounting the release version container mirror image, checking the release version container mirror image and deploying the mirror image into a container running environment;
the file processing module of the edge host is used for judging whether the file identifier exists in the published version container mirror image or not, and if so, replacing the file corresponding to the historical version container mirror image;
the edge host data management module is used for checking whether the published version container mirror image has the data block identifier or not, and if so, replacing the data block corresponding to the historical version container mirror image;
And the edge host machine history version management module extracts a corresponding history version container mirror image from a local history version container mirror image library according to the published version container mirror image element identifier, deletes the local history version container mirror image and updates the local history version container mirror image library.
It should be noted that, the above technical features and beneficial effects described in the embodiments of the cloud edge application-oriented collaborative container mirror image updating method are applicable to the embodiments of the cloud edge application-oriented collaborative container mirror image updating system, and specific content may be referred to the description of the embodiments of the method of the present invention, which is not repeated herein, so that it is stated here.
In addition, in the embodiment of the above-mentioned cloud-edge application-oriented collaborative container mirror update system, the logic division of each module is merely illustrative, and in practical application, the above-mentioned function allocation may be performed by different modules according to needs, for example, in view of configuration requirements of corresponding hardware or convenience of implementation of software, that is, the internal structure of the above-mentioned cloud-edge application-oriented collaborative container mirror update system is divided into different modules, so as to perform all or part of the above-mentioned functions.
Those skilled in the art will appreciate that all or part of the processes in the methods of the above embodiments may be implemented by a computer program for instructing relevant hardware, where the program may be stored in a non-volatile computer readable storage medium, and where the program, when executed, may include processes in the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (SYNCHLINK) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above examples, and any other changes, modifications, substitutions, combinations, and simplifications that do not depart from the spirit and principle of the present invention should be made in the equivalent manner, and the embodiments are included in the protection scope of the present invention.
Claims (9)
1. The cloud edge application collaboration-oriented container mirror image updating method is characterized by comprising the following steps of:
mounting a release version container mirror image through a container mirror image management server, mounting a history version container mirror image of the release version container mirror image, and acquiring a file index of the history version container mirror image;
Performing hash operation on all files of the release version container mirror image to obtain a first hash value, constructing a release version container mirror image file index based on the obtained first hash value, performing file deduplication operation on the release version container mirror image by combining the file index of the history version container mirror image, and replacing repeated files of the release version container mirror image with file identifications;
Obtaining byte stream data blocks of missed files in the historical version container mirror image through a blocking algorithm, carrying out hash operation on the byte stream data blocks to obtain a second hash value, constructing a historical version container mirror image data block index based on the obtained second hash value, obtaining byte stream data blocks of files which are not replaced by the file identifiers in the release version container mirror image through the blocking algorithm, carrying out hash operation on the byte stream data blocks to obtain a third hash value, and carrying out data block deduplication operation on the release version container mirror image based on the obtained third hash value;
and carrying out data block deduplication operation on the published version container image based on the acquired third hash value, wherein the data block deduplication operation specifically comprises the following steps:
inquiring the third hash value in the historical version container mirror image data block index, if the third hash value is hit, replacing the byte stream data block corresponding to the release version container mirror image with a data block identifier, and marking the data block identifier as hit in the historical version container mirror image data block index;
The container mirror image management server updates a history version container mirror image library and updates a history version container mirror image file index library;
Performing redundancy elimination operation on the file index of the historical version container mirror image and the data block index of the historical version container mirror image, packaging the file index and the data block index of the historical version container mirror image with the release version container mirror image, and transmitting the packaged file index and the data block index of the historical version container mirror image to an edge host;
The edge host receives the release version container mirror image, mounts the release version container mirror image, and mounts the history version container mirror image of the release version container mirror image;
The edge host checks whether the file identifier exists in the published version container mirror image, if so, the file corresponding to the historical version container mirror image is used for replacing, and checks whether the data block identifier exists in the published version container mirror image, if so, the data block corresponding to the historical version container mirror image is used for replacing;
And verifying the published version container mirror image, deploying the mirror image into a container running environment, deleting the local historical version container mirror image, and updating a local historical version container mirror image library.
2. The cloud-edge application-oriented collaborative container image updating method according to claim 1, wherein the history version container image of the release version container image is mounted, and a file index of the history version container image is obtained, specifically:
and inquiring and mounting a corresponding historical version container mirror image in a historical version container mirror image library based on the meta-identification of the release version container mirror image, and extracting a file index of the corresponding historical version container mirror image from a file index library of the historical version container mirror image.
3. The cloud-edge application-oriented collaborative container image updating method according to claim 1, wherein the file deduplication operation is performed on the release version container image in combination with the file index of the history version container image, specifically:
and inquiring the first hash value in the file index of the historical version container image, if the first hash value is hit, replacing the file corresponding to the release version container image with a file identifier, and marking the file identifier as hit in the file index of the historical version container image.
4. The cloud-edge application-oriented collaborative container mirror updating method according to claim 1, wherein the updating of the historical version container mirror library and the updating of the historical version container mirror file index library are specifically as follows:
And replacing the file index of the historical version container image with the file index of the release version container image in the historical version container image file index library.
5. The cloud-edge application-oriented collaborative container image updating method according to claim 1, wherein performing redundancy elimination on the file index of the historical version container image and the historical version container image data block index comprises:
deleting the missed file index in the file index of the historical version container mirror image, and deleting the missed data block index in the data block index of the historical version container mirror image.
6. The cloud-edge application-oriented collaborative container image updating method according to claim 1, wherein the history version container image of the release version container image is mounted, specifically:
and the edge host queries and mounts the corresponding historical version container mirror image in a local historical version container mirror image library based on the meta identification of the release version container mirror image.
7. The cloud-edge application-oriented collaborative container image updating method according to claim 1 is characterized in that if the cloud-edge application-oriented collaborative container image updating method exists, file replacement corresponding to the historical version container image is used, specifically, a file index of the historical version container image is queried based on the file identification, a file corresponding to the file identification is obtained in the historical version container image, and the file identification is replaced;
And if so, replacing the data block corresponding to the historical version container mirror image by using the data block corresponding to the data block identifier, namely inquiring the historical version container mirror image data block index based on the data block identifier, and acquiring the data block corresponding to the data block identifier from the historical version container mirror image to replace the data block identifier.
8. The cloud-edge application-oriented collaborative container image updating method according to claim 1 is characterized by updating a local historical version container image library, specifically by replacing the historical version container image with the release version container image in the local historical version container image library.
9. The cloud edge application collaboration-oriented container mirror image updating system is characterized by comprising the following modules:
The server container mirror image processing module is used for mounting the release version container mirror image through the container mirror image management server, mounting the history version container mirror image of the release version container mirror image, and acquiring the file index of the history version container mirror image;
the server file processing module is used for carrying out hash operation on all files of the release version container mirror image to obtain a first hash value, constructing a release version container mirror image file index based on the obtained first hash value, carrying out file deduplication operation on the release version container mirror image by combining the file index of the history version container mirror image, and replacing repeated files of the release version container mirror image with file identifiers;
The server data block processing module is used for obtaining byte stream data blocks of missed files in the historical version container mirror image through a blocking algorithm, carrying out hash operation on the byte stream data blocks to obtain a second hash value, constructing a historical version container mirror image data block index based on the obtained second hash value, obtaining byte stream data blocks of files which are not replaced by the file identification in the release version container mirror image through the blocking algorithm, carrying out hash operation on the byte stream data blocks to obtain a third hash value, carrying out data block deduplication operation on the release version container mirror image based on the obtained third hash value, and carrying out data block deduplication operation on the release version container mirror image based on the obtained third hash value, wherein the method specifically comprises the following steps:
inquiring the third hash value in the historical version container mirror image data block index, if the third hash value is hit, replacing the byte stream data block corresponding to the release version container mirror image with a data block identifier, and marking the data block identifier as hit in the historical version container mirror image data block index;
The server history version management module is used for updating the history version container mirror image library and updating the history version container mirror image file index library;
The edge host machine container mirror image processing module is used for receiving the release version container mirror image, mounting the release version container mirror image, checking the release version container mirror image and deploying the mirror image into a container running environment;
the file processing module of the edge host is used for judging whether the file identifier exists in the published version container mirror image or not, and if so, replacing the file corresponding to the historical version container mirror image;
the edge host data management module is used for checking whether the published version container mirror image has the data block identifier or not, and if so, replacing the data block corresponding to the historical version container mirror image;
and the edge host history version management module is used for extracting a corresponding history version container mirror image from a local history version container mirror image library according to the meta identification of the release version container mirror image, deleting the local history version container mirror image and updating the local history version container mirror image library.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210797456.XA CN115357254B (en) | 2022-07-08 | 2022-07-08 | A container image update method and system for cloud-edge application collaboration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210797456.XA CN115357254B (en) | 2022-07-08 | 2022-07-08 | A container image update method and system for cloud-edge application collaboration |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115357254A CN115357254A (en) | 2022-11-18 |
| CN115357254B true CN115357254B (en) | 2025-04-25 |
Family
ID=84030240
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210797456.XA Active CN115357254B (en) | 2022-07-08 | 2022-07-08 | A container image update method and system for cloud-edge application collaboration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115357254B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115941784B (en) * | 2022-11-30 | 2024-07-30 | 天翼云科技有限公司 | Method and system for reliably downloading large files in edge weak network environment |
| CN117369947B (en) * | 2023-10-26 | 2024-08-30 | 深圳海规网络科技有限公司 | A container image management method and management system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111324360A (en) * | 2020-01-09 | 2020-06-23 | 华中科技大学 | Container mirror image construction method facing edge calculation |
| CN112559133A (en) * | 2020-12-22 | 2021-03-26 | 北京滴普科技有限公司 | Cloud-edge coordination system and cloud-edge coordination method based on native container technology |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112559452B (en) * | 2020-12-11 | 2021-12-17 | 北京云宽志业网络技术有限公司 | Data deduplication processing method, device, equipment and storage medium |
| CN114296933B (en) * | 2021-12-30 | 2024-10-11 | 中国电信股份有限公司 | Implementation method of lightweight container under end-edge cloud architecture and data processing system |
| CN114625316A (en) * | 2022-02-11 | 2022-06-14 | 华南理工大学 | Content-based blocking method, system and medium applied to data de-duplication |
-
2022
- 2022-07-08 CN CN202210797456.XA patent/CN115357254B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111324360A (en) * | 2020-01-09 | 2020-06-23 | 华中科技大学 | Container mirror image construction method facing edge calculation |
| CN112559133A (en) * | 2020-12-22 | 2021-03-26 | 北京滴普科技有限公司 | Cloud-edge coordination system and cloud-edge coordination method based on native container technology |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115357254A (en) | 2022-11-18 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10983868B2 (en) | Epoch based snapshot summary | |
| Dobre et al. | Hybris: Robust hybrid cloud storage | |
| US9501545B2 (en) | System and method for caching hashes for co-located data in a deduplication data store | |
| US10216949B1 (en) | Dynamic quorum membership changes | |
| US20160077926A1 (en) | System and method for multi-hop data backup | |
| US20140101113A1 (en) | Locality Aware, Two-Level Fingerprint Caching | |
| CN115357254B (en) | A container image update method and system for cloud-edge application collaboration | |
| Liu et al. | Cfs: A distributed file system for large scale container platforms | |
| CN111723056A (en) | Small file processing method, device, device and storage medium | |
| CN113157645B (en) | Cluster data migration method, device, equipment and storage medium | |
| EP3963461B1 (en) | Garbage collection for a deduplicated cloud tier | |
| CN103067519A (en) | Method and device of data distribution storage under heterogeneous platform | |
| Cruz et al. | A scalable file based data store for forensic analysis | |
| US10628391B1 (en) | Method and system for reducing metadata overhead in a two-tier storage architecture | |
| US10387384B1 (en) | Method and system for semantic metadata compression in a two-tier storage system using copy-on-write | |
| CN115203255A (en) | Data query method and device, electronic equipment and storage medium | |
| US20040117437A1 (en) | Method for efficient storing of sparse files in a distributed cache | |
| US20220283902A1 (en) | Writing data blocks directly to object storage | |
| CN106776795A (en) | Method for writing data and device based on Hbase databases | |
| CN116975095A (en) | Data storage method, data reading method, data processing system and storage medium | |
| Al-Masadeh et al. | Tiny datablock in saving Hadoop distributed file system wasted memory. | |
| CN119847673A (en) | Docker container mirror image management method | |
| US11151082B1 (en) | File system operation cancellation | |
| Krishna et al. | Improving performance of a distributed file system using a speculative semantics-based algorithm | |
| Zhang et al. | SimpleSync: A parallel delta synchronization method based on Flink |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |