CN107197022B - OpenStack storage optimization method and system - Google Patents

Abstract

The invention discloses an OpenStack storage optimization method and system, and on one hand, the invention provides an OpenStack storage optimization method, which comprises the following processing steps: step 1: OpenStack takes the Ceph RBD as a uniform storage back end of a mirror image service Glance, a computing service Nova and a block storage service circular; step 2: OpenStack stores the mirror image service Glance mirror image in a Ceph RBD in a layered mode to form a mirror image pool containing a system mirror image and a software mirror image, and meanwhile, the virtual machine is started in the system mirror image, full backup is conducted on the virtual machine, the virtual machine is started in the software mirror image, and incremental backup is conducted on the virtual machine, so that different user requirements are met. In another aspect, the invention provides an OpenStack storage optimization system. The OpenStack unified block storage system is constructed based on the Ceph RBD, and the mirror image layered storage is realized on the basis, so that the storage space can be effectively saved, the data storage reliability is improved, the maintenance cost is reduced, and different user requirements are met.

Description

OpenStack storage optimization method and system

technical field

The invention relates to the field of cloud computing, in particular to an OpenStack storage optimization method, and also to an OpenStack storage optimization system.

Background technique

Cloud computing is a new computing model developed from virtualization technology and distributed computing. Cloud computing has a very rapid development momentum and is widely used in transportation, communication, engineering, education, medical care and e-commerce. "Transformation" has become a new development trend. The architecture model of cloud computing is mainly divided into three layers: infrastructure as a service (IaaS), platform as a service (PaaS) and software as a service (SaaS), which can provide users with a service ranging from IT resources to software services, network applications, data storage Platform for the delivery of different types of services.

Ceph is an open source unified distributed storage system and one of the most mainstream open source storage projects. Ceph has outstanding advantages, for example, it can provide three storage access methods of object/block/file system to meet various application requirements; support data storage of PB level and above, multiple backups, no central structure, no single point of failure, good scalability, etc. The biggest innovation and advantage of Ceph is that data addressing adopts the CRUSH algorithm, which is more efficient and reliable than the addressing methods of other storage systems. In recent years, Ceph has moved closer to cloud computing and has gradually become a solution to replace traditional storage in cloud computing environments. Ceph can provide external local storage interfaces and high-level application interfaces. The high-level application interfaces include RBD, RGW, and CephFS, providing an upper-level interface with a higher abstraction level that is more convenient for applications or clients to use. RBD is a block storage device that supports snapshot and clone operations. RBD snapshot refers to saving the state of an RBD image at a certain moment and generating a read-only image copy. RBD snapshots actually only update the metadata of the image. RBD clone refers to the use of copy-on-write (COW) technology to quickly restore a new RBD image from a snapshot. The original snapshot is called a "parent image", and the RBD clone image is called a "child image". There is a dependency relationship between the parent and child mirrors. The child mirror does not occupy any storage space at the beginning and shares all the data of the parent mirror. When performing a read operation, first traverse the parent image to find it. When a write operation is performed, the object is first copied from the parent image to the subimage, and then modified, and the parent image is not directly manipulated.

OpenStack is one of the most active cloud computing open source projects. It can use virtualization technology to integrate and manage hardware resources, form a large resource pool, and provide IaaS services. Users can build a private cloud platform based on OpenStack to provide services to the enterprise, or build a public cloud platform to provide cloud services to the outside world. OpenStack adopts a modular design, and each module has unique functions. Most modules themselves do not provide storage functions and rely on the support of the back-end storage system. Generally, they can be regarded as storage agents, which mainly play the role of service process control and management. In order to avoid constraints when deploying enterprise or personal cloud platforms, OpenStack is designed to support a variety of back-end storage technologies. This design optimizes its applicability and scalability, but causes many other problems. For example, cloud platform system maintenance costs are high, interaction performance between components is poor, and data storage reliability is poor. In addition, more and more users choose to deploy application services on virtual machines in the cloud platform, and the image storage method of the image service component Glance cannot well support users' software service requirements, which are embodied in two aspects:

Users need to manually deploy the required software, which is inconvenient to use; the same system image data in different images is stored repeatedly, resulting in a waste of storage space.

SUMMARY OF THE INVENTION

The first objective of the present invention is to provide an OpenStack storage optimization method, which can effectively save storage space, improve data storage reliability, reduce maintenance costs, and meet different user needs.

The second objective of the present invention is to provide a system of the method.

In order to achieve the first objective, an OpenStack storage optimization method is provided, and the method includes the following processing steps:

Step 1: OpenStack uses Ceph RBD as a unified storage backend for mirroring service Glance, computing service Nova and block storage service Cinder;

Step 2: OpenStack stores the image service Glance image layered in Ceph RBD to form an image pool containing system images and software images, and supports starting virtual machines in system images and performing full backup of virtual machines and software images. Start virtual machines And incremental backup of virtual machines to meet different user needs.

Preferably, in step 1, in order to implement Ceph RBD as the unified storage backend of the mirroring service Glance, the computing service Nova and the block storage service Cinder, the following processing is specifically performed:

Step 101: Deploy the Ceph cluster so that OpenStack can access the Ceph RBD storage service, that is, use the OpenStack computing node as the RBD client; create three pools named the first pool images pool, the second pool volumes pool and the third pool vms in the Ceph cluster Pool's Ceph storage pool RBD pool is used to store image Glanceimage, cloud hard disk Cinder volume and virtual machine disk image file Nova disk; Ceph uses cephx authentication method to create new Ceph user names for Glance, Cinder, and Nova, respectively, for new image services glance and the new block storage service cinder, and set user access rights;

Step 102: Glance is integrated with RBD. In the process of uploading the image, the image service application programming interface Glance API calls the RBD storage backend, creates the RBD image storage block, writes the image data, and also modifies the glance configuration file. It supports RBD storage;

Step 103: Cinder is integrated with RBD. In the process of creating a cloud disk, the volume cindervolume of the new block storage calls the RBD backend storage to create an RBD image storage block; if it is a bootable cloud disk, write to the RBD image block Mirror data; if it is a data cloud hard disk, when it is mounted on a virtual machine, read the RBD image block information, mount it on the corresponding virtual machine, and then perform a write operation when the user writes data; also Modify the cinder configuration file to support RBD storage;

Step 104: Nova integrates with RBD. In the process of creating a virtual machine, Novacompute, the computing function module of Nova, calls the RBD back-end storage, creates an RBD image storage block, clones the Glance image data to quickly restore the virtual machine disk file, and writes In the corresponding RBD storage block; also modify the Nova configuration file to support RBD storage.

Preferably, in step 101, Ceph uses the cephx authentication method to create new Ceph users for Glance, Cinder, and Nova, respectively. The Ceph user name corresponding to Glance in the new mirror service glance and the new block storage cinder is glance, and Cinder and Nova share the same name. A Ceph username cinder.

Preferably, in step 2, in order to realize the layered storage of the mirror service Glance image in the Ceph RBD, a mirror pool containing system mirror and software mirror is formed, and the virtual machine can be started in the system mirror and the virtual machine can be fully backed up and stored. Start the virtual machine in the software image and perform incremental backup of the virtual machine. The specific processing is as follows:

Step 201 : Store the system image in the images pool, start the virtual machine in the system image in the images pool, and store the disk file of the virtual machine in the vms pool to realize full backup and meet the high requirements for virtual machine backup ;

Step 202: store the software image in the volumes pool, create a virtual machine from the bootable cloud hard disk, deploy the software service, and create a snapshot of the virtual machine to realize the software image;

Step 203: Optimize the snapshot method of OpenStack through the snapshot and clone features of Ceph RBD to realize incremental snapshots;

Step 204: On the basis of realizing the incremental snapshot, complete the software mirroring layer by layer to realize the chained storage of glance mirrors;

Step 205: Start the virtual machine from the software image, which can implement incremental backup of the virtual machine.

Preferably, in step 203, the process of implementing incremental snapshots is to first create a cloud hard disk from the mirror pool, and start the virtual machine from the cloud hard disk; then create a snapshot Snapshot of the virtual machine disk; then generate a new disk by cloning the snapshot; The Ceph of the disk stores the snapshot RBD Snapshot and stores it in the corresponding volumes pool.

Preferably, in step 205, when the virtual machine is started from the software image, the virtual machine disk file is in the volumespool.

Preferably, the user can choose to start the virtual machine in the system image or the software image from the image pool according to the virtual machine backup requirement.

In order to achieve the second purpose, the present invention provides an OpenStack storage optimization system, comprising an operation interface unit, a network connection unit, a management system unit and an authentication unit respectively connected with the interface unit; it also includes blocks connected with the interface unit respectively. A storage service unit, a computing service unit, a mirroring service unit and an editing unit, the authentication units are respectively connected with the network connection unit, the block storage service unit, the computing service unit, the mirroring service unit and the editing unit, and the management system unit is respectively connected with the network The connection unit, the block storage service unit, the computing service unit and the mirroring service unit are connected, and the mirroring service unit is connected with the editing unit; the block storage service unit, the computing service unit and the mirroring service unit are respectively connected with the distributed storage system Ceph. Physical resource unit connection.

Preferably, the distributed storage system Ceph includes an object storage unit rados, an object storage interface librados, a block storage library librbd and a block storage unit RBD connected in sequence, the object storage unit rados is connected with the physical resource unit, and the block storage The unit RBD is respectively connected with the block storage service unit, the computing service unit and the mirroring service unit, wherein the block storage unit RBD includes a first pool images pool, a second pool volumes pool and a third pool vmspool, which are respectively used for storage Mirror Glance image, cloud hard disk Cinder volume and virtual machine disk image file Novadisk.

Preferably, the physical resource unit includes a server, a storage device and a network module.

Compared with the prior art, the present invention has the following beneficial effects:

The invention builds an OpenStack unified block storage system based on Ceph RBD and realizes image layered storage on this basis, which can effectively save storage space, improve data storage reliability, reduce maintenance costs, and meet different user needs. The image pool of OpenStack in the present invention provides system image and software image, realizes the system image function based on the parent image, and implements the software image function based on multiple subimages. In the present invention, after logging in to the dashboard (OpenStack control panel), the user can select a suitable software image from the image pool to start the virtual machine according to the software service required by himself, so that the software service can be used without installation, and the cloud platform can be optimized to provide external software services. In the present invention, a dependency relationship is established between the system image and the software image, which are respectively stored in different RBD pools of Ceph, thereby saving image storage space. The cloud platform in the present invention can provide two virtual machine backup modes, including full and incremental, to meet different user requirements.

Description of drawings

Fig. 1 is the flow chart of the present invention;

Fig. 2 is the OpenStack storage structure block diagram in the present invention;

FIG. 3 is a structural block diagram of the OpenStack unified storage part in the present invention.

Detailed ways

Below in conjunction with embodiment, the present invention is further described, but does not constitute any limitation to the present invention, and any limited modification done in the scope of the claims of the present invention is still within the scope of the claims of the present invention.

As shown in FIG. 1-FIG. 3, the present invention provides an OpenStack storage optimization method, and the method includes the following processing steps:

Step 1: OpenStack uses Ceph RBD as a unified storage backend for mirroring service Glance, computing service Nova and block storage service Cinder;

Step 2: OpenStack stores the image service Glance image layered in Ceph RBD to form an image pool containing system images and software images, and supports starting virtual machines in system images and performing full backup of virtual machines and software images. Start virtual machines And incremental backup of virtual machines to meet different user needs.

In step 1, in order to realize the unified storage backend of using Ceph RBD as the mirror service Glance, the computing service Nova and the block storage service Cinder, the specific processing is as follows:

Step 101: Deploy the Ceph cluster so that OpenStack can access the Ceph RBD storage service, that is, use the OpenStack computing node as the RBD client; create three pools named the first pool images pool, the second pool volumes pool and the third pool vms in the Ceph cluster Pool's Ceph storage pool RBD pool is used to store image Glanceimage, cloud hard disk Cinder volume and virtual machine disk image file Nova disk; Ceph uses cephx authentication method to create new Ceph user names for Glance, Cinder, and Nova, respectively, for new image services glance and the new block storage service cinder, and set user access rights;

In step 101, Ceph uses the cephx authentication method to create new Ceph users for Glance, Cinder, and Nova, respectively. The Ceph user name corresponding to Glance in the new image service glance and the new block storage cinder is glance, and Cinder and Nova share a Ceph user. name cinder.

Step 102: Glance is integrated with RBD. In the process of uploading the image, the image service application programming interface Glance API calls the RBD storage backend, creates the RBD image storage block, writes the image data, and also modifies the glance configuration file. It supports RBD storage.

Step 103: Cinder is integrated with RBD. In the process of creating a cloud disk, the volume cindervolume of the new block storage calls the RBD backend storage to create an RBD image storage block; if it is a bootable cloud disk, write to the RBD image block Mirror data; if it is a data cloud hard disk, when it is mounted on a virtual machine, read the RBD image block information, mount it on the corresponding virtual machine, and then perform a write operation when the user writes data; also The cinder configuration file is modified to support RBD storage.

Step 104: Nova integrates with RBD. In the process of creating a virtual machine, Novacompute, the computing function module of Nova, calls the RBD back-end storage, creates an RBD image storage block, clones the Glance image data to quickly restore the virtual machine disk file, and writes In the corresponding RBD storage block; also modify the Nova configuration file to support RBD storage.

In step 2, in order to realize the layered storage of the image service Glance image in the Ceph RBD, a mirror pool containing system images and software images is formed, and the virtual machine can be started in the system image, and the virtual machine can be fully backed up and stored in the software image. Start the virtual machine and perform incremental backup of the virtual machine. The specific processing is as follows:

Step 201 : Store the system image in the images pool, start the virtual machine in the system image in the images pool, and store the disk file of the virtual machine in the vms pool to realize full backup and meet the high requirements for virtual machine backup .

Step 202: Store the software image in the volumes pool, create a virtual machine from the bootable cloud hard disk, deploy software services, and create a snapshot of the virtual machine to implement the software image.

Step 203: Optimize the snapshot method of OpenStack through the snapshot and clone features of Ceph RBD to realize incremental snapshots;

In step 203, the process of realizing the incremental snapshot is to first create a cloud disk from the mirror pool, and start the virtual machine from the cloud disk; then create a snapshot snapshot of the virtual machine disk; then generate a new disk by cloning the Clone snapshot; Ceph stores RBD Snapshot and stores it in the corresponding volumes pool.

Step 204: On the basis of implementing the incremental snapshot, complete the software image layer by layer to realize the chained storage of glance images.

Step 205: Start the virtual machine from the software image, which can realize incremental backup of the virtual machine;

In step 205, when the virtual machine is started from the software image, the virtual machine disk file is in the volumes pool.

The user can choose to start the virtual machine in the system image or the software image from the image pool according to the virtual machine backup requirements.

In this embodiment, if the user has high requirements for the use and backup of the virtual machine, the user can choose to start the virtual machine from the system image in the imagespool, so that when backing up the virtual machine, full backup can be achieved; Quickly create virtual machine snapshots, users can choose to start the virtual machine from the software image in the volumes pool, which can realize incremental backup.

In this embodiment, the Ceph storage system has outstanding advantages and integrates features related to OpenStack. Ceph RBD supports snapshot and clone operations, which is very suitable for application in cloud platforms. Therefore, RBD can be used to provide block storage for OpenStack. In the case of single-layer Glance mirror storage, the mirrors are independent of each other, and the same data in different mirrors is stored repeatedly, resulting in a waste of storage space. Dividing OpenStack images into system images and software images and establishing dependencies between images can effectively save storage space.

As shown in FIG. 2-FIG. 3, the present invention also provides an OpenStack storage optimization system, including an operation interface unit 1, a network connection unit 2, a management system unit 3 and an authentication unit 8 respectively connected to the interface unit 1; It is characterized in that: it also includes a block storage service unit 4, a computing service unit 5, a mirroring service unit 6 and an editing unit 7 which are respectively connected with the interface unit 1, and the authentication unit 8 is respectively connected with the network connection unit 2, the block storage service unit 4, the computing The service unit 5, the mirroring service unit 6 and the editing unit 7 are connected, the management system unit 3 is respectively connected with the network connection unit 2, the block storage service unit 4, the computing service unit 5 and the mirroring service unit 6, and the mirroring service unit 6 is connected with the editing unit 7 Connection: The block storage service unit 4, the computing service unit 5 and the mirror service unit 6 are respectively connected to the physical resource unit 9 through the Ceph connection of the distributed storage system.

The distributed storage system Ceph includes an object storage unit rados, an object storage interface librados, a block storage library librbd, and a block storage unit RBD, which are connected in sequence. The object storage unit rados is connected to the physical resource unit 9, and the block storage unit RBD is respectively connected with the block storage service unit. 4. The computing service unit 5 is connected to the mirroring service unit 6, wherein the block storage unit RBD includes a first pool images pool, a second pool volumes pool, and a third pool vms pool, which are used to store mirror image Glance image and cloud hard disk respectively. Cinder volume and virtual machine disk image file Nova disk. The physical resource unit 9 includes a server 91 , a storage device 92 and a network module 93 .

In this embodiment, the interface unit 1 is used to provide a web interface for graphical operations, the network connection unit 2 is used to provide network connection services, the authentication unit 8 is used to provide authorization and authentication of operations, and the block storage service unit 4 is used to provide A cloud hard disk is provided, the computing service unit 5 is used to manage virtual machines, and the image service unit 6 is used to store and call images.

The working principle of this embodiment: after the user logs in, the interface unit 1 performs graphical operations and the authentication unit 8 authorizes and authenticates each operation. During the operation, the network connection unit 2, the block storage service unit 4, the block storage service unit 4, the The services of the computing service unit 5 and the mirroring service unit 6 perform computing operations in the management system unit 3 . Wherein the block storage service unit 4, the computing service unit 5 and the mirror service unit 6 use Ceph RBD as the unified storage back end; the mirror service Glance image is layered and stored in the Ceph RBD to form a mirror pool containing system mirroring and software mirroring, At the same time, it supports starting virtual machines in system images and performing full backups of virtual machines, and starting virtual machines in software images and performing incremental backups of virtual machines to meet different user needs.

This embodiment can effectively save storage space, improve data storage reliability, reduce maintenance costs, and meet different user requirements.

The above are only the preferred embodiments of the present invention, and it should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several modifications and improvements can be made, which will not affect the effect and effect of the present invention. Utility of Patents.

Claims (8)

1. An OpenStack storage optimization method is characterized by comprising the following processing steps:

step 1: OpenStack takes the Ceph RBD as a uniform storage back end of a mirror image service Glance, a computing service Nova and a block storage service circular;

step 2: OpenStack stores the mirror image service Glance mirror image in a Ceph RBD in a layered mode to form a mirror image pool containing a system mirror image and a software mirror image, and simultaneously supports the starting of a virtual machine in the system mirror image, full backup of the virtual machine, starting of the virtual machine in the software mirror image and incremental backup of the virtual machine so as to meet different user requirements;

in step 1, in order to realize that the Ceph RBD is used as the unified storage back end of the mirror image service Glance, the computing service Nova and the block storage service Cinder, the following processing is specifically performed:

step 101: deploying a Ceph cluster to enable OpenStack to access Ceph RBD storage service, namely, taking an OpenStack computing node as an RBD client; creating three Ceph storage pool RBD pools named as a first pool images pool, a second pool volume pool and a third pool vms pool, wherein the RBD pools are respectively used for storing a mirror image Glance image, a cloud disk vendor volume and a virtual machine disk image file Nova disk; creating new Ceph user names for the gland, the Cinder and the Nova by the Ceph in a cephx authentication mode, respectively serving as a new mirror image service gland and a new block storage service Cinder, and setting the access authority of the users;

step 102: the method comprises the steps that a gloss and an RBD are integrated, in the process of uploading a mirror image, a mirror image service application programming interface gloss API calls an RBD storage back end, an RBD mirror image storage block is created, mirror image data is written in, and a gloss configuration file is modified to support RBD storage;

step 103: the circle and the RBD are integrated, and in the process of creating the cloud hard disk, a circle volume stored in a new block calls the RBD back end to store, and an RBD mirror image storage block is created; if the cloud hard disk can be started, mirror image data is written into the RBD mirror image block; if the data cloud hard disk is the data cloud hard disk, when the data cloud hard disk is mounted on a virtual machine, reading RBD mirror image block information, mounting the RBD mirror image block information on the corresponding virtual machine, and performing writing operation when a user writes data; the cis configuration file is also modified to support RBD storage;

step 104: the method comprises the steps that Nova and RBD are integrated, in the process of creating a virtual machine, a Nova computing function module Nova computer calls RBD rear-end storage, an RBD mirror image storage block is created, and Glance mirror image data is cloned to quickly recover a virtual machine disk file and written into the corresponding RBD storage block; the Nova configuration file is also modified to support RBD storage;

in step 2, in order to implement the purpose of hierarchically storing the mirror image service Glance mirror image in the Ceph RBD, form a mirror image pool including a system mirror image and a software mirror image, and simultaneously support the system mirror image to start a virtual machine and perform full backup on the virtual machine, and the software mirror image to start the virtual machine and perform incremental backup on the virtual machine, the following processing is specifically performed:

step 201: storing the system image in an images pool, starting a virtual machine in the system image in the images pool, and storing the disk file of the virtual machine in a vms pool to realize full backup and meet the condition of high backup requirement on the virtual machine;

step 202: storing the software mirror image in a volumes pool, creating a virtual machine from a bootable cloud hard disk, deploying software service, and creating a virtual machine snapshot to realize the software mirror image;

step 203: optimizing the snapshot mode of OpenStack through snapshot and clone characteristics of Ceph RBD to realize incremental snapshot;

step 204: on the basis of realizing the incremental snapshot, completing software mirroring layer by layer to realize the linked storage of the angle mirroring;

step 205: and starting the virtual machine from the software image, so that the incremental backup of the virtual machine can be realized.

2. The OpenStack storage optimization method according to claim 1, wherein: in step 101, a Ceph user creates a new Ceph user for the gland, the circle and the Nova in a Ceph authentication mode, and the Ceph user respectively creates a gland for the gland of the new mirror image service and the Ceph user name corresponding to the gland in the new block storage circle, and the circle and the Nova share one Ceph user name circle correspondingly.

3. The OpenStack storage optimization method according to claim 1, wherein: in step 203, the process of implementing the incremental snapshot is to create a cloud hard disk from the mirror image pool first, and start a virtual machine from the cloud hard disk; then creating a Snapshot Snapshot of the virtual machine disk; then generating a new disk by cloning the Clone snapshot; the Ceph storage snapshot rbdsnaphot of the new disk is created and stored into the corresponding volumes pool.

4. The OpenStack storage optimization method according to claim 1, wherein: in step 205, the disk file of the virtual machine is in the volumes pool when the virtual machine is started from the software image.

5. The OpenStack storage optimization method according to claim 1, wherein: the user can select to start the virtual machine in the system image or the software image from the image pool according to the backup requirement of the virtual machine.

6. An OpenStack storage optimization system according to the OpenStack storage optimization method of any one of claims 1 to 5, comprising an operation interface unit (1), a network connection unit (2), a management system unit (3), and an authentication unit (8) respectively connected to the operation interface unit (1); the method is characterized in that: the system is characterized by further comprising a block storage service unit (4), a calculation service unit (5), a mirror image service unit (6) and an editing unit (7) which are respectively connected with the operation interface unit (1), wherein the authentication unit (8) is respectively connected with the network connection unit (2), the block storage service unit (4), the calculation service unit (5), the mirror image service unit (6) and the editing unit (7), the management system unit (3) is respectively connected with the network connection unit (2), the block storage service unit (4), the calculation service unit (5) and the mirror image service unit (6), and the mirror image service unit (6) is connected with the editing unit (7); the block storage service unit (4), the calculation service unit (5) and the mirror image service unit (6) are respectively connected with the physical resource unit (9) through a distributed storage system Ceph.

7. The OpenStack storage optimization system according to claim 6, wherein: the distributed storage system Ceph comprises an object storage unit rados, an object storage interface librardos, a block storage library librbd and a block storage unit RBD which are sequentially connected, wherein the object storage unit rados is connected with a physical resource unit (9), the block storage unit RBD is respectively connected with a block storage service unit (4), a computing service unit (5) and a mirror image service unit (6), and the block storage unit RBD comprises a first pool image pool, a second pool volume pool and a third pool vm pool which are respectively used for storing a mirror image Glance image, a cloud hard disk circular divider volume and a virtual machine disk mirror image Nova disk.

8. The OpenStack storage optimization system according to claim 6, wherein: the physical resource unit (9) comprises a server (91), a storage device (92) and a network module (93).

Images