CN117742578A - Data storage system, method, apparatus, electronic device, and computer storage medium - Google Patents

Abstract

The application discloses a data storage management system, a data storage management method, a data storage management device, an electronic device and a computer storage medium, wherein the system comprises: the system comprises a management platform, an execution platform and an infrastructure layer; the management platform comprises: a scheduling layer and a management layer; the scheduling layer comprises a scheduling decision module and a scheduling execution module; the scheduling decision module is used for generating a scheduling instruction and sending the scheduling instruction to the scheduling execution module for execution; the scheduling execution module is used for calling an interface of the cloud storage device management service module to execute storage capacity adjustment operation on the cloud storage device based on the scheduling instruction; the management layer comprises a user storage path management module and a cloud storage device management service module; the execution platform comprises: the cloud storage device comprises a storage atom management and control module and a cloud storage device atom management and control module; the infrastructure layer includes a plurality of distributed cloud storage devices, the infrastructure layer for storing user data.

Description

Data storage system, method, apparatus, electronic device, and computer storage medium

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to a data storage system, a method, an apparatus, an electronic device, and a computer storage medium.

Background

Along with the rapid development of technology, informatization is closely related to us, and the problem of mass information storage is brought along with the informatization. Aiming at ultra-large-scale user information storage, the existing data storage mode is high in cost and has the risk of data loss. Therefore, it is currently desirable to provide a solution to mass data storage to solve the problem of ultra-large scale user data storage.

Disclosure of Invention

The embodiment of the application provides a data storage management system, a method, a device, electronic equipment and a computer storage medium, wherein the data storage system can realize low-cost data storage.

The embodiment of the application provides a data storage management system, which comprises: the system comprises a management platform, an execution platform and an infrastructure layer; the management platform comprises: a scheduling layer and a management layer; the scheduling layer is used for deciding whether storage capacity adjustment operation is needed for the cloud storage device or not; the management layer is used for managing the storage path of the user data; the scheduling layer comprises a scheduling decision module and a scheduling execution module; the scheduling decision module is used for generating a scheduling instruction and sending the scheduling instruction to the scheduling execution module for execution; the scheduling execution module is used for calling an interface of the cloud storage device management service module to execute storage capacity adjustment operation on the cloud storage device based on the scheduling instruction; the management layer comprises a user storage path management module, wherein the user storage path management module is used for managing a storage path of user data; the execution platform comprises: the cloud storage device comprises a storage atom management and control module and a cloud storage device atom management and control module; the storage atom management and control module is used for managing and controlling the storage of the user data; the cloud storage device atomic management and control module is used for managing and controlling the cloud storage device; the infrastructure layer comprises a plurality of distributed cloud storage devices, and is used for storing user data; the management platform is used for sending a management and control instruction to the execution platform according to user data management requirements and cloud storage device management requirements, the execution platform is used for executing the management and control instruction, the basic setting layer is used for providing a cloud storage device management interface for the execution platform, and the execution platform is further used for utilizing the cloud storage device management interface to manage and control storage of user data and cloud storage devices.

Optionally, the management platform further includes: a data acquisition module;

the data acquisition module is used for acquiring attribute information of the cloud storage device and sending the attribute information to the scheduling decision module;

the scheduling decision module is used for generating scheduling instructions, and comprises:

the scheduling decision module is used for generating scheduling instructions according to the attribute information, and the scheduling instructions are used for judging whether storage capacity adjustment operation needs to be executed for the cloud storage device or not.

Optionally, the attribute information includes at least one of the following information:

storage capacity information of the cloud storage device;

the consumption information of the cloud storage device;

user presence information;

the cloud storage device inputs and outputs data information.

Optionally, the scheduling instruction includes: an operation instruction for expanding the capacity, an operation instruction for reducing the capacity, and an operation instruction for keeping the capacity unchanged;

the storage capacity adjustment operation includes: an operation of expanding the capacity, an operation of reducing the capacity, and an operation of keeping the capacity unchanged.

Optionally, the scheduling layer further includes: a movable module; the activity module is used for recording an execution log corresponding to each scheduling behavior; the execution log includes at least one of the following information: time information for activity creation, activity period information, activity status information, and activity classification information.

Optionally, the management layer further includes: a storage strategy module; the storage strategy module is used for setting a storage strategy of the user data; the storage policy includes: storing a capacity expansion strategy and a roaming strategy; the storage expansion and contraction strategy is a strategy that a pointer executes expansion of storage capacity and reduction of storage capacity on a cloud storage device; the stored roaming policy is a policy in which pointers reserve and clean up user data.

Optionally, the user storage path management module includes: a storage path allocation sub-module, a storage path updating sub-module, a storage path deleting sub-module and a storage path inquiring sub-module; the allocation storage path sub-module is used for allocating a storage path for the user data, the update storage path sub-module is used for updating the storage path of the user data when the storage path of the user data changes, the deletion storage path sub-module is used for deleting the storage path of the user data, and the query storage path sub-module is used for querying the storage path of the user.

Optionally, the cloud storage device management service module includes at least one module of a cloud storage device expansion capacity management sub-module, a cloud storage device reduction capacity management sub-module, a cloud storage device capacity query management sub-module, and a user data storage path management sub-module.

Optionally, the storage atom management and control module includes at least one module of a storage classification sub-module, a storage creation sub-module, a storage update sub-module, a storage upgrade sub-module, and a storage roaming policy sub-module.

Optionally, the cloud storage device atomic control includes at least one module of a cloud storage device control sub-module, a user data control sub-module, and a storage path control sub-module;

the cloud storage device management and control submodule is used for managing and controlling the cloud storage device, the user data management and control submodule is used for managing and controlling the user data, and the storage path submodule is used for managing and controlling a user data storage path.

The embodiment of the application also provides a data storage management method, which comprises the following steps: responding to the login operation of the user, and obtaining a user data distribution storage path query request message sent by the user side and used for querying the storage path distributed by the user data of the current user; distributing a storage path query request message according to the user data to obtain attribute information of the cloud storage device; and distributing a storage path for the user data according to the attribute information of the cloud storage device.

Optionally, the attribute information includes at least one of the following information:

Storage capacity information of the cloud storage device;

the consumption information of the cloud storage device;

user presence information;

the cloud storage device inputs and outputs data information.

Optionally, the method further comprises:

a plurality of distributed cloud storage devices is obtained, the cloud storage devices for storing user data.

Optionally, after responding to the login operation of the user, the method further comprises: judging whether the user is a new user or not;

if the user is a new user, carrying out authorization management on the new user;

and if the user is not a new user, performing authority verification on the user.

Optionally, the method further comprises:

collecting attribute information of the cloud storage device;

generating a scheduling instruction based on the attribute information of the cloud storage device, wherein the scheduling instruction is used for judging whether storage capacity adjustment operation needs to be executed for the cloud storage device or not;

and executing a storage capacity adjustment operation for the cloud storage device based on the scheduling instruction.

Optionally, the scheduling instruction includes: an operation instruction for expanding the capacity, an operation instruction for reducing the capacity, and an operation instruction for keeping the capacity unchanged;

the storage capacity adjustment operation includes: an operation of expanding the capacity, an operation of reducing the capacity, and an operation of keeping the capacity unchanged.

Optionally, the method further comprises:

determining whether the user logs in again within a preset time length according to the login state of the user;

and if not, migrating the user data to a designated cloud storage device.

Optionally, the determining, according to the login state of the user, whether the user logs in again within a preset duration includes:

obtaining login operation record data of a user;

according to the login operation record data and a preset algorithm, periodic data of a desktop used by a user are obtained;

and determining whether the user logs in again within a preset time period based on the period data.

Optionally, after migrating the user data to the designated cloud storage device, the method further includes: the user data storage path of the current user is updated.

Optionally, the method further comprises:

inquiring the consumption of the cloud storage device;

and if the consumption of the cloud storage device is in an unused state, unloading the cloud storage device in the unused state.

Optionally, the method further comprises:

if the user is set to be in a deleting state, migrating the user data to a designated cloud storage device or performing cold standby storage on the user data;

and if the user is restored to the normal state within the designated time, restoring the user data to the original cloud storage device.

The embodiment of the application also provides a data storage management device, which comprises: the first obtaining unit is used for responding to the login operation of the user and obtaining a user data distribution storage path query request message sent by the user side and used for querying the storage path distributed by the user data of the current user; the second obtaining unit is used for distributing a storage path query request message according to the user data to obtain attribute information of the cloud storage device; and the distribution unit is used for distributing a storage path for the user data according to the attribute information of the cloud storage device.

The application also provides an electronic device comprising a processor and a memory; the memory stores a computer program, and the processor executes the method after running the computer program.

The present application also provides a computer storage medium storing a computer program which, when executed by a processor, performs the above method.

Compared with the prior art, the embodiment of the application has the following advantages:

the embodiment of the application provides a data storage management system, which comprises: the system comprises a management platform, an execution platform and an infrastructure layer; the management platform comprises: a scheduling layer and a management layer; the scheduling layer is used for deciding whether storage capacity adjustment operation is needed for the cloud storage device or not; the management layer is used for managing the storage path of the user data; the scheduling layer comprises a scheduling decision module and a scheduling execution module; the scheduling decision module is used for generating a scheduling instruction and sending the scheduling instruction to the scheduling execution module for execution; the scheduling execution module is used for calling an interface of the cloud storage device management service module to execute storage capacity adjustment operation on the cloud storage device based on the scheduling instruction; the management layer comprises a user storage path management module, wherein the user storage path management module is used for managing a storage path of user data; the execution platform comprises: the cloud storage device comprises a storage atom management and control module and a cloud storage device atom management and control module; the storage atom management and control module is used for managing and controlling the storage of the user data; the cloud storage device atomic management and control module is used for managing and controlling the cloud storage device; the infrastructure layer comprises a plurality of distributed cloud storage devices, and is used for storing user data; the management platform is used for sending a management and control instruction to the execution platform according to user data management requirements and cloud storage device management requirements, the execution platform is used for executing the management and control instruction, the basic setting layer is used for providing a cloud storage device management interface for the execution platform, and the execution platform is further used for utilizing the cloud storage device management interface to manage and control storage of user data and cloud storage devices.

According to the embodiment of the application, the scheduling decision module is used for generating the scheduling instruction, the scheduling execution module is used for executing the scheduling instruction, and the interface of the cloud storage device management service module is called for executing the storage capacity adjustment operation on the cloud storage device, so that the capacity of the cloud storage device can be flexibly scheduled in real time according to the user data; in addition, by providing a plurality of distributed cloud storage devices, low-cost data storage can be realized.

Drawings

FIG. 1 is a schematic view of an application scenario of a data storage management system according to an embodiment of the present application;

FIG. 2 is a flow chart of a data storage management method provided in a first embodiment of the present application;

FIG. 3 is a schematic diagram of a data storage management device according to a second embodiment of the present application;

fig. 4 is a schematic diagram of an electronic device according to a third embodiment of the present application.

Detailed Description

In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is, however, susceptible of embodiment in many other ways than those herein described and similar generalizations can be made by those skilled in the art without departing from the spirit of the application and the application is therefore not limited to the specific embodiments disclosed below.

First, in order to enable those skilled in the art to better understand the present application, a detailed description is given below of a specific application scenario of an embodiment of the present application based on a data storage management system provided in the present application. Fig. 1 is a schematic application scenario diagram of a data storage management system according to an embodiment of the present application.

Before introducing an application scenario of a data storage management system provided by the embodiments of the present application, the following technical background is first briefly introduced, generally, a conventional cloud desktop delivery scale is generally within a thousand level, there are few tens of thousands of delivery cases, and when the cloud desktop delivery reaches over ten thousands of levels, many problems that are difficult to be encountered in the industry, such as login, data loss, and high storage cost, are encountered. Based on this, the data storage management system provided by the embodiment of the application can solve the problems of login blocking, data loss and high storage cost with low cost, and the scheme has linear upgrading capability, and can rapidly complete storage performance upgrading and cost reduction of service through flexible elastic scheduling capability.

The following describes an application scenario of a data storage management system provided in the embodiment of the present application, where in a specific implementation process, a specific application scenario of the data storage management system in the embodiment of the present application is data storage for a cloud desktop. The data storage management system includes: the system comprises a management platform, an execution platform and an infrastructure layer; the management platform comprises: a scheduling layer and a management layer; the scheduling layer is used for deciding whether storage capacity adjustment operation is needed for the cloud storage device or not; the management layer is used for managing the storage path of the user data; the scheduling layer comprises a scheduling decision module and a scheduling execution module; the scheduling decision module is used for generating a scheduling instruction and sending the scheduling instruction to the scheduling execution module for execution; the scheduling execution module is used for calling an interface of the cloud storage device management service module to execute storage capacity adjustment operation on the cloud storage device based on the scheduling instruction; the management layer comprises a user storage path management module, wherein the user storage path management module is used for managing a storage path of user data; the execution platform comprises: the cloud storage device comprises a storage atom management and control module and a cloud storage device atom management and control module; the storage atom management and control module is used for managing and controlling the storage of the user data; the cloud storage device atomic management and control module is used for managing and controlling the cloud storage device; the infrastructure layer comprises a plurality of distributed cloud storage devices, and is used for storing user data; the management platform is used for sending a management and control instruction to the execution platform according to user data management requirements, the execution platform is used for executing the management and control instruction, the infrastructure layer is used for providing a cloud storage device management interface for the execution platform, and the execution platform is further used for utilizing the cloud storage device management interface to manage and control storage of user data and cloud storage devices.

In this embodiment of the present application, the user data management requirement may specifically be management of a user data storage path, including allocation, update, deletion, query, and the like of the user data storage path; the cloud storage device management requirement can be specifically management of the capacity of the cloud storage device, including capacity expansion, capacity shrinkage, capacity inquiry and the like of the cloud storage device. The control instruction can be specifically a control instruction of a user data storage path, a control instruction of a cloud storage device and the like, and the control instruction of the user data storage path can be specifically an allocation instruction, an update instruction, a deletion instruction, a query instruction and the like of the user data storage path; the control instruction of the cloud storage device can be specifically an expansion instruction, a contraction instruction, a capacity query instruction and the like of the cloud storage device

In specific implementation, first, the data storage management system includes a management platform, an execution platform, and an infrastructure layer, where the management platform includes: a scheduling layer and a management layer; in a specific implementation, the management platform may be a consumer-manager (i.e. a management state), the scheduling layer may be an ADO scheduling layer, and the management layer may be an ADO management layer, where ADO (ActiveX Data Objects, active x data object) is a program object, which is used to represent a data structure and contained data in a user database. The scheduling layer is used for deciding whether storage capacity adjustment operation is needed for the cloud storage device or not; the management layer is used for managing the storage path of the user data. After a user logs in a cloud desktop, sending a request message for inquiring a storage strategy and a storage path of user data to an ADO management layer of a user-manager, wherein the request message is used for judging which data need to be reserved when logging in next time and mounting a remote SMB service path on the cloud desktop, and the remote SMB service path exists as a D disc, namely a user data disc, wherein SMB (Server Message Block) is a protocol name which can be used for Web connection and information communication between a client and a server; then the user can use the desktop normally; after the user uses the desktop to log out, the desktop service synchronizes the data to be kept into the file of the remote storage path according to the storage strategy. The management platform responds to the request message, the data acquisition module of the management platform acquires attribute information of the cloud storage device, namely, the ADO scheduling layer acquires data information of the current system at fixed time, such as information of online condition, storage water level, storage data quantity, user authorization quantity and the like of each delivery group, and in the online low-peak period of the user, algorithm judgment is carried out by scheduling decision through the acquired information, and then the information is submitted to scheduling execution for execution, and the scheduling execution invokes disk management service and storage management service to complete corresponding disk purchasing, mounting, unloading, deleting and other actions.

In the embodiment of the application, a cloud desktop, also called a virtual desktop, is a server-based computing model, and is a technology for realizing remote dynamic access of a desktop system and unified hosting of a data center at an enterprise level. The cloud storage device may be a device storing data such as a cloud disk, a magnetic disk, or the like. The attribute information includes at least one of the following information: the storage capacity information of the cloud storage device, namely, the cloud disk capacity, the consumption information of the cloud storage device, namely, the cloud disk consumption, the user online state information, and the Input and Output data information of the cloud storage device, namely, the cloud disk IO (Input/Output). After the data acquisition module acquires the attribute information of the cloud storage device, on one hand, the management platform can allocate a storage path for the user data according to the attribute information of the cloud storage device, the management layer comprises a user storage path management module, and the user storage path management module is used for managing the storage path of the user data. On the other hand, the scheduling layer comprises a scheduling decision module and a scheduling execution module, the scheduling decision module generates a scheduling instruction according to the attribute information, the scheduling instruction is used for judging whether the storage capacity adjustment operation needs to be executed for the cloud storage device or not, the scheduling instruction is sent to the scheduling execution module for execution, and the scheduling execution module is used for calling an interface of the cloud storage device management service module to execute the storage capacity adjustment operation for the cloud storage device based on the scheduling instruction.

The scheduling execution module comprises: scheduling classification, scheduling expansion, scheduling contraction and scheduling maintenance. The scheduling classification refers to scheduling capacity expansion, scheduling capacity reduction and scheduling maintenance. The scheduling capacity expansion refers to an operation of expanding the storage capacity of the cloud storage device, and specific capacity expansion operations may include increasing the number of disks, increasing the capacity of the disks, increasing the specification of the disks, and the like, and performing one of the three types of capacity expansion behaviors after a decision is made by a management state according to collected data. The scheduling capacity reduction refers to the operation of reducing the storage capacity of the cloud storage device, and the specific capacity reduction operation may include reducing the number of disks, reducing the capacity of the disks, reducing the specification of the disks, and the like, and performing one of the three types of capacity reduction behaviors after a management state makes a decision according to collected data. Schedule maintenance refers to maintaining the current disk size and storage capacity without any action or instruction. The effect of scheduling execution is to execute instructions resulting from scheduling decisions, such as expand instructions, contract instructions, hold instructions.

The management platform further comprises: a data acquisition module; the data acquisition module is used for acquiring attribute information of the cloud storage device and sending the attribute information to the scheduling decision module; the scheduling decision module is used for generating scheduling instructions, and comprises: the scheduling decision module is used for generating scheduling instructions according to the attribute information, and the scheduling instructions are used for judging whether storage capacity adjustment operation needs to be executed for the cloud storage device or not. The scheduling instruction includes: an operation instruction for expanding the capacity, an operation instruction for reducing the capacity, and an operation instruction for keeping the capacity unchanged; the storage capacity adjustment operation includes: an operation of expanding the capacity, an operation of reducing the capacity, and an operation of keeping the capacity unchanged.

After the user logs in, the method further comprises the operation of judging whether the user number is a new user, if the user is the new user, authorization management is conducted on the new user, and when the user is authorized, a storage path of the designated cloud storage device is distributed according to the attribute information of the cloud storage device; if the user is not a new user, performing authority verification on the user, and after the user logs in, in the elastic scheduling, according to the attribute information of the cloud storage device, adding or removing the cloud storage device, and possibly changing the storage path of the user, so that the storage path needs to be redistributed during login.

The scheduling layer further includes: a movable module; the activity module is used for recording an execution log corresponding to each scheduling behavior; the execution log includes at least one of the following information: time information for activity creation, activity period information, activity status information, and activity classification information. The activity refers to the behavior and execution condition log of each scheduling, when the execution starts, the current system information such as online quantity, disk consumption, occurrence time, behavior and the like is recorded, when the scheduling ends, the behavior result is recorded, and the information of the starting and ending is combined into one scheduling activity and then persisted into a database. The scheduling layer also comprises scheduling traceability, wherein the scheduling traceability refers to scheduling behavior and data traceability. The management layer further comprises: a storage strategy module; the storage strategy module is used for setting a storage strategy of the user data; the storage policy includes: storing a capacity expansion strategy and a roaming strategy; the storage strategy refers to decision logic for capacity expansion or capacity shrinkage. The storage expansion and contraction strategy is a strategy that a pointer executes expansion of storage capacity and reduction of storage capacity on a cloud storage device; the stored roaming policy is a policy in which pointers reserve and clean up user data. The stored roaming policy refers to data that a user can keep when logging in each time, and data that is not in the roaming policy configuration can be automatically cleaned when the user logs out of the session.

The user storage path management module includes: a storage path allocation sub-module, a storage path updating sub-module, a storage path deleting sub-module and a storage path inquiring sub-module; the distribution storage path submodule is used for distributing storage paths for user data, and the distribution storage path submodule is specifically used for calculating the distribution paths by a management state according to the current system load condition when authorizing users. When a storage path is dynamically allocated for user data, judging whether the user is a new user or not, and if the user is the new user, carrying out authorization management for the new user; if the user is not a new user, performing authority verification on the user, and then dynamically distributing the storage path of the user by ADO. Dynamic allocation includes two scenarios: one is that when the user authorizes, a storage path of a designated drive is allocated according to the storage capacity, the consumption and the number of users in the current space; another is that when a user logs in, in the flexible scheduling, a disk is newly added or removed according to the capacity, the usage amount, the number of users and the online condition of the storage, and the storage path of the user may be changed, so that the changed storage path is allocated during logging in. The updating storage path submodule is used for updating the storage path of the user data when the storage path of the user data changes, and the updating storage path submodule is particularly used for calculating and updating the storage path of the user according to the disk condition by a management state when the capacity is expanded or contracted. It should be noted that, schedule (distributed task scheduling) is scheduled regularly, according to the user login status, it is decided whether the user is no longer logged in within a certain time, and then the user data is migrated to the designated drive letter to be reserved, and the ADO side updates the user storage path. This is because, when the user logs in, the ADO allocates a storage path to the user, if it is determined that the user is no longer logged in within a certain time according to the user login status, the user data is migrated to the designated disc, and at this time, the storage path of the user is changed, and the ADO side needs to further update the storage path of the user. The storage path deleting sub-module is used for deleting the storage path of the user data, and the storage path deleting sub-module is specifically used for deleting the storage path of the user when the user leaves the office or the manager starts to recover the right. The inquiry storage path submodule is used for inquiring the storage path of the user, and the inquiry storage path submodule is particularly used for inquiring the management state by the desktop management service when the user logs in the desktop, and inquiring the storage path of the user so as to mount a disk. The cloud storage device management service module comprises at least one module of a cloud storage device expansion capacity management sub-module, a cloud storage device reduction capacity management sub-module, a cloud storage device capacity query management sub-module and a user data storage path management sub-module. The cloud storage device management service may specifically be a disk management service. In the specific implementation, the disk management service realizes capacity expansion and capacity shrinkage by mounting and unloading the disk. On the one hand, the cloud storage device management service module receives a scheduling instruction and manages the cloud storage device, and specific management operations comprise capacity expansion operation of the cloud storage device, capacity shrinkage operation of the cloud storage device and capacity query operation of the cloud storage device; on the other hand, the cloud storage device management service module receives the storage path information of the user data sent by the user storage path management module, and manages the storage path information of the user data. The cloud storage device capacity inquiry management submodule is specifically realized through a block device interface of the Windows system, the user data storage path management submodule is specifically used for managing a user data storage path, the data path is stored in a database after being distributed with the path, and a desktop service inquires the path to a management state when a user logs in.

It should be noted that, the relationship between the user allocation storage path and the disk capacity expansion and contraction is as follows: when the capacity of the disk is insufficient, a path is allocated to the user after capacity expansion is performed; when shrinking, the original storage path may change by adjusting the arrangement and number of the disks, and the storage path of the user needs to be updated.

The execution platform comprises: the cloud storage device comprises a storage atom management and control module and a cloud storage device atom management and control module; the execution platform may also be referred to as a initiator-center (data warehouse execution side, i.e., execution state); the storage atom management and control module is used for managing and controlling the storage of the user data; the cloud storage device atomic management and control module is used for managing and controlling the cloud storage device. The storage atom management and control module comprises at least one module of a storage classification sub-module, a storage creation sub-module, a storage update sub-module, a storage lifting sub-module and a storage roaming strategy sub-module. The storage classification refers to classification of storage behaviors, and comprises storage creation, storage update and storage upgrade; the storage creation, storage update and storage upgrade are all operated by the storage atom management and control service calling a specific server interface. The cloud storage device atomic control comprises at least one module of a cloud storage device control sub-module, a user data control sub-module and a storage path control sub-module; the cloud storage device management and control submodule is used for managing and controlling the cloud storage device, such as disk classification, cloud disk purchase, cloud disk mounting, unloading Yun Pan, deletion Yun Pan, cloud disk expansion, cloud disk compression, cloud disk inquiry, cloud disk storage and the like. The disk classification refers to disk specifications, such as a first performance level disk specification, a second performance level disk specification and a third performance level disk specification, the purchase of the cloud disk refers to the purchase behavior completion by calling a specific server interface, the mounting of the cloud disk and the unloading of the cloud disk are realized through disk management services on a storage machine, the disk management services on the storage machine provide interface capability, the business operation is completed through the disk management services of an ADO management layer, and the cloud disk compression is realized through disk data arrangement and large file compression to save space. The user data management and control submodule is used for managing and controlling the user data, such as mobile user data, mobile data results, deleted user data, deleted data results and the like. The storage path submodule is used for managing and controlling a user data storage path, such as storage path reporting, storage path deleting and the like, wherein the storage path reporting is that a storage path is written into a storage machine configuration by a disk management service, and a desktop service realizes subsequent mounting service by reading the configuration. The execution platform further comprises: desktop atom management and control refers to management and control of desktops, such as desktop opening, closing, removing, distributing, recycling and other atomic behaviors.

The infrastructure layer includes a plurality of distributed cloud storage devices, the infrastructure layer for storing user data. Each cloud storage device may be a block storage cloud disk, each block storage being mounted as a separate drive to a cloud server ECS (Elastic Compute Service, elastic computer service) storage machine, each block storage disk accommodating 875 users at most. In this embodiment of the present application, the management platform is configured to send a management and control instruction to the execution platform according to a user data management requirement and a cloud storage device management requirement, the execution platform is configured to execute the management and control instruction, the base setting layer is configured to provide a cloud storage device management interface for the execution platform, and the execution platform is further configured to utilize the cloud storage device management interface to manage and control storage of user data and a cloud storage device. The management platform issues corresponding instructions to the execution platform according to service requirements and the instructions are executed by the execution platform, wherein the ECS storage machine provides storage management service for providing SMB service and a storage management interface for the execution platform, and specifically, the execution platform sends scheduling instructions to an infrastructure layer and selects corresponding cloud storage devices from the infrastructure layer. In addition, after the user logs in the cloud desktop, the specified path can be mounted as a D disk of the user, specifically, the SMB service path is mounted in the user desktop service and is displayed to the user in a D disk form, for example, if the physical path of the user is AUTODFS 00038\F\username, the physical path is mounted as the D disk of the user desktop system through SMB service.

In this embodiment of the present application, by splitting a block storage cloud disk of a third performance level disk specification of a high performance and large storage capacity into a plurality of block storage cloud disks of a first performance level disk specification, for example, a D disk, an E disk, and the like included in an ECS storage machine in fig. 1 are all block storage cloud disks of the first performance level disk specification, and by adopting the block storage cloud disk of the first performance level disk specification to replace the block storage cloud disk of the third performance level disk specification of the high performance and large storage capacity, storage cost is effectively reduced. When the user logs in, the current load condition is different from the load condition when the user authorizes, so that the storage path can be redistributed again based on the load condition of the current system, and the storage path can be dynamically distributed for the user. Meanwhile, flexible scheduling is realized based on scheduling instructions, so that storage space can be rapidly allocated for users, useless space is reduced, and cost of data storage is reduced.

The foregoing is a whole process analysis of a data storage management system, and an embodiment of the present application provides a data storage management system, including: the system comprises a management platform, an execution platform and an infrastructure layer; the management platform comprises: a scheduling layer and a management layer; the scheduling layer is used for deciding whether storage capacity adjustment operation is needed for the cloud storage device or not; the management layer is used for managing the storage path of the user data; the scheduling layer comprises a scheduling decision module and a scheduling execution module; the scheduling decision module is used for generating a scheduling instruction and sending the scheduling instruction to the scheduling execution module for execution; the scheduling execution module is used for calling an interface of the cloud storage device management service module to execute storage capacity adjustment operation on the cloud storage device based on the scheduling instruction; the management layer comprises a user storage path management module, wherein the user storage path management module is used for managing a storage path of user data; the execution platform comprises: the cloud storage device comprises a storage atom management and control module and a cloud storage device atom management and control module; the storage atom management and control module is used for managing and controlling the storage of the user data; the cloud storage device atomic management and control module is used for managing and controlling the cloud storage device; the infrastructure layer comprises a plurality of distributed cloud storage devices, and is used for storing user data; the management platform is used for sending a management and control instruction to the execution platform according to user data management requirements and cloud storage device management requirements, the execution platform is used for executing the management and control instruction, the basic setting layer is used for providing a cloud storage device management interface for the execution platform, and the execution platform is further used for utilizing the cloud storage device management interface to manage and control storage of user data and cloud storage devices.

According to the embodiment of the application, the scheduling decision module is used for generating the scheduling instruction, the scheduling execution module is used for executing the scheduling instruction, and the interface of the cloud storage device management service module is called for executing the storage capacity adjustment operation on the cloud storage device, so that the capacity of the cloud storage device can be flexibly scheduled in real time according to the user data; in addition, by providing a plurality of distributed cloud storage devices, low-cost data storage can be realized.

The present application is described in detail below with reference to a number of examples and figures.

First embodiment

A data storage management method according to a first embodiment of the present application is described in detail below with reference to fig. 2, and fig. 2 is a flowchart of the data storage management method according to the first embodiment of the present application.

Step S201: and responding to the login operation of the user, and obtaining a user data distribution storage path query request message sent by the user side and used for querying the storage path distributed by the user data of the current user.

The step is used for obtaining a request message for inquiring a storage path allocated by user data sent by a user side, and after the user logs on a cloud desktop, the step further comprises the steps of: and judging whether the user is a new user or not. The manager side performs authorization management on logged-in users, firstly judges whether the users are new users, and performs authorization management on the new users if the users are new users; if the user is not a new user, authorization is verified for the user, and then the storage path of the user is dynamically allocated by ADO (ActiveX Data Objects, activeX data object).

Step S202: and distributing a storage path query request message according to the user data to obtain attribute information of the cloud storage device.

The method is used for obtaining attribute information of the cloud storage device. The cloud storage device may be a device for storing data, such as a cloud disk, a magnetic disk, and the like. The attribute information includes at least one of the following information: the storage capacity information of the cloud storage device, namely, the cloud disk capacity, the consumption information of the cloud storage device, namely, the cloud disk consumption, the user online state information, and the Input and Output data information of the cloud storage device, namely, the cloud disk IO (Input/Output).

In this embodiment of the present application, further includes: collecting attribute information of the cloud storage device; generating a scheduling instruction based on the attribute information of the cloud storage device, wherein the scheduling instruction is used for judging whether storage capacity adjustment operation needs to be executed for the cloud storage device or not; and executing a storage capacity adjustment operation for the cloud storage device based on the scheduling instruction. The scheduling instruction includes: an operation instruction for expanding the capacity, an operation instruction for reducing the capacity, and an operation instruction for keeping the capacity unchanged; the storage capacity adjustment operation includes: an operation of expanding the capacity, an operation of reducing the capacity, and an operation of keeping the capacity unchanged.

It should be noted that, the data acquisition content includes: the information such as the user on-line state, the machine load condition, the disk storage data capacity and the like is used for judging whether storage expansion and contraction are needed. The scheduling decision (storage capacity expansion, capacity shrinkage and the like) is then determined by regularly acquiring the user online state and the storage capacity. The scheduling decision is an instruction for generating decision scheduling according to the capacity of each cloud disk, the consumption of the cloud disk, the IO of the cloud disk and the online condition of a user, and the instruction comprises a capacity expansion instruction, a capacity shrinkage instruction and a constant instruction.

Step S203: and distributing a storage path for the user data according to the attribute information of the cloud storage device.

The step is used for distributing a storage path for the user data according to the attribute information of the cloud storage device.

In an embodiment of the present application, further includes: a plurality of distributed cloud storage devices is obtained, the cloud storage devices for storing user data. Each cloud storage device may be a block storage cloud disk, each block storage is mounted as an independent disk symbol to a cloud server ECS (Elastic Compute Service, elastic computer service) storage machine, each block storage disk accommodates 875 users at most, and it should be noted that, user data that each block storage disk accommodates at most can be dynamically adjusted according to online user volume, and computing logic of user data that each block storage disk accommodates at most: 350MB throughput/2 MB per person maximum usage/20% online = 875 person.

When a user logs in, the storage path of the current user in the storage machine data is acquired from the bottom layer, the storage path of the user is dynamically allocated by ADO when the user authorizes, and the dynamic allocation comprises two scenes: one is that when the user authorizes, a storage path of a designated drive is allocated according to the storage capacity, the consumption and the number of users in the current space; another is that when a user logs in, in the flexible scheduling, a disk is newly added or removed according to the capacity, the usage amount, the number of users and the online condition of the storage, and the storage path of the user may be changed, so that the changed storage path is allocated during logging in.

In an embodiment of the present application, further includes: determining whether the user logs in again within a preset time length according to the login state of the user; and if not, migrating the user data to a designated cloud storage device. After migrating the user data to the designated cloud storage device, the method further comprises: the user data storage path of the current user is updated. Determining whether the user logs in again within a preset time length according to the login state of the user comprises the following steps: obtaining login operation record data of a user; according to the login operation record data and a preset algorithm, periodic data of a desktop used by a user are obtained; and determining whether the user logs in again within a preset time period based on the period data.

It should be noted that, schedule (distributed task scheduling) is scheduled regularly, according to the user login status, it is decided whether the user is no longer logged in within a certain time, and then the user data is migrated to the designated drive letter to be reserved, and the ADO side updates the user storage path. This is because, when the user logs in, the ADO allocates a storage path to the user, if it is determined that the user is no longer logged in within a certain time according to the user login status, the user data is migrated to the designated disc, and at this time, the storage path of the user is changed, and the ADO side needs to further update the storage path of the user. The method comprises the steps of judging whether a user is not logged in any more in a certain time, specifically, collecting login records of the user, assisting algorithm calculation, presuming the periodic rule of the desktop used by the user, and deciding whether the user is not logged in any more in a certain time according to the periodic rule data of the desktop used by the user. Migrating the user data to the specific drive letter to be reserved in the specific drive letter refers to the drive letter of the user data in the storage machine.

In an embodiment of the present application, further includes: inquiring the consumption of the cloud storage device; and if the consumption of the cloud storage device is in an unused state, unloading the cloud storage device in the unused state. Here, if the disk usage is 0, the disk drive with the usage of 0 is unloaded, and the block of storage cloud disk is deleted from the ECS.

In an embodiment of the present application, further includes: if the user is set to be in a deleting state, migrating the user data to a designated cloud storage device or performing cold standby storage on the user data; and if the user is restored to the normal state within the designated time, restoring the user data to the original cloud storage device. It should be noted that cold storage is a data storage technology, in which data is usually stored in a location where real-time access is not required by using an offline medium, and when a user leaves or passively deletes, the user is migrated to a designated drive or cold backup storage, and the user resumes automatically when entering the job again within a designated period.

According to the data storage management method, the distributed small-size storage machines are combined, so that the performance of far oversized specifications can be achieved with the advantage of low cost; in addition, the storage space can be dynamically allocated for the user in real time, and the disk can be dynamically mounted and dismounted; meanwhile, the capacity expansion, capacity contraction and the like can be prepared for the user at any time by the real-time scheduling capability. Therefore, according to the data storage management method, the third performance level disk specification block storage with high performance and large storage capacity is split into the plurality of first performance level disk specification block storage, the logic storage paths of users are dynamically allocated, and the elastic scheduling capability is added, so that the data storage cost is greatly reduced, the users are not blocked when logging in, the limit of the online quantity of the traditional third performance level disk specification block storage is broken through, and the problems of logging in, data storage and the like of ultra-large-scale users are solved.

The embodiment of the application provides a data storage management method, which comprises the following steps: responding to the login operation of the user, and obtaining a user data distribution storage path query request message sent by the user side and used for querying the storage path distributed by the user data of the current user; distributing a storage path query request message according to the user data to obtain attribute information of the cloud storage device; and distributing a storage path for the user data according to the attribute information of the cloud storage device. According to the data storage management method, the storage path can be dynamically allocated to the user data according to the attribute information of the cloud storage device.

Second embodiment

In the first embodiment described above, a data storage management method is provided, and in response thereto, a second embodiment of the present application provides a data storage management apparatus. Since the apparatus embodiment is substantially similar to the first embodiment of the method, the description is relatively simple, and reference is made to the description of the method embodiment for relevant points. The device embodiments described below are merely illustrative.

Referring to fig. 3, a schematic diagram of a data storage management device according to a second embodiment of the present application is provided.

The data storage management device 300 includes:

A first obtaining unit 301, configured to obtain, in response to a login operation of a user, a user data allocation storage path query request message sent by a user side for querying a storage path allocated by user data of a current user;

a second obtaining unit 302, configured to obtain attribute information of the cloud storage device according to the storage path query request message allocated by the user data;

and the allocation unit 303 is configured to allocate a storage path to the user data according to the attribute information of the cloud storage device.

Third embodiment

Corresponding to the method embodiment of the application, the third embodiment of the application also provides an electronic device. Fig. 4 is a schematic view of an electronic device according to a third embodiment of the present application, as shown in fig. 4. The electronic device includes: at least one processor 401, at least one communication interface 402, at least one memory 403, and at least one communication bus 404; alternatively, the communication interface 402 may be an interface of a communication module, such as an interface of a GSM module; the processor 401 may be a processor CPU or a specific integrated circuit ASIC (Application Specific Integrated Circuit) or one or more integrated circuits configured to implement embodiments of the present invention. Memory 403 may comprise high-speed RAM memory and may also comprise non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 403 stores a program, and the processor 401 calls the program stored in the memory 403 to execute the method provided in the above embodiment of the present application.

Fourth embodiment

The fourth embodiment of the present application also provides a computer storage medium corresponding to the above method of the present application. The computer storage medium stores a computer program that is executed by a processor to perform the methods provided in the above-described embodiments of the present application.

While the preferred embodiment has been described, it is not intended to limit the invention thereto, and any person skilled in the art may make variations and modifications without departing from the spirit and scope of the present invention, so that the scope of the present invention shall be defined by the claims of the present application.

In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). Memory is an example of computer-readable media.

1. Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer readable media, as defined herein, does not include non-transitory computer readable media (transmission media), such as modulated data signals and carrier waves.

2. It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

It should be noted that, in the embodiments of the present application, the use of user data may be involved, and in practical applications, user specific personal data may be used in the schemes described herein within the scope allowed by applicable legal regulations in the country where the applicable legal regulations are met (for example, the user explicitly agrees to the user to actually notify the user, etc.).

It should be noted that, the user information (including but not limited to user equipment information, user personal information, etc.) and the data (including but not limited to data for analysis, stored data, presented data, etc.) related to the present application are information and data authorized by the user or fully authorized by each party, and the collection, use and processing of the related data need to comply with the related laws and regulations and standards of the related country and region, and provide corresponding operation entries for the user to select authorization or rejection.

Claims (14)

1. A data storage management system, comprising: the system comprises a management platform, an execution platform and an infrastructure layer;

the management platform comprises: a scheduling layer and a management layer; the scheduling layer is used for deciding whether storage capacity adjustment operation is needed for the cloud storage device or not; the management layer is used for managing the storage path of the user data;

the scheduling layer comprises a scheduling decision module and a scheduling execution module; the scheduling decision module is used for generating a scheduling instruction and sending the scheduling instruction to the scheduling execution module for execution; the scheduling execution module is used for calling an interface of the cloud storage device management service module to execute storage capacity adjustment operation on the cloud storage device based on the scheduling instruction; the management layer comprises a user storage path management module, wherein the user storage path management module is used for managing a storage path of user data;

the execution platform comprises: the cloud storage device comprises a storage atom management and control module and a cloud storage device atom management and control module; the storage atom management and control module is used for managing and controlling the storage of the user data; the cloud storage device atomic management and control module is used for managing and controlling the cloud storage device;

The infrastructure layer comprises a plurality of distributed cloud storage devices, and is used for storing user data;

the management platform is used for sending a management and control instruction to the execution platform according to user data management requirements and cloud storage device management requirements, the execution platform is used for executing the management and control instruction, the basic setting layer is used for providing a cloud storage device management interface for the execution platform, and the execution platform is further used for utilizing the cloud storage device management interface to manage and control storage of user data and cloud storage devices.

2. The data storage management system of claim 1, wherein the management platform further comprises: a data acquisition module;

the data acquisition module is used for acquiring attribute information of the cloud storage device and sending the attribute information to the scheduling decision module;

the scheduling decision module is used for generating scheduling instructions, and comprises:

the scheduling decision module is used for generating scheduling instructions according to the attribute information, and the scheduling instructions are used for judging whether storage capacity adjustment operation needs to be executed for the cloud storage device or not.

3. The data storage management system of claim 1, wherein the attribute information includes at least one of the following information:

Storage capacity information of the cloud storage device;

the consumption information of the cloud storage device;

user presence information;

the cloud storage device inputs and outputs data information.

4. The data storage management system of claim 2, wherein the scheduling instructions comprise: an operation instruction for expanding the capacity, an operation instruction for reducing the capacity, and an operation instruction for keeping the capacity unchanged;

the storage capacity adjustment operation includes: an operation of expanding the capacity, an operation of reducing the capacity, and an operation of keeping the capacity unchanged.

5. The data storage management system of claim 1, wherein the management layer further comprises: a storage strategy module; the storage strategy module is used for setting a storage strategy of the user data; the storage policy includes: storing a capacity expansion strategy and a roaming strategy; the storage expansion and contraction strategy is a strategy that a pointer executes expansion of storage capacity and reduction of storage capacity on a cloud storage device; the stored roaming policy is a policy in which pointers reserve and clean up user data.

6. The data storage management system of claim 1, wherein the user storage path management module comprises: a storage path allocation sub-module, a storage path updating sub-module, a storage path deleting sub-module and a storage path inquiring sub-module; the allocation storage path sub-module is used for allocating a storage path for the user data, the update storage path sub-module is used for updating the storage path of the user data when the storage path of the user data changes, the deletion storage path sub-module is used for deleting the storage path of the user data, and the query storage path sub-module is used for querying the storage path of the user.

7. The data storage management system of claim 1, wherein the cloud storage management services module comprises at least one of a cloud storage expansion capacity management sub-module, a cloud storage reduction capacity management sub-module, a cloud storage capacity query management sub-module, and a user data storage path management sub-module.

8. The data storage management system of claim 1, wherein the cloud storage device atomic management comprises at least one of a cloud storage device management sub-module, a user data management sub-module, and a storage path management sub-module;

the cloud storage device management and control submodule is used for managing and controlling the cloud storage device, the user data management and control submodule is used for managing and controlling the user data, and the storage path submodule is used for managing and controlling a user data storage path.

9. A data storage management method, comprising:

responding to the login operation of the user, and obtaining a user data distribution storage path query request message sent by the user side and used for querying the storage path distributed by the user data of the current user;

distributing a storage path query request message according to the user data to obtain attribute information of the cloud storage device;

And distributing a storage path for the user data according to the attribute information of the cloud storage device.

10. The data storage management method according to claim 9, further comprising, after responding to a login operation of a user: judging whether the user is a new user or not;

if the user is a new user, carrying out authorization management on the new user;

and if the user is not a new user, performing authority verification on the user.

11. The data storage management method of claim 9, further comprising:

determining whether the user logs in again within a preset time length according to the login state of the user;

and if not, migrating the user data to a designated cloud storage device.

12. The data storage management method of claim 9, further comprising:

if the user is set to be in a deleting state, migrating the user data to a designated cloud storage device or performing cold standby storage on the user data;

and if the user is restored to the normal state within the designated time, restoring the user data to the original cloud storage device.

13. An electronic device comprising a processor and a memory;

The memory having stored therein a computer program which, when executed by the processor, performs the method of any of claims 9-12.

14. A computer storage medium, characterized in that the computer storage medium stores a computer program which, when executed by a processor, performs the method of any of claims 9-12.