CN102035884B - Cloud storage system and data deployment method thereof - Google Patents

Abstract

The invention discloses a cloud storage system, which is used to provide cloud storage services for users, and is characterized in that the cloud storage system includes: a cloud storage service provider (103), a first-level agent node (102), and a second-level agent node and storage cloud (105), the user sends a service request, the first-level proxy node selects the best cloud storage service provider configuration service plan according to the request, and the second-level proxy node selects the cloud storage service provider in the data storage cloud The storage center implements cloud storage services. The invention also provides a data deployment method of the cloud storage system, which is applied to delete, modify or add user data. The present invention can support services provided by various cloud storage service providers, speed up the response speed of users using services, and can dynamically configure the underlying storage architecture according to the characteristics of users, so as to have the advantages of master-slave and peer-to-peer storage architectures at the same time .

Description

A cloud storage system and data deployment method thereof

technical field

The invention relates to network data storage technology, in particular to a cloud storage system and a data deployment method thereof.

Background technique

With the increasing development of cloud storage technology, the number and types of cloud storage service providers will also continue to increase. However, the existing cloud storage systems all adopt a closed structure, which can only support data storage services provided by specific cloud storage service providers; moreover, various cloud storage service providers cannot Efficient exchange and sharing of data resources. This will hinder the diversified development of cloud storage services and the cooperation between cloud storage service providers, making users limited to a certain cloud storage service provider.

In the traditional network storage method, users usually communicate directly with the data storage center. This method is feasible when the number of users is small and the amount of data storage is not large. However, in the cloud storage system, with the rapid increase in the number of users using cloud storage services and the relatively slow increase in network transmission speed, if this method is still used, the response time of user requests will be too long, which will affect cloud storage. the quality of the service.

At present, there are two typical cloud storage architectures. One is the master-slave storage architecture represented by Google. The main advantage of this architecture is that it is easy to maintain and data synchronization and update; the other is represented by Amazon. The peer-to-peer storage architecture, the main advantages of this architecture are no hot spots, no single point of failure, and strong self-management. However, there is currently no cloud storage system that can dynamically configure the underlying architecture on demand according to the user's spatiotemporal behavior characteristics, so as to maximize the adaptability of the storage architecture to the environment.

Contents of the invention

The technical problem to be solved by the present invention is to provide a cloud storage system and its data deployment method, so that it can support services provided by various cloud storage service providers, speed up the response speed of users using services, and can The spatio-temporal behavior characteristics dynamically configure the underlying storage architecture to have the advantages of both master-slave and peer-to-peer storage architectures.

In order to solve the above technical problems, the present invention provides a cloud storage system, so that users can efficiently obtain cloud storage services from various cloud storage service providers. The cloud storage system includes:

1. Cloud storage service provider: its role is to provide users with data storage services according to their requests;

2. First-level agent node: select the appropriate cloud storage service provider configuration scheme according to the user's selection strategy;

3. Secondary agent node: select the appropriate data storage center configuration scheme according to the user's selection strategy;

4. Storage cloud: including data storage center and auxiliary storage nodes;

5. User: refers to a terminal that can be connected to the Internet, including ordinary terminals or mobile terminals, which sends service requests and uses the best cloud storage service provider configuration selected by the first-level proxy node to provide a service plan.

The cloud storage service provider stores data in multiple storage clouds, and the storage cloud is composed of a data storage center and auxiliary storage nodes.

The data storage center includes multiple virtual nodes, and data is stored in the virtual nodes. The virtual node is an abstract node formed by deploying virtualized storage software on a real physical node.

Further, one or more virtual nodes may be deployed on a real physical node, or one virtual node may be deployed on multiple real physical nodes.

Further, the virtual nodes are connected through an internal private network.

Further, the peer-to-peer structure may be completely adopted among the virtual nodes, which are composed of virtual peer-to-peer nodes, and each of the virtual peer-to-peer nodes is a virtual node capable of simultaneously storing user data information and corresponding metadata information.

Further, the virtual nodes may also completely adopt a master-slave structure, consisting of virtual management nodes and virtual storage nodes. The virtual management node is a virtual node for storing and managing metadata information, and is responsible for managing virtual storage nodes. The virtual storage node is a virtual node for storing user data information.

Further, the virtual nodes can adopt a peer-to-peer structure and a master-slave structure at the same time, first configure the virtual nodes adopting the master-slave structure to form a master-slave structure network; then configure other non-master-slave structures The said virtual nodes are in a peer-to-peer structure, forming a peer-to-peer structure network, and then each master-slave structure network is added to the peer-to-peer structure network as a whole.

The peer-to-peer structure network may have at most one or none in one data storage center.

The master-slave network may have multiple or no master-slave networks in one data storage center. When there is no master-slave network or there is only one virtual node in the master-slave network, the data storage center That is, a peer-to-peer structure is completely adopted; when there is only one master-slave network and includes all the virtual nodes in the data storage center, the data storage center completely adopts a master-slave structure.

Further, whether the user's data is stored in the master-slave structure or in the peer-to-peer structure, or both in the master-slave structure and in the peer-to-peer structure, mainly depends on the user's spatiotemporal characteristics and data characteristics. for dynamic configuration. When the user's spatiotemporal characteristics or data characteristics change, the underlying storage structure for storing data can be dynamically adjusted, and data can also be migrated to different storage architectures.

The spatio-temporal characteristics of the user mainly refer to the characteristics of the user processing data within a specific time period or in a specific geographic location. For example, the number of data processing is particularly large during holidays, but less during normal times; the number of data processing at location A is particularly large, while that at location B is particularly small.

The characteristics of the data may include high data reliability, high availability, high security, and high access speed.

The auxiliary storage nodes are deployed in multiple different areas in the storage cloud. Generally, a storage cloud has one data storage center, but there are multiple auxiliary storage nodes.

Further, the data storage center can be connected to multiple auxiliary storage nodes; one auxiliary storage node can only be connected to one data storage center.

Further, the auxiliary storage node is used for storing user data whose usage frequency exceeds a preset threshold. The frequency of use refers to the number of times data is accessed within a specific time period.

Further, the data stored in the auxiliary storage node has no other backup in the auxiliary storage node, and its backup is placed in the connected data storage center.

Further, in order to simplify its structure, the auxiliary storage node adopts a traditional storage architecture (such as SAN, NAS, etc.), but does not limit which typical cloud storage architecture is used.

Further, the auxiliary storage node may be composed of a plurality of virtual nodes, or may be composed of physical nodes, which are internally connected by a dedicated network.

The first-level proxy node is composed of a first-level decision-making layer and a first-level scheduling layer.

The first-level decision-making layer is based on the user's request information and the information of the cloud storage service provider, and according to a specific selection strategy, to help the user choose the most suitable configuration of the cloud storage service provider.

The selection strategy is mainly considered from four aspects: storage cost, geographical location, access speed and security level.

The first-level scheduling layer is responsible for the communication between the user and the cloud storage service provider selected by the first-level decision-making layer, and synchronizes and shares the cloud storage service provider with other first-level agent nodes information; when the load of the first-level proxy node exceeds a preset threshold, the user is transferred to other said first-level proxy nodes.

Further, after obtaining the most suitable cloud storage service provider, the user obtains the most suitable data storage center of the cloud storage service provider through a secondary proxy node. The user then communicates with the data storage center.

The second-level proxy node is composed of a second-level decision-making layer and a second-level scheduling layer.

The secondary decision-making layer is based on the user's request information and the information of the data storage center, and according to a specific selection strategy, to help the user choose the most suitable data storage center.

The second-level scheduling layer is responsible for the communication between the user and the data storage center selected by the second-level decision-making layer, and synchronizes the information of the data storage center with other second-level proxy nodes; When the load of the second proxy node exceeds a preset threshold, the user is transferred to other secondary proxy nodes.

In order to solve the above technical problem, the present invention also discloses a data deployment method after the user operates the data, so as to facilitate the user to quickly process the data and reduce the response time for the user to interact with the data storage center. The data deployment method includes:

1. Data deployment method after adding user data;

2. Data deployment method after modifying user data;

3. Data deployment method after deleting user data.

Further, the data deployment method after adding user data is performed in the following steps:

1. The user establishes a connection with the data storage center and sends an increase request;

2. The data storage center finds the auxiliary storage node that is most suitable for the user to add data through a preset strategy;

3. The user establishes a communication connection with the auxiliary node, and adds data to the auxiliary storage node; the auxiliary storage node simultaneously creates a temporary copy of the user data;

4. When the auxiliary storage node has finished receiving user data, send a message of completion of adding data to the data storage center, and the data storage center initializes the usage frequency of the data and the location information of the auxiliary storage node , and send the message that the addition is completed to the user;

5. The auxiliary storage node and the data storage center regularly update data synchronously according to a preset strategy, so as to ensure that there is new data added by the user in the data storage center;

6. After the synchronous update, delete the temporary copy of the newly added data in the auxiliary storage node to free up more space for storing other data;

7. When the usage frequency of the newly added data does not reach the preset threshold after a preset period of time, the data will be deleted in the auxiliary storage node, and the metadata of the data will be modified. The location information of the auxiliary storage node described above.

Further, the method for the user to establish a connection with the data storage center may be any of the following:

1. The user selects the most suitable cloud storage service provider through the first-level proxy node; then, within the scope of the data storage center associated with the selected cloud storage service provider, the second-level proxy node selects the most suitable cloud storage service provider. Appropriate said data storage center to establish a connection.

2. The user selects the most suitable cloud storage service provider through the first-level proxy node; and then directly communicates with the specific data storage center within the scope of the data storage center associated with the selected cloud storage service provider to establish a connection.

3. The user directly specifies the specific cloud storage service provider; then within the scope of the data storage center associated with the selected cloud storage service provider, the secondary proxy node selects the most suitable data storage center to establish a connection.

4. The user directly specifies the specific cloud storage service provider and the data storage center to establish a connection.

Further, the data deployment method after modifying user data is performed in the following steps:

1. The user establishes a connection with the data storage center and sends a modification request;

2. The data storage center finds the required data through a search process;

3. If a piece of data requested by the user is stored in the auxiliary storage node, the user modifies the data in the auxiliary storage node. After the auxiliary storage node receives the modified data from the user, it sends the modification completion information to the data storage center, and the data storage center updates the data usage frequency information, and sends the modification completion information to the user, and then the data The storage center performs synchronous updates with the data modified by the user in the auxiliary storage node according to the preset strategy;

4. If the user data is only stored in the data storage center, the user directly modifies the data in the data storage center, and the data storage center updates the frequency of use of the data after accepting the modified data from the user, and Send the modified information to the user;

5. If the user only browses and views the data without modifying it, it is divided into two cases for processing: one is that the data is stored in the auxiliary storage node, and after the user views the data in the auxiliary storage node, then Send relevant information to the data storage center, and the data storage center updates the usage frequency of the data; in another case, if the data is stored in the data storage center, the user directly updates the usage frequency of the data after viewing the data in the data storage center frequency.

6. The data storage center checks the frequency of use of user data according to a preset policy, when the frequency of use of user data is higher than a set threshold and the user data is not stored in any of the auxiliary storage nodes , send the user data to the auxiliary storage node, and modify the location information of the auxiliary storage node in the metadata of the data, when the usage frequency of the user data is lower than the set threshold and the user data is stored When in the auxiliary storage node, delete the data stored in the auxiliary storage node, and modify the location information of the auxiliary storage node in the metadata of the data.

Further, the data deployment method after deleting user data is performed in the following steps:

1. The user establishes a connection with the data storage center and sends a deletion request;

2. The data storage center finds the required data through a search process;

3. If there is a copy of the data requested by the user stored in the auxiliary storage node, delete the data in the auxiliary storage node; after the deletion is completed, the auxiliary storage node sends the deletion completion information to the data storage center , the data storage center deletes all backups of the data, but retains a copy of the user data, and sets a remaining period of time to prevent the loss of frequently used data due to improper operations by the user, and to facilitate data recovery by the user. When the remaining period is zero, delete the user data;

4. If the user data is only stored in the data storage center, directly delete the user data and its backup data.

Further, the method for the user to establish a connection with the data storage center may be any of the following:

1. The user selects the most suitable cloud storage service provider through the first-level proxy node; then, within the scope of the data storage center associated with the selected cloud storage service provider, the second-level proxy node selects the most suitable cloud storage service provider. Appropriate said data storage center to establish a connection.

2. The user selects the most suitable cloud storage service provider through the first-level proxy node; and then directly communicates with the specific data storage center within the scope of the data storage center associated with the selected cloud storage service provider to establish a connection.

3. The user directly specifies the specific cloud storage service provider; then within the scope of the data storage center associated with the selected cloud storage service provider, the secondary proxy node selects the most suitable data storage center to establish a connection.

4. The user directly specifies the specific cloud storage service provider and the data storage center to establish a connection.

Before the user modifies and deletes the data, he must first find the required data through the search process. The steps for the data storage center to find the required data through the search process include:

1. The user sends a search request to the data storage center;

2. The data storage center searches for the data requested by the user;

3. If the data requested by the user is stored in the data storage center, it can be confirmed whether the data is still stored in the auxiliary storage node through the location information of the auxiliary storage node in the metadata of the data;

4. If the data requested by the user does not exist, return the information that the data does not exist to the user.

Compared with the existing cloud storage system, the present invention has at least the following advantages:

(1) The present invention introduces a two-level agency model to help users intelligently select the most suitable cloud storage service provider and the most suitable data storage center. It supports multiple cloud storage service providers and multiple data storage centers, which is conducive to providing diversified cloud storage services and broadening the range of choices for users.

(2) The present invention can dynamically configure the underlying cloud storage architecture of the data storage center, and can flexibly adjust the underlying storage architecture without interrupting the service. The underlying architecture can be completely a master-slave structure or a peer-to-peer structure. It can also be a combination of the two structures. In this way, it can not only have the advantages of no hot spot, no single point of failure, and strong self-management of the peer-to-peer structure; it also has the advantages of easy maintenance of the master-slave structure, and easy data synchronization and update.

(3) The present invention sets auxiliary storage nodes around the data storage center to save the most frequently used (system configurable) data resources by users, which not only shortens the distance between the user and the required data resources, but also shortens the The response time of the cloud storage system to user requests; and effectively alleviate the access pressure of the data storage center.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the description, together with the embodiments of the present invention, are used to explain the present invention, and do not constitute a limitation of the present invention. In the attached picture:

FIG. 1 is a schematic diagram of the composition and structure of a cloud storage system according to an embodiment of the present invention.

FIG. 2 is a schematic structural diagram of a first-level proxy node according to an embodiment of the present invention.

Fig. 3 is a schematic structural diagram of a cloud storage service provider according to an embodiment of the present invention.

FIG. 4 is a schematic diagram of the structure of a secondary proxy node according to an embodiment of the present invention.

FIG. 5 is a schematic diagram of a storage cloud structure according to an embodiment of the present invention.

FIG. 6 is a schematic structural diagram of a data storage center according to an embodiment of the present invention.

Fig. 7 is a schematic diagram of metadata information structure according to an embodiment of the present invention.

Fig. 8 is a schematic flow diagram of a user selecting a cloud storage system in an embodiment of the present invention.

FIG. 9 is a schematic flow diagram of a user adding data to a cloud storage system in an embodiment of the present invention.

FIG. 10 is a schematic flow diagram of a user searching for data in a cloud storage system in an embodiment of the present invention.

FIG. 11 is a schematic flow diagram of a user modifying data in a cloud storage system in an embodiment of the present invention.

FIG. 12 is a schematic flow diagram of a user deleting data in a cloud storage system in an embodiment of the present invention.

Detailed ways

The implementation of the present invention will be described in detail below in conjunction with the accompanying drawings and examples, so as to fully understand and implement the process of how to apply technical means to solve technical problems and achieve technical effects in the present invention.

It should be noted that, if there is no conflict, the embodiments of the present invention and various features in the embodiments can be combined with each other, and all are within the protection scope of the present invention. In addition, although a logical order is shown in the flowcharts, in some cases the steps shown or described may be performed in an order different from that shown or described herein.

FIG. 1 is a schematic diagram of the composition and structure of a cloud storage system. The system embodiment mainly includes a user 101 , a primary proxy node 102 , a cloud storage service provider 103 , a secondary proxy node 104 and a storage cloud 105 . in:

The user 101 selects the most suitable cloud storage service provider 103 through the primary proxy node 102 , and then selects the storage cloud 105 through the secondary proxy node 104 .

The user 101 can be an ordinary terminal user or a mobile terminal user, and only needs to be able to connect to the Internet.

FIG. 2 is a schematic diagram of the composition and structure of a first-level agent node 102 . This system embodiment mainly includes a first-level decision-making layer 201 and a first-level scheduling layer 202 . in:

The first-level decision-making layer 201 selects the most suitable cloud storage service provider 103 for the user according to the information provided by the user. The information provided by the user may include preferentially selecting the cloud storage service provider with the lowest storage price, or the one with the closest distance Cloud storage service providers, but not limited thereto.

The functions of the first-level scheduling layer 202 mainly include three aspects: one is to ensure normal communication between the user and the selected cloud storage service provider 103; the other is to synchronize cloud storage services with other first-level agent nodes 102 information of the provider 103; thirdly, when the load of the proxy node exceeds a preset value, transfer the user to another first-level proxy node 102 .

FIG. 3 is a schematic diagram of the composition and structure of a cloud storage service provider 103. This system embodiment mainly includes a large number of storage clouds 105, and each storage cloud 105 can communicate through an internal dedicated network or an optical fiber communication network. The storage clouds between different cloud storage service providers communicate through the external network, and the external network can be ordinary 100M Ethernet or Gigabit Ethernet. The main purpose of connecting different cloud storage service providers is to better exchange or share the data they own.

FIG. 4 is a schematic diagram of the composition structure of the secondary proxy node 104, the system embodiment mainly includes a secondary decision-making layer 401 and a secondary scheduling layer 402, wherein:

The secondary decision-making layer 401 mainly selects the most suitable data storage center 106 to provide to the user according to the information provided by the user. The selection information provided by the user may include preferentially selecting the data storage center with the fastest access speed, or preferentially selecting the security level The highest data storage center, but not limited to it.

The function of the secondary scheduling layer 402 mainly has three aspects: one is to ensure the normal communication between the user and the selected data storage center 106; information; thirdly, when the load of the proxy node exceeds a preset value, the user is transferred to other secondary proxy nodes 104 .

One of the core ideas of the present invention is to use a two-level agent, the user selects the most suitable cloud storage service provider by using the first-level agent node, and selects the most suitable data storage center by using the second-level agent; the user does not need to use a The first-level agent can directly designate a specific cloud storage service provider, or directly designate a specific data storage center in the selected cloud storage service provider without using a second-level agent.

Fig. 7 is a schematic diagram of the composition and structure of the metadata of the data. The metadata information includes basic information such as data copy information and routing information. The present invention adds the usage frequency 702 of the data and the location information 703 of the auxiliary storage node to the metadata. ,in:

The frequency of use 702 records the frequency of use of the data, and the calculation method of the frequency of use is the number of visits to a certain data within a certain period of time.

The location information 703 of the auxiliary storage node is to record which auxiliary storage node the data is stored in. If the usage frequency does not reach the preset threshold, the data will not be stored in the auxiliary storage node, and the location information of the auxiliary storage node will be Is empty.

Fig. 5 is a schematic diagram of the composition and structure of the storage cloud 105. This system embodiment mainly includes an auxiliary storage node 501 and a data storage center 106. The auxiliary storage node 501 is used to store data resources whose frequency of use by users exceeds the threshold preset by the system. The auxiliary Each data resource in the storage node 501 stores only one copy, and the data storage center 106 is used for storing original data and backup data. When a user operates the data resources stored in the auxiliary storage node 501, the usage frequency 702 of the corresponding data resource is updated, and the auxiliary storage node 501 regularly scans the usage frequency 702 of each data resource. When the usage frequency 702 of a data resource is low When the threshold value set by the system is reached, the data resource is deleted to free up more storage space for other data resources with higher usage frequency. The threshold can be manually configured as required.

The auxiliary storage node 501 may be a virtual node 503 or a real physical node 502 , or may be composed of a virtual node 503 and a real physical node 502 . Among them, the virtual node 503 is constructed by deploying virtualization software on the real physical node 502; one physical node 502 can run multiple virtual nodes 503, and one virtual node 503 can also run on multiple physical nodes 502. By using virtualized nodes, resource utilization and load balancing can be better improved, and security is also higher.

The data storage center 106 regularly checks the use of each data, and for data resources whose usage frequency exceeds a preset threshold and is not stored in any auxiliary storage node 501, the data storage center 106 will send a copy of data to the auxiliary storage node 501 in. And update the auxiliary storage node information 703 of the data, and record the location of the auxiliary storage node.

Wherein, the frequency of use 702 records the frequency of use of the data, and the calculation method of the frequency of use is the number of visits to a certain data within a certain period of time. The location information 703 of the auxiliary storage node is to record which auxiliary storage node the data is stored in. If the usage frequency does not reach the preset threshold, the data will not be stored in the auxiliary storage node, and the location information of the auxiliary storage node will be will be empty.

The usage frequency 702 and the location information 703 of the auxiliary storage node are stored in the metadata information. FIG. 7 is a schematic diagram of a composition structure of metadata of data.

The second core idea of the present invention is to use an auxiliary storage node. When the user's usage frequency of a specific data resource in the data storage center exceeds a preset threshold, the data resource is copied and stored in the auxiliary storage node. The auxiliary node is selected through a preset strategy, and the strategy may be short distance priority, high access speed priority, etc., but not limited to these two. In this way, the response speed to user requests is improved, and the request pressure of the data storage center is relieved.

FIG. 6 is a schematic diagram of the composition and structure of the data storage center 106 . The system embodiment may be a master-slave storage network 603 or a peer-to-peer storage network 605 . in:

The master-slave storage network 603 mainly includes a virtual management node 601 and a virtual storage node 602 .

The peer-to-peer storage network 605 is a DHT distributed network, which may be completely composed of virtual peer nodes 604, or may be composed of multiple virtual peer nodes 604 and multiple master-slave storage networks.

The virtual management node 601 serves as the management node of the master-slave storage network 603 on the one hand, and serves as the virtual peer node 604 of the peer-to-peer storage network 605 on the other hand.

The virtual peer node 604 and the virtual management node 601 record the usage frequency 702 of each data item and whether the data item is stored in the auxiliary storage node 501, and if it is stored in the auxiliary storage node 501, record the location of the auxiliary storage node 501 .

When it is necessary to set the underlying architecture as master-slave, it is only necessary to migrate the virtual storage nodes 602 in the master-slave network 603, withdraw the virtual storage nodes 602 to be migrated from the master-slave storage network 603, and then serve as a The newly added virtual peer node 604 joins the peer-to-peer storage network 605 . The original user data stored in the virtual storage node 602 does not need to be modified, and can still be used after being added to the peer-to-peer storage network.

The third core idea of the present invention is to adopt a dynamically adjustable underlying storage architecture. Cloud storage service providers can adjust the underlying storage architecture of the data storage center 106 to a master-slave structure or a peer-to-peer structure according to their own needs, so as to make full use of the master-slave structure for easy maintenance and easy data synchronization and It can effectively use the advantages of no hot spots, no single point of failure, and strong self-management in the peer-to-peer structure.

Fig. 8 is a schematic flowchart of an embodiment of a method for a user to select a data storage center. The method shown in FIG. 8 begins with step S801.

In step S802, it is judged whether the user selects a cloud storage service provider through a first-level proxy, and if yes, then jumps to step S803; otherwise, jumps to step S811.

Step S803, if the user chooses to use a first-level proxy, send a related request to the first-level proxy service node.

Step S804, when the first-level proxy node receives the request from the user, it selects the most suitable cloud storage service provider through the first-level decision-making layer.

Step S805, after that, the first-level proxy node is responsible for the communication between the cloud storage service provider and the user through the first-level scheduling layer, and returns the information of the cloud storage service provider to the user.

Step S806, after receiving the information of the most suitable cloud storage service provider selected by the first-level proxy node, the user connects to the cloud storage service provider.

In step S811, if the user does not select a cloud storage service provider through a primary agent, the user directly connects to the designated cloud storage service provider.

Step S812, and then judge whether the user selects a data storage center by using a second-level proxy, and if so, go to step S813; otherwise, go to step S821.

Step S821, if the user does not use a secondary proxy node, directly establish a connection between the user and the data storage center designated by the user. So far, the connection with the designated data storage center in a certain cloud storage service provider has been established.

Step S813, if the user chooses to use the second-level agent to select the data storage center, send related request information to the second-level agent service node.

Step S814, the secondary agent selects the most suitable data storage center through the secondary decision-making layer.

Step S815, the second-level agent is responsible for the communication between the user and the selected data storage center through the second-level scheduling layer, and returns the information of the data storage center to the user.

Step S816, the user receives the information of the selected most suitable data storage center sent by the secondary agent, and connects to the data storage center.

Step S830, so far, the user can either directly designate a specific cloud storage service provider, or select the most suitable cloud storage service provider through a first-level proxy node, and either directly designate a specific data storage center, or pass two The level agent selects the most suitable data storage center, which is convenient for users to establish a connection with the data storage center of the cloud storage service provider.

Fig. 9 is a schematic flowchart of an embodiment of a method for a user to add data to a cloud storage system. The method shown in FIG. 9 begins with step S901.

Step S902, when the user sends a request for adding data, the data storage center selects the most suitable auxiliary storage node according to a pre-established strategy.

In step S903, the user directly adds data to the selected auxiliary storage node, and makes a temporary backup of the data.

Step S904, when the addition of user data is completed, the auxiliary storage node sends an addition completion message to the data storage center, and the data storage node creates metadata information of the data, and initializes the use frequency of the data and the location information of the auxiliary storage node.

Step S905, when the load is lower than a certain threshold, the temporary copies of all data in the auxiliary storage node are updated to the data storage center, and the data storage center adds backup information of the data and modifies related metadata information.

In step S906, the auxiliary storage node deletes all temporary copies of data, so as to free up more space for storing new data.

Finally, the adding step ends in step S907.

Fig. 10 is a schematic flowchart of an embodiment of a method for a user to search for data in a cloud storage system. The method shown in FIG. 10 starts with step S1001.

In step S1002, the user first sends a data search request to the data storage center.

Step S1003, whether the data is stored in the data storage center, if it is stored in the data center, go to step S094; otherwise, go to step S1007.

Step S1007, if the data is not stored in the data storage center, return the information that the data does not exist to the user.

Step S1004, if the data is stored in the data storage center, then check the auxiliary node location information of the data, if the data is still stored in the auxiliary storage node, then go to step S1005; otherwise, go to step S1006.

Step S1006, if the data is not stored in the auxiliary storage node, return the information of the data in the data storage center to the user.

Step S1005, if the data is stored in the auxiliary storage node, return the information of the data in the auxiliary storage node to the user.

Finally, the searching step ends at S1008.

Fig. 11 is a schematic flowchart of an embodiment of a method for a user to modify data in a cloud storage system. The method shown in FIG. 11 starts at step S1101.

Step S1102, the user finds the required data through the search process.

Step S1103, judge whether the data is stored in the auxiliary storage node, if yes, go to step S1109; otherwise, go to S1104.

Step S1104, if the data is not stored in the auxiliary storage node, the user directly establishes a connection with the data storage center to modify the data.

Step S1105, updating the usage frequency of the data.

Step S1106, judging whether the usage frequency of the data reaches the set threshold, if so, go to step S1107; otherwise, go to step S1112.

Step S1107, when the usage frequency of the data reaches the set threshold, select an appropriate auxiliary storage node according to the set algorithm; and then update the information of the auxiliary storage node in the metadata information of the data.

Step S1108, sending the data to the auxiliary storage node.

Step S1109, if the data to be searched is in the auxiliary storage node, establish a connection with the auxiliary storage node, and modify the data in the auxiliary storage node.

Step S1110, when the data modification is completed, the auxiliary storage node sends the modification completion information to the data storage center, and the data storage center updates the usage frequency of the data.

In step S1111, the data storage center periodically updates data with the auxiliary storage nodes.

Finally, the modifying step ends at S1112.

Fig. 12 is a schematic flowchart of an embodiment of a method for a user to modify data in a cloud storage system. The method shown in FIG. 12 starts at step S1201.

Step S1202, the user finds the required data through the search process.

Step S1203, judging whether the data is stored in the auxiliary storage node, if yes, jump to S1204; otherwise, jump to S1206.

Step S1204, if the data to be deleted is stored in the auxiliary storage node, delete the copy of the data in the data storage center, but keep a copy of the original data, because generally the data blocks that can be stored in the auxiliary storage node are less frequently used. High, the possibility of accidental deletion is relatively high, which is convenient for users to quickly restore; when the retained user data exceeds the set time value, it will be deleted.

Step S1205, after deleting the relevant data in the data storage center, delete the data stored in the auxiliary storage node.

Step S1206, if the data the user is looking for is not in the auxiliary storage node, directly delete the original data and all backup data of the data in the data storage center.

Finally, the deletion step ends at S1207.

So far, users can select cloud storage systems through two-level agents, and can add, search, modify and search operations on data by using auxiliary storage nodes.

The above are only specific embodiments of the present invention, and it should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made. These improvements and modifications It should also be regarded as the protection scope of the present invention.

Claims (14)

1. a cloud storage system is used to the user that the cloud stores service is provided, and it is characterized in that, this cloud storage system comprises:

Cloud storage service provider (103): it provides data storage service according to the service request that the user sends to the user;

First-level agent's node (102): be used for the selection strategy according to the user, select appropriate cloud storage service provider (103);

Storage cloud (105): be subordinated to cloud storage service provider (103), comprise data storage center (106), be used for carrying out the data storage; Also comprise auxiliary storage node (501), be used for depositing the data resource that user's frequency of utilization surpasses the predefined threshold value of system, all only storage is a for each data resource in this auxiliary storage node (501);

Secondary agent's node: according to the cloud storage service provider (103) of user selection, from the storage cloud (105) of its subordinate, select appropriate data storage center (106) to carry out the data storage; With

The user: it sends service request, and described first-level agent node is selected cloud storage service provider (103) according to this request, and the storage cloud (105) of the corresponding cloud of described secondary agent point selection storage service provider is realized the cloud stores service.

2. cloud storage system according to claim 1, it is characterized in that, described first-level agent node (102) comprises one-level decision-making level (201) and schedule level one layer (202), and described one-level decision-making level (201) gives the user according to the only cloud of the Information Selection storage service provider of cloud storage service provider (103) in user's service request; Described schedule level one layer (202) be used for guaranteeing between user and the selected cloud storage service provider (103) proper communication, with other first-level agent's node (102) between with the information of buyun storage service provider (103) and when the load of this first-level agent's node during above the value preset, transferring user arrives other first-level agent's node (102).

3. cloud storage system according to claim 2 is characterized in that, described cloud storage service provider information can for the preferential cheapest cloud of Selective storage storage service provider, perhaps be the nearest cloud storage service provider of preferential chosen distance.

4. described cloud storage system one of according to claim 1-3, it is characterized in that, described secondary agent node (104) comprises secondary decision-making level (401) and second-level dispatching layer (402), wherein, described secondary decision-making level (401) is used for offering the user according to the only data storage center of data storage center Information Selection (106) of user's service request; Described second-level dispatching layer (402) be used for guaranteeing between user and the selected data storage center (106) proper communication, with other secondary agent's node (104) between synchrodata storage center (106) information and when the load of this agent node during above the value preset, transferring user arrives other secondary agent's node (104).

5. cloud storage system according to claim 4 is characterized in that, described data storage center information can be selected the fastest data storage center of access speed for preferential, or preferentially selects the highest data storage center of level of security.

6. according to claim 1-3 and one of 5 described cloud storage systems, it is characterized in that, described auxiliary storage node (501) is dummy node (503) or real physical node (502), perhaps be to be formed by dummy node (503) and real physical node (502), wherein, described dummy node (503) is by making up at the real virtual storing software of physical node (502) deploy.

7. according to claim 1-3 and one of 5 described cloud storage systems, it is characterized in that, described data storage center (106) can be master-slave mode storage networking (603), it also can be reciprocity storage networking (605), wherein, described master-slave mode storage networking (603) comprises virtual management node (601) and dummy storage node (602); Described is the DHT distributed network to equation storage networking (605), and it can for being made of virtual peer node (604), perhaps be constituted by a plurality of virtual peer nodes (604) and a plurality of master-slave mode storage networking (603).

8. the data deployment method of each described cloud storage system of claim 1-7, being applied to increases user data in system, and the method comprises the steps:

(1) user sets up and being connected of described data storage center (106), and sends to increase and ask;

(2) described data storage center (106) finds the most suitable user to increase the auxiliary storage node (501) of data by predefined strategy;

(3) user establishes a communications link with this described auxiliary storage node (501), and increases data to this described auxiliary storage node (501), and this described auxiliary storage node (501) is set up an interim copy of this user data simultaneously;

(4) when this described auxiliary storage node (501) when receiving user data, send one and increase message that data finish to described data storage center (106), the described frequency of utilization of described data storage center (106) these data of initialization and described auxiliary storage node (501) positional information, and the information that the transmission increase is finished is to the user;

(5) described auxiliary storage node (501) and described data storage center (106) be according to the regular synchronous new data more of predefined strategy, to guarantee the data that have the user to newly increase in the described data storage center (106);

(6) after the synchronously renewal, delete the interim copy of newly-increased data in the described auxiliary storage node (501);

(7) when the data that newly increase after one predefined period, described frequency of utilization does not reach predefined threshold value, will in described auxiliary storage node (501), delete these data so, and revise auxiliary storage node described in this data metadata (501) positional information.

9. the data deployment method of each described cloud storage system of claim 1-7 is applied to revise user data in system, and the method comprises the steps:

(1) user sets up and being connected of described data storage center (106), and sends to revise and ask;

(2) described data storage center (106) finds the data of required modification by search procedure;

(3) if the data that have a user to ask are stored in described auxiliary storage node (501), then the user revises these data in the described auxiliary storage node (501), described auxiliary storage node (501) has received after user's the Update Table, the information that the transmission modification is finished is to described data storage center (106), described data storage center (106) is the described frequency of utilization information of new data more, and send to revise the information finished to the user, data storage center (106) upgrades synchronously according to user institute Update Table in predefined strategy and the auxiliary storage node afterwards;

(4) if user data only is stored in the described data storage center (106), then the user directly revises the data in the described data storage center (106), described data storage center (106) is accepted after user's the Update Table, upgrade the described frequency of utilization of these data, and the information that the transmission modification is finished is to the user;

(5) described data storage center (106) checks the described frequency of utilization of user data according to predefined strategy, when the described frequency of utilization of user data is higher than the threshold value of setting and this user data and is not stored in any described auxiliary storage node (501), send this user data in described auxiliary storage node (501), and revise described auxiliary storage node (501) positional information in the metadata of these data, when the described frequency of utilization of user data is lower than the threshold value of setting and this storage of subscriber data in described auxiliary storage node (501) time, deletion is stored in these data in the described auxiliary storage node (501), and revises described auxiliary storage node (501) positional information in the metadata of these data.

10. data deployment method according to claim 9, it is characterized in that, if the user only browses data and checks and when not revising, if data are stored in the auxiliary storage node (501), the user has checked these data in auxiliary storage node (501) after, then send relevant information to data storage center (106), data storage center (106) upgrades the frequency of utilization of these data; If data are stored in the data storage center (106), then the user directly upgrades the frequency of utilization of these data after data storage center (106) has been checked these data.

11. the data deployment method of each described cloud storage system of claim 1-7 is applied to delete user data in system, the method comprises the steps:

(1) user sets up and being connected of described data storage center (106), and the transmission removal request;

(2) described data storage center (106) finds needed data by search procedure;

(3) if the data that have a user to ask are stored in described auxiliary storage node (501), then delete these data in the described auxiliary storage node (501); After deletion is finished, the information that described auxiliary storage node (501) transmission deletion is finished is to described data storage center (106), described data storage center (106) is deleted all backups of these data, but keep a this user data, and set a residue time in time limit, because the improper operation of user causes the higher loss of data of frequency of utilization, make things convenient for the user to recover data to prevent, when the residue time in time limit is zero, again this part user data is deleted;

(4) if user data only is stored in the described data storage center (106), then directly delete this user data and Backup Data thereof.

12. one of according to claim 8-11 described method is characterized in that, the user set up with the mode that is connected of described data storage center (106) can for following any:

(A) user selects only described cloud storage service provider (103) by described first-level agent node; Then in data storage center (106) scope that selected cloud storage service provider (103) is associated, select only described data storage center (106) to connect by described secondary agent node;

(B) user selects only described cloud storage service provider (103) by described first-level agent node; Then in data storage center (106) scope that selected cloud storage service provider is associated, directly and the described data storage center (106) of appointment to connect;

(C) user directly specifies described cloud storage service provider (103), then in data storage center (106) scope that selected cloud storage service provider (103) is associated, select only described data storage center (106) to connect by described secondary agent node;

(D) user directly specifies described cloud storage service provider (103) and described data storage center (106) to connect.

13. one of according to claim 8-11 described method is characterized in that, comprises the frequency of utilization (702) of data and the positional information (703) of auxiliary storage node (501) in the metadata of described data.

14. one of according to claim 9-11 described method is characterized in that, described data storage center (106) finds the step of desired data to be specially by search procedure:

(1) user sends search request to described data storage center (106);

(2) described data storage center (106) is searched the data that the user asks;

(3) if the data that the user asks are stored in the described data storage center (106), can confirm by the described auxiliary storage node location information in the metadata of these data whether these data also are stored in the described auxiliary storage node (501);

(4) if the data that the user asks do not exist, then the non-existent information of return data is to the user.

Images