CN108038168A - Data outer storage method - Google Patents

Abstract

The invention discloses data outer storage method, it comprises the following steps：External storage system is managed, monitor, configures authority；For data to be stored, external storage system is selected by storing operation module；The operating right of external storage system selected by verification；Stored after data to be stored is converted to agreement form, and record fileinfo；Create the index file of the data to be stored for agreement form；Based on index file, search the storage location of corresponding data and be downloaded or delete operation.The present invention can realize the diversification of data external storage, and enterprise can flexibly select according to their needs, reduce the use cost and maintenance cost of business data external storage.

Description

Data storage method

technical field

The invention relates to the technical field of data storage, in particular to a method and system for external data storage.

Background technique

In the era of big data, the rapid popularization of applications such as mobile Internet, social network, data analysis, and cloud services has put forward revolutionary demands on data centers, and storage infrastructure has become one of the core IT. Emerging applications emerge in endlessly in various fields such as government, military industry, scientific research institutes, aerospace, large commercial chains, medical care, finance, new media, and radio and television. The value of data has become increasingly prominent, and data has become an indispensable asset. As a data carrier and driving force, the storage system has become the most critical core of the big data infrastructure.

Traditional data centers are far from meeting the needs of emerging applications in terms of performance, efficiency, return on investment, and security. Data center businesses urgently need new types of big data processing centers to support them. In addition to the traditional high reliability, high redundancy, and green energy saving, the new big data center also needs to have a series of features such as virtualization, modularization, elastic expansion, and automation to meet the application requirements with big data characteristics. These unprecedented demands have brought about unprecedented changes in the architecture and functions of storage systems.

Based on the application requirements of big data, the concept of "application-defined storage" was proposed. As the core data foundation of the data center, the storage system is no longer just a traditional scattered and single underlying device. In addition to features such as high performance, high security, and high reliability, virtualization, parallel distribution, automatic layering, elastic expansion, heterogeneous resource integration, and global cache acceleration are also required to meet the needs of large-scale Business application requirements for data characteristics. Big data puts forward higher requirements on the capacity, read and write performance, reliability, and scalability of storage devices. It is necessary to fully consider the degree of functional integration, data security, data stability, system scalability, performance, and cost. aspect factors.

The current storage architectures on the market mainly include the following types:

(1) Storage system based on embedded architecture

Embedded system is a big concept. For an embedded system with operating system and file system (such as embedded linux system), its data storage method (file) is not much different from that of a standard computer system. In addition to supporting some special In addition to standard file systems (such as yaffs, jffs), it also supports file systems implemented by many standard computer systems (such as ext2, ext3, fat, etc.) and network file systems. In addition, many embedded systems also support devices such as U disks and SD cards, which can completely exchange files with standard file systems. Benefits of embedded storage switching technology include higher reliability, better performance, and the ability to add hard drives without sacrificing performance.

(2) Storage system based on X86 architecture

A storage system based on the X86 architecture, such as the HDFS distributed file management system, is composed of a large number of X86 system devices at the bottom layer, which can greatly reduce the cost of enterprise data storage while ensuring fault tolerance and high data availability.

(3) Storage system based on cloud technology

Cloud storage is a new concept extended and developed on the concept of cloud computing. It is an emerging network storage technology. A large number of different types of storage devices work together through application software to jointly provide data storage and business access systems.

At present, when enterprises or institutions need to store data externally, they can only choose a single external storage architecture (such as cloud storage system), instead of using their own external storage resources to flexibly choose, which undoubtedly increases the The cost of external data storage.

Contents of the invention

The purpose of the present invention is to provide a method for external data storage, which can realize the diversification of external data storage, and enterprises can choose flexibly according to their own needs, reducing the use cost and maintenance cost of enterprise data external storage.

To achieve the above object, the present invention adopts the following technical solutions:

A method for external data storage, which is implemented based on an external data storage system. The external data storage system includes a system management module, a rights management module, and a storage operation module. The system management module is used to manage the basic information of the external storage system and monitor the external The operating state of the storage system, the authority management module is used to define the operation authority of each external storage system, and the storage operation module is used to perform corresponding data operations on the external storage system;

The method includes the following steps:

S1. The system management module collects and manages the basic information of each external storage system, and monitors the operation status of each external storage system. The authority management module sets the operation authority of each external storage system according to different user roles;

S2. For the data to be stored, select an external storage system through the storage operation module, and apply for storage space;

S3. The authority management module verifies whether the current user role has the operation authority of the external storage system corresponding to the application storage space, if so, execute step S4, otherwise return to step S2;

S4. The storage operation module converts the data to be stored into an agreed format and stores it in the applied storage space, and records the file information. The agreed format is a reversible AES encrypted format, and the file information includes storage location information and data number of bytes;

S5. Collect the file information, and create an index file for the data to be stored in the agreed format;

S6. Based on the index file, search for the storage location of the corresponding data through the storage operation module, and perform a download or delete operation.

Preferably, the external storage system includes an embedded storage system, an X86 storage system, a cloud storage system, an FTP server, and a cloud platform.

Preferably, the location information in step S4 includes data storage start location information and end location information.

Preferably, the step S4 also includes creating a data index for the data stored in the storage space; the step S6 also includes, based on the index file and the data index, searching for the data containing the corresponding information, the storage of the data location and where that information is stored, and read it.

After adopting the technical solution, the present invention has the following advantages compared with the background technology:

The invention can realize the diversification of external storage of data, and enterprises can choose flexibly according to their own needs, reducing the use cost and maintenance cost of external storage of enterprise data.

Description of drawings

Fig. 1 is a schematic flow chart of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

Example

Please refer to Fig. 1, the present invention discloses a data external storage method, which is implemented based on an external data storage system. The external data storage system includes a system management module, a rights management module and a storage operation module. The system management module is used to manage the basis of the external storage system information and monitor the operating status of the external storage system, the authority management module is used to define the operation authority of each external storage system, and the storage operation module is used to perform corresponding data operations on the external storage system. In this embodiment, the external storage system includes an embedded storage system, an X86 storage system, a cloud storage system, an FTP server, and a cloud platform. Of course, the external storage system may also include other types of storage architectures, which are not specifically limited in the present invention.

The data external storage method of the present invention comprises the following steps:

S1. The system management module collects and manages the basic information of each external storage system, and monitors the operating status of each external storage system. The operating status of the external storage system includes network connection information and remaining storage space information; the authority management module according to different User roles set the operation permissions for each external storage system.

S2. For the data to be stored, select an external storage system through the storage operation module, and apply for a storage space.

S3. The authority management module verifies whether the current user role has the operation authority of the external storage system corresponding to the application storage space, if so, execute step S4, otherwise return to step S2.

S4. The storage operation module converts the data to be stored into the agreed format and stores it in the applied storage space, and records the file information. The agreed format is a reversible AES encryption format, which can realize the encryption of the data to be stored and prevent the third party from directly knowing the data content. The file information includes storage location information and the number of data bytes, and the location information includes data storage start location information and end location information.

S5. Collect file information, and create an index file for the data to be stored in the agreed format, and the index file is in units of rows.

S6. Based on the index file, search for the storage location of the corresponding data through the storage operation module, and perform a download or delete operation. Of course, other data operations, such as data transfer, can also be performed. At this time, the data to be transferred is used as the data to be stored, and steps S4 and S5 are repeated.

In order to realize the direct reading of data after external storage, step S4 also includes creating a data index for the data stored in the storage space; step S6 also includes, based on the index file and data index, searching for data containing corresponding information, the data The storage location and the storage location of this information, and perform a read operation.

The above is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present invention can easily think of changes or Replacement should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be determined by the protection scope of the claims.

Claims (4)

1. The data external storage method is characterized in that the method is implemented based on an external data storage system, and the external data storage system includes a system management module, a rights management module and a storage operation module, and the system management module is used to manage the external storage system basic information and monitor the operating status of the external storage system, the authority management module is used to define the operation authority of each external storage system, and the storage operation module is used to perform corresponding data operations on the external storage system; The method includes the following steps: S1. The system management module collects and manages the basic information of each external storage system, and monitors the operation status of each external storage system. The authority management module sets the operation authority of each external storage system according to different user roles; S2. For the data to be stored, select an external storage system through the storage operation module, and apply for storage space; S3. The authority management module verifies whether the current user role has the operation authority of the external storage system corresponding to the application storage space, if so, execute step S4, otherwise return to step S2; S4. The storage operation module converts the data to be stored into an agreed format and stores it in the applied storage space, and records the file information. The agreed format is a reversible AES encrypted format, and the file information includes storage location information and data number of bytes; S5. Collect the file information, and create an index file for the data to be stored in the agreed format; S6. Based on the index file, search for the storage location of the corresponding data through the storage operation module, and perform a download or delete operation. 2. The data external storage method according to claim 1, wherein the external storage system includes an embedded storage system, an X86 storage system, a cloud storage system, an FTP server, and a cloud platform. 3. The method for external data storage according to claim 1 or 2, characterized in that: the location information in step S4 includes data storage start location information and end location information. 4. The method for external data storage according to claim 1, characterized in that: said step S4 also includes creating a data index for the data stored in the storage space; said step S6 also includes, based on said index file and data index, find the data containing the corresponding information, the storage location of the data, and the storage location of the information, and perform the read operation.