CN107193500A - A kind of distributed file system Bedding storage method and system - Google Patents
A kind of distributed file system Bedding storage method and system Download PDFInfo
- Publication number
- CN107193500A CN107193500A CN201710385437.5A CN201710385437A CN107193500A CN 107193500 A CN107193500 A CN 107193500A CN 201710385437 A CN201710385437 A CN 201710385437A CN 107193500 A CN107193500 A CN 107193500A
- Authority
- CN
- China
- Prior art keywords
- storage
- data
- disk
- stored
- cold
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 27
- 238000013500 data storage Methods 0.000 claims abstract description 138
- 238000001514 detection method Methods 0.000 claims description 8
- 239000007787 solid Substances 0.000 claims description 8
- 230000005012 migration Effects 0.000 claims description 7
- 238000013508 migration Methods 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 4
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0602—Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
- G06F3/0604—Improving or facilitating administration, e.g. storage management
- G06F3/0607—Improving or facilitating administration, e.g. storage management by facilitating the process of upgrading existing storage systems, e.g. for improving compatibility between host and storage device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
- G06F11/1464—Management of the backup or restore process for networked environments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0628—Interfaces specially adapted for storage systems making use of a particular technique
- G06F3/0646—Horizontal data movement in storage systems, i.e. moving data in between storage devices or systems
- G06F3/0647—Migration mechanisms
- G06F3/0649—Lifecycle management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/067—Distributed or networked storage systems, e.g. storage area networks [SAN], network attached storage [NAS]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/06—Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
- G06F3/0601—Interfaces specially adapted for storage systems
- G06F3/0668—Interfaces specially adapted for storage systems adopting a particular infrastructure
- G06F3/0671—In-line storage system
- G06F3/0683—Plurality of storage devices
- G06F3/0685—Hybrid storage combining heterogeneous device types, e.g. hierarchical storage, hybrid arrays
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明提供了一种分布式文件系统分层存储方法及系统，该方法包括：当接收到待存储的数据时，将待存储的数据存储到第一层存储目录下的热数据存储磁盘中；周期性的检测存储在热数据存储磁盘中的第一存储数据的访问次数；当第一存储数据的访问次数小于预设的第一热度阈值时，将第一存储数据迁移到第二层存储目录下的冷数据存储磁盘中，当冷数据存储磁盘中存储的数据所占的内存占冷数据存储磁盘存储容量的比例超过了预设的比例阈值时，将满足预设规则的第二存储数据在第三层存储目录下的备份磁盘中备份。通过本实施例的方法，将不同访问量的数据分层存储，不仅控制了磁盘的使用成本，而且简化了分布式文件系统数据存储的流程。
The present invention provides a distributed file system layered storage method and system, the method comprising: when receiving the data to be stored, storing the data to be stored in a hot data storage disk under the first layer storage directory; Periodically detect the number of accesses to the first storage data stored in the hot data storage disk; when the number of accesses to the first storage data is less than the preset first heat threshold, migrate the first storage data to the second-tier storage directory Among the cold data storage disks below, when the ratio of the memory occupied by the data stored in the cold data storage disk to the storage capacity of the cold data storage disk exceeds the preset ratio threshold, the second stored data that meets the preset rules will be in the Back up in the backup disk under the third-tier storage directory. Through the method of this embodiment, data with different access volumes are stored in layers, which not only controls the cost of using the disk, but also simplifies the process of data storage in the distributed file system.
Description
技术领域technical field
本发明涉及分布式文件系统存储的领域，尤其涉及一种分布式文件系统分层存储方法及系统。The invention relates to the field of distributed file system storage, in particular to a distributed file system layered storage method and system.
背景技术Background technique
目前，各种应用产生的数据成指数型、爆发式增长，产生的这些数据需要在分布式文件系统中进行存储。对于不同访问频次的数据，对存储硬盘的要求也不同，对于访问频次较高的热数据需要IOPS较高的磁盘，例如SSD(英文全称：Solid State Disk，中文全称：固态硬盘)、转速较高的SAS(英文全称：Serial Attached SCSI，中文全称：串行连接SCSI)盘或者闪存，而对于访问频次较低的冷数据可以用普通硬盘或者磁带库进行存储。At present, the data generated by various applications grows exponentially and explosively, and the generated data needs to be stored in the distributed file system. For data with different access frequencies, the requirements for storage hard disks are also different. For hot data with higher access frequencies, disks with higher IOPS are required, such as SSD (full name in English: Solid State Disk, full name in Chinese: Solid State Disk), high speed SAS (English full name: Serial Attached SCSI, Chinese full name: Serial Attached SCSI) disk or flash memory, and cold data with low access frequency can be stored with ordinary hard disk or tape library.
但是，现有技术中，无法根据数据的热度，智能化的存储到相应的磁盘中。However, in the prior art, it is impossible to intelligently store the data into corresponding disks according to the heat of the data.
发明内容Contents of the invention
有鉴于此，本发明实施例提供了一种分布式文件系统分层存储方法及系统，解决了现有技术中，无法根据存储数据的不同访问量，智能化的存储到相应的磁盘中，并且控制了磁盘的使用成本。In view of this, the embodiment of the present invention provides a distributed file system hierarchical storage method and system, which solves the problem that in the prior art, the storage data cannot be intelligently stored in the corresponding disk according to the different access volumes, and Disk usage costs are controlled.
本发明实施例提供的一种分布式文件系统分层存储方法，所述方法包括：A distributed file system hierarchical storage method provided by an embodiment of the present invention, the method includes:
当接收到待存储的数据时，将所述待存储的数据存储到第一层存储目录下的热数据存储磁盘中；When receiving the data to be stored, storing the data to be stored in the hot data storage disk under the first layer storage directory;
周期性的检测存储在所述热数据存储磁盘中的第一存储数据的访问次数；periodically detecting the number of accesses of the first storage data stored in the hot data storage disk;
当所述第一存储数据的访问次数小于预设的第一热度阈值时，将所述第一存储数据迁移到第二层存储目录下的冷数据存储磁盘中。When the number of accesses of the first stored data is less than the preset first temperature threshold, the first stored data is migrated to the cold data storage disk under the second-tier storage directory.
可选的，还包括：Optionally, also include:
建立第一层存储目录和第二层存储目录；Establish a first-tier storage directory and a second-tier storage directory;
将所述第一层存储目录绑定热数据存储磁盘；其中，所述热数据存储磁盘包括：SSD固态硬盘、SAS盘或者闪存；Binding the first-tier storage directory to a hot data storage disk; wherein, the hot data storage disk includes: SSD solid state disk, SAS disk or flash memory;
将所述第二层存储目录绑定冷数据存储磁盘；其中，述冷数据存储磁盘包括：普通硬盘。Binding the second-tier storage directory to a cold data storage disk; wherein, the cold data storage disk includes: a common hard disk.
可选的，还包括：Optionally, also include:
当所述冷数据存储磁盘中存储的数据所占的内存占所述冷数据存储磁盘存储容量的比例超过了预设的比例阈值时，确定所述冷数据存储磁盘中满足预设规则的第二存储数据；When the ratio of the memory occupied by the data stored in the cold data storage disk to the storage capacity of the cold data storage disk exceeds the preset ratio threshold, it is determined that the second memory of the cold data storage disk that meets the preset rule Storing data;
将所述第二存储数据在第三层存储目录下的备份磁盘中备份；Backing up the second storage data in a backup disk under the third-tier storage directory;
删除所述冷数据存储磁盘中的第二存储数据。Deleting the second storage data in the cold data storage disk.
可选的，所述确定所述冷数据存储磁盘中满足预设规则的第二存储数据，包括：Optionally, the determining the second stored data in the cold data storage disk that satisfies preset rules includes:
检测存储在所述冷数据存储磁盘中的存储数据的访问次数；detecting the number of accesses of the storage data stored in the cold data storage disk;
将在预设的时间段内访问次数均为零的所述存储数据确定为第二存储数据；determining the stored data whose access times are all zero within a preset time period as the second stored data;
或者，or,
判断存储在所述冷数据存储磁盘中的存储数据是否超过了预设时间的存储数据；judging whether the storage data stored in the cold data storage disk exceeds the storage data of a preset time;
将超过了预设时间的存储数据确定为第二存储数据。The stored data exceeding the preset time is determined as the second stored data.
可选的，还包括：Optionally, also include:
周期性的检测存储在冷数据存储磁盘中的第三存储数据的访问次数；Periodically detect the access times of the third storage data stored in the cold data storage disk;
当所述第三存储数据的访问次数大于预设的第二热度阈值时，将所述第三存储数据迁移到第一层存储目录下的热数据存储磁盘中。When the access times of the third stored data is greater than the preset second heat threshold, the third stored data is migrated to the hot data storage disk under the first layer storage directory.
本发明实施例还提供了一种分布式文件系统分层存储系统，所述系统包括：第一存储单元，用于当接收到待存储的数据时，将所述待存储的数据存储到第一层存储目录下的热数据存储磁盘中；An embodiment of the present invention also provides a distributed file system hierarchical storage system, the system includes: a first storage unit, configured to store the data to be stored in the first In the hot data storage disk under the layer storage directory;
第一检测单元，用于周期性的检测存储在所述热数据存储磁盘中的第一存储数据的访问次数；a first detection unit, configured to periodically detect the number of accesses of the first storage data stored in the hot data storage disk;
第一迁移单元，用于当所述第一存储数据的访问次数小于预设的第一热度阈值时，将所述第一存储数据迁移到第二层存储目录下的冷数据存储磁盘中。The first migration unit is configured to migrate the first stored data to the cold data storage disk under the second-tier storage directory when the number of accesses of the first stored data is less than a preset first heat threshold.
可选的，所述系统还可以包括：Optionally, the system may also include:
建立单元，用于建立第一层存储目录和第二层存储目录；An establishment unit is used to establish a first-level storage directory and a second-level storage directory;
第一绑定单元，用于将所述第一层存储目录绑定热数据存储磁盘；其中，所述热数据存储磁盘包括：SSD固态硬盘、SAS盘或者闪存；The first binding unit is configured to bind the first-tier storage directory to a hot data storage disk; wherein the hot data storage disk includes: SSD solid state disk, SAS disk or flash memory;
第二绑定单元，用于将所述第二层存储目录绑定冷数据存储磁盘；其中，述冷数据存储磁盘包括：普通硬盘。The second binding unit is configured to bind the second layer storage directory to a cold data storage disk; wherein the cold data storage disk includes: a common hard disk.
可选的，所述系统还可以包括：Optionally, the system may also include:
确定单元，用于当所述冷数据存储磁盘中存储的数据所占的内存占所述冷数据存储磁盘存储容量的比例超过了预设的比例阈值时，确定所述冷数据存储磁盘中满足预设规则的第二存储数据；A determination unit, configured to determine that the cold data storage disk meets the preset requirements when the ratio of the memory occupied by the data stored in the cold data storage disk to the storage capacity of the cold data storage disk exceeds a preset ratio threshold. Set the second storage data of the rule;
备份单元，用于将所述第二存储数据在第三层存储目录下的备份磁盘中备份；a backup unit, configured to back up the second storage data in a backup disk under the third-tier storage directory;
删除单元，用于删除所述冷数据存储磁盘中的第二存储数据。A deleting unit, configured to delete the second stored data in the cold data storage disk.
可选的，所述确定单元，包括：Optionally, the determining unit includes:
检测子单元，用于检测存储在所述冷数据存储磁盘中的存储数据的访问次数；a detection subunit, configured to detect the number of accesses of the storage data stored in the cold data storage disk;
第一确定子单元，用于将在预设的时间段内访问次数均为零的所述存储数据确定为第二存储数据；a first determining subunit, configured to determine the stored data whose access times are all zero within a preset time period as the second stored data;
或者，or,
判断子单元，用于判断存储在所述冷数据存储磁盘中的存储数据是否超过了预设时间的存储数据；A judging subunit, configured to judge whether the storage data stored in the cold data storage disk exceeds the storage data of a preset time;
第二确定子单元，用于将超过了预设时间的存储数据确定为第二存储数据。The second determining subunit is configured to determine the stored data exceeding the preset time as the second stored data.
可选的，所述系统还可以包括：Optionally, the system may also include:
第二检测单元，用于周期性的检测存储在冷数据存储磁盘中的第三存储数据的访问次数；The second detection unit is used to periodically detect the number of accesses of the third storage data stored in the cold data storage disk;
第二迁移单元，用于当所述第三存储数据的访问次数大于预设的第二热度阈值时，将所述第三存储数据迁移到第一层存储目录下的热数据存储磁盘中。The second migration unit is configured to migrate the third stored data to a hot data storage disk under the first-tier storage directory when the access times of the third stored data is greater than a preset second heat threshold.
本实施例中，当接收到待存储的数据时，将所述待存储的数据存储到第一层存储目录下的热数据存储磁盘中；周期性的检测存储在所述热数据存储磁盘中的第一存储数据的访问次数；当所述第一存储数据的访问次数小于预设的第一热度阈值时，将所述第一存储数据迁移到第二层存储目录下的冷数据存储磁盘中。通过本实施例的方法，将不同访问量的数据分层存储，不仅控制了磁盘的使用成本，而且简化了分布式文件系统数据存储的流程。In this embodiment, when the data to be stored is received, the data to be stored is stored in the hot data storage disk under the first layer storage directory; the data stored in the hot data storage disk is periodically detected The number of accesses of the first stored data; when the number of accesses of the first stored data is less than the preset first heat threshold, the first stored data is migrated to the cold data storage disk under the second-tier storage directory. Through the method of this embodiment, data with different access volumes are stored in layers, which not only controls the cost of using the disk, but also simplifies the process of data storage in the distributed file system.
附图说明Description of drawings
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only It is an embodiment of the present invention, and those skilled in the art can also obtain other drawings according to the provided drawings without creative work.
图1示出了本发明实施例提供的一种分布式文件系统分层存储方法的流程示意图；FIG. 1 shows a schematic flowchart of a distributed file system hierarchical storage method provided by an embodiment of the present invention;
图2示出了本发明实施例提供的一种分布式文件系统分层存储方法的流程示意图；FIG. 2 shows a schematic flowchart of a distributed file system hierarchical storage method provided by an embodiment of the present invention;
图3示出了本发明实施例提供的一种分布式文件系统分层存储方法的流程示意图；FIG. 3 shows a schematic flowchart of a distributed file system hierarchical storage method provided by an embodiment of the present invention;
图4示出了本发明实施例提供的一种分布式文件系统分层存储逻辑的示意图；FIG. 4 shows a schematic diagram of a distributed file system hierarchical storage logic provided by an embodiment of the present invention;
图5示出了本发明实施例提供的一种分布式文件系统分层系统的结构示意图。FIG. 5 shows a schematic structural diagram of a distributed file system layered system provided by an embodiment of the present invention.
具体实施方式detailed description
下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.
参考图1，示出了本发明实施例提供的一种分布式文件系统分层存储方法的流程示意图，在本实施例中，所述方法可以包括以下步骤中的S101～S103：Referring to FIG. 1 , it shows a schematic flowchart of a distributed file system hierarchical storage method provided by an embodiment of the present invention. In this embodiment, the method may include steps S101 to S103 in the following steps:
S101：当接收到待存储的数据时，将所述待存储的数据存储到第一层存储目录下的热数据存储磁盘中；S101: When receiving the data to be stored, store the data to be stored in the hot data storage disk under the first layer storage directory;
本实施例中，在S101之前还可以包括以下步骤中的S201～S203：In this embodiment, S201-S203 in the following steps may also be included before S101:
S201：建立第一层存储目录和第二层存储目录；S201: Establish a first-tier storage directory and a second-tier storage directory;
S202：将所述第一层存储目录绑定热数据存储磁盘；其中，所述热数据存储磁盘包括：SSD固态硬盘、SAS盘或者闪存；S202: Bind the first layer storage directory to a hot data storage disk; wherein the hot data storage disk includes: SSD solid state disk, SAS disk or flash memory;
S203：将所述第二层存储目录绑定冷数据存储磁盘；其中，所述冷数据存储磁盘包括：普通硬盘。S203: Bind the second-tier storage directory to a cold data storage disk; wherein, the cold data storage disk includes: a common hard disk.
本实施例中，第一层存储目录下的热数据存储磁盘可以理解为存储热度值较高的数据，也可以理解为存储访问次数较高的数据。第二层存储目录下的冷数据存储磁盘可以理解为存储热度值较低的数据，也可以理解为存储访问次数较低或者为零的数据。其中，第一层存储目录还可以表示为hotdir目录，第二层存储目录还可以表示为colddir目录。In this embodiment, the hot data storage disks under the first-tier storage directory can be understood as storing data with a higher heat value, and can also be understood as storing data with a higher number of access times. The cold data storage disks under the second-tier storage directory can be understood as storing data with a low heat value, and can also be understood as storing data with a low number of accesses or zero. Wherein, the first layer storage directory may also be represented as a hotdir directory, and the second layer storage directory may also be represented as a colddir directory.
本实施例中，对于热数据存储磁盘可以是IOPS较高的磁盘，例如可以包括：SSD、转速较高的SAS盘、闪存等；冷数据存储磁盘可以包括：普通磁盘，例如可以包括：机械硬盘。In this embodiment, the disk for hot data storage can be a disk with higher IOPS, for example, it can include: SSD, SAS disk with higher rotation speed, flash memory, etc.; the cold data storage disk can include: ordinary disk, for example, it can include: mechanical hard disk .
本实施例中，第一层存储目录和第二层存储目录相当于元数据保存的两个数据结构，第一层存储目录和热数据存储磁盘绑定以及第二层存储目录和冷数据存储磁盘绑定后，存储到第一存储目录或第二存储目录下的数据，实际上，相当于将这些数据存储在热数据存储磁盘中或者冷数据存储磁盘中。In this embodiment, the first-level storage directory and the second-level storage directory are equivalent to two data structures for metadata storage, the first-level storage directory is bound to the hot data storage disk, and the second-level storage directory is bound to the cold data storage disk After binding, the data stored in the first storage directory or the second storage directory is actually equivalent to storing these data in a hot data storage disk or a cold data storage disk.
本实施例中，第一次存入到分布式文件系统中的数据可以为热度值较高的数据，这些数据需要存储到第一层存储目录下的热数据存储磁盘中。In this embodiment, the data stored in the distributed file system for the first time may be data with a high heat value, and these data need to be stored in the hot data storage disk under the first-tier storage directory.
S102：周期性的检测存储在所述热数据存储磁盘中的第一存储数据的访问次数；S102: Periodically detect the access times of the first stored data stored in the hot data storage disk;
本实施例中，第一存储数据可以表示热数据存储磁盘中的任何一个数据。In this embodiment, the first stored data may represent any data in the hot data storage disk.
本实施例中，可以周期性的检测一个周期时间段内第一存储数据的访问次数，也就是检测任何一个存储数据的访问次数。In this embodiment, the number of accesses to the first stored data within a periodic time period may be detected periodically, that is, the number of accesses to any stored data may be detected.
具体的，S102还可以包括：实时的检测单位时间内第一存储数据的访问次数，然后再周期性的统计一个周期时间段内的访问次数。Specifically, S102 may also include: detecting the number of accesses to the first stored data per unit time in real time, and then periodically counting the number of accesses within a period of time.
S103：当所述第一存储数据的访问次数小于预设的第一热度阈值时，将所述第一存储数据迁移到第二层存储目录下的冷数据存储磁盘中。S103: When the number of accesses of the first stored data is less than a preset first heat threshold, migrate the first stored data to a cold data storage disk under a second-tier storage directory.
本实施例中，第一热度阈值可以是技术人员根据经验设定的。当第一存储数据的访问次数较低或者为零的时候，将第一存储数据从热数据存储磁盘中迁移到第二层存储目录下的冷数据存储磁盘中。本实施例中，当第一存储数据的访问次数，大于等于预设的第一热度阈值时，不移动第一存储数据。In this embodiment, the first temperature threshold may be set by a technician based on experience. When the access times of the first stored data are low or zero, the first stored data is migrated from the hot data storage disk to the cold data storage disk under the second-tier storage directory. In this embodiment, when the number of accesses to the first stored data is greater than or equal to a preset first heat threshold, the first stored data is not moved.
本实施例中，当存储数据迁移到第二层存储目录下的冷数据存储磁盘中时，由于冷数据存储磁盘的存储容量有限，当冷数据存储磁盘中存储了较多的存储数据后，若不及时清理，会影响后续存储数据从热数据存储磁盘向冷数据存储磁盘的数据迁移，因此，还可以包括：以下步骤中的S301～S303：In this embodiment, when the storage data is migrated to the cold data storage disk under the second-tier storage directory, due to the limited storage capacity of the cold data storage disk, when more storage data is stored in the cold data storage disk, if If it is not cleaned up in time, it will affect the subsequent data migration from the hot data storage disk to the cold data storage disk. Therefore, it may also include: S301-S303 in the following steps:
S301：当所述冷数据存储磁盘中存储的数据所占的内存占所述冷数据存储磁盘存储容量的比例超过了预设的比例阈值时，确定所述冷数据存储磁盘中满足预设规则的第二存储数据；S301: When the ratio of the memory occupied by the data stored in the cold data storage disk to the storage capacity of the cold data storage disk exceeds a preset ratio threshold, determine that the cold data storage disk meets the preset rule second storage data;
S302：将所述第二存储数据在第三层存储目录下的备份磁盘中备份；S302: Back up the second storage data in a backup disk under the third-tier storage directory;
S303：删除所述冷数据存储磁盘中的第二存储数据。S303: Delete the second stored data in the cold data storage disk.
本实施例中，对于预设的规则可以包括多个，例如可以根据存储数据的访问次数也可以根据存储数据的存储时间确定，具体的，S301可以包括以下的两个方面：In this embodiment, there may be multiple preset rules, for example, it may be determined according to the number of accesses to the stored data or the storage time of the stored data. Specifically, S301 may include the following two aspects:
第一方面：检测存储在所述冷数据存储磁盘中的存储数据的访问次数；The first aspect: detecting the number of accesses of the storage data stored in the cold data storage disk;
将在预设的时间段内访问次数均为零的所述存储数据作为第二存储数据；using the stored data whose access times are all zero within a preset time period as the second stored data;
或者，or,
第二方面：判断存储在所述冷数据存储磁盘中的存储数据中是否包括超过了预设时间的存储数据；The second aspect: judging whether the stored data stored in the cold data storage disk includes stored data exceeding a preset time;
将超过了预设时间的存储数据作为第二存储数据。The stored data exceeding the preset time is used as the second stored data.
举例说明：for example:
针对于第一方面：假设冷数据存储磁盘中存储的数据占该冷数据存储磁盘存储容量的80％，检测存储在冷数据存储磁盘中存储数据的访问次数，若检测到在一段时间内(例如可以为：半个月或者一个月)内访问次数均为零，则将这些数据备份到第三层存储目录下的备份磁盘中，并在冷数据存储磁盘中删除这些数据。For the first aspect: assuming that the data stored in the cold data storage disk accounts for 80% of the storage capacity of the cold data storage disk, detect the number of accesses stored in the cold data storage disk, if it is detected within a period of time (for example It can be: the number of visits is zero within half a month or one month), then these data are backed up in the backup disk under the third-tier storage directory, and these data are deleted in the cold data storage disk.
针对于第二方面：假设冷数据存储磁盘中存储的数据占该冷数据存储磁盘存储容量的80％，可以将2个月前保存的存储数据，备份到第三层存储磁盘下的备份磁盘中，并在冷数据存储磁盘中删除这些数据。For the second aspect: Assuming that the data stored in the cold data storage disk accounts for 80% of the storage capacity of the cold data storage disk, the stored data saved 2 months ago can be backed up to the backup disk under the third-tier storage disk , and delete the data in the cold data storage disk.
本实施例中，对于第三层存储磁盘中的备份磁盘可以为磁带库等其它成本较低的存储磁盘。In this embodiment, the backup disks in the third-tier storage disks may be other low-cost storage disks such as tape libraries.
本实施例中，对于存储在第二层存储目录下的存储数据，还可以因为访问量的增加，从第二层目录中的冷数据存储磁盘中迁移到第一目录下的热数据存储磁盘中，具体的，还可以包括：In this embodiment, for the storage data stored in the second-level storage directory, due to the increase in access volume, it can also be migrated from the cold data storage disk in the second-level directory to the hot data storage disk in the first directory , specifically, may also include:
周期性的检测存储在冷数据存储磁盘中的第三存储数据的访问次数；Periodically detect the access times of the third storage data stored in the cold data storage disk;
当所述第三存储数据的访问次数大于预设的第二热度阈值时，将所述第三存储数据迁移到第一层存储目录下的热数据存储磁盘中。When the access times of the third stored data is greater than the preset second heat threshold, the third stored data is migrated to the hot data storage disk under the first layer storage directory.
本实施例中，第三存储数据可以是存储在冷数据存储磁盘中的任何一个数据。In this embodiment, the third stored data may be any data stored in the cold data storage disk.
本实施例中，当接收到待存储的数据时，将所述待存储的数据存储到第一层存储目录下的热数据存储磁盘中；周期性的检测存储在所述热数据存储磁盘中的第一存储数据的访问次数；当所述第一存储数据的访问次数小于预设的第一热度阈值时，将所述第一存储数据迁移到第二层存储目录下的冷数据存储磁盘中。通过本实施例的方法，将不同访问量的数据分层存储，不仅控制了磁盘的使用成本，而且简化了分布式文件系统数据存储的流程。In this embodiment, when the data to be stored is received, the data to be stored is stored in the hot data storage disk under the first layer storage directory; the data stored in the hot data storage disk is periodically detected The number of accesses of the first stored data; when the number of accesses of the first stored data is less than the preset first heat threshold, the first stored data is migrated to the cold data storage disk under the second-tier storage directory. Through the method of this embodiment, data with different access volumes are stored in layers, which not only controls the cost of using the disk, but also simplifies the process of data storage in the distributed file system.
参考图4，示出了本发明实施例提供的一种分布式文件系统分层存储逻辑的示意图；Referring to FIG. 4 , it shows a schematic diagram of a distributed file system hierarchical storage logic provided by an embodiment of the present invention;
如图所示：在分布式文件系统中建立hotdir目录和colddir目录，其中，将hotdir目录与SSD绑定；将colddir目录与普通硬盘绑定；并将磁带库与该分布式文件系统相连接。当客户端中产生待存储的数据时，先将该待存储的数据存储在hotdir目录下的SSD盘中。周期性的检测SSD盘存储的第一存储数据的访问次数，当第一存储数据的访问次数小于预设的第一热度阈值时，将所述第一存储数据迁移到colddir目录下的普通硬盘中。若是colddir目录下的普通硬盘中存储的数据的内存占普通硬盘存储容量的比例超过了预设的比例阈值时，可以将一段时间内访问量一直为零的数据在磁带库中备份，并删除普通硬盘中的一段时间内访问量一直为零的数据；或者，还可以将存储了较长一段时间的数据在磁带库中备份，并删除普通硬盘中这些存储了较长一段时间的数据。As shown in the figure: create a hotdir directory and a colddir directory in the distributed file system, where the hotdir directory is bound to the SSD; the colddir directory is bound to the ordinary hard disk; and the tape library is connected to the distributed file system. When the data to be stored is generated in the client, the data to be stored is first stored in the SSD disk under the hotdir directory. Periodically detect the access times of the first storage data stored on the SSD disk, and when the access times of the first storage data is less than the preset first heat threshold, migrate the first storage data to the ordinary hard disk under the colddir directory . If the ratio of the memory of data stored in the common hard disk under the colddir directory to the storage capacity of the common hard disk exceeds the preset ratio threshold, you can back up the data with zero access for a period of time in the tape library and delete the common hard disk. The data in the hard disk that has been accessed for a period of time has been zero; or, the data that has been stored for a long period of time can also be backed up in the tape library, and the data that has been stored for a long period of time in the ordinary hard disk can be deleted.
其中，对于偶尔产生的数据也可以直接存储在colddir目录下的普通硬盘中。Among them, the occasional data can also be directly stored in the ordinary hard disk under the colddir directory.
本实施例中，将不同访问量的数据分层存储，不仅控制了磁盘的使用成本，而且简化了分布式文件系统数据存储的流程。In this embodiment, data with different access volumes are stored in layers, which not only controls the cost of using the disk, but also simplifies the process of data storage in the distributed file system.
参考图5，示出了本发明实施例提供的一种分布式文件系统分层系统的结构示意图，在本实施例中，所述系统包括：Referring to FIG. 5 , it shows a schematic structural diagram of a distributed file system layered system provided by an embodiment of the present invention. In this embodiment, the system includes:
第一存储单元501，用于当接收到待存储的数据时，将所述待存储的数据存储到第一层存储目录下的热数据存储磁盘中；The first storage unit 501 is configured to store the data to be stored in a hot data storage disk under the first layer storage directory when receiving the data to be stored;
第一检测单元502，用于周期性的检测存储在所述热数据存储磁盘中的第一存储数据的访问次数；The first detection unit 502 is configured to periodically detect the number of accesses of the first storage data stored in the hot data storage disk;
第一迁移单元503，用于当所述第一存储数据的访问次数小于预设的第一热度阈值时，将所述第一存储数据迁移到第二层存储目录下的冷数据存储磁盘中。The first migration unit 503 is configured to migrate the first stored data to the cold data storage disk under the second-tier storage directory when the access times of the first stored data is less than a preset first heat threshold.
可选的，所述系统还可以包括：Optionally, the system may also include:
建立单元，用于建立第一层存储目录和第二层存储目录；An establishment unit is used to establish a first-level storage directory and a second-level storage directory;
第一绑定单元，用于将所述第一层存储目录绑定热数据存储磁盘；其中，所述热数据存储磁盘包括：SSD固态硬盘、SAS盘或者闪存；The first binding unit is configured to bind the first-tier storage directory to a hot data storage disk; wherein the hot data storage disk includes: SSD solid state disk, SAS disk or flash memory;
第二绑定单元，用于将所述第二层存储目录绑定冷数据存储磁盘；其中，述冷数据存储磁盘包括：普通硬盘。The second binding unit is configured to bind the second layer storage directory to a cold data storage disk; wherein the cold data storage disk includes: a common hard disk.
可选的，所述系统还可以包括：Optionally, the system may also include:
确定单元，用于当所述冷数据存储磁盘中存储的数据所占的内存占所述冷数据存储磁盘存储容量的比例超过了预设的比例阈值时，确定所述冷数据存储磁盘中满足预设规则的第二存储数据；A determination unit, configured to determine that the cold data storage disk meets the preset requirements when the ratio of the memory occupied by the data stored in the cold data storage disk to the storage capacity of the cold data storage disk exceeds a preset ratio threshold. Set the second storage data of the rule;
备份单元，用于将所述第二存储数据在第三层存储目录下的备份磁盘中备份；a backup unit, configured to back up the second storage data in a backup disk under the third-tier storage directory;
删除单元，用于删除所述冷数据存储磁盘中的第二存储数据。A deleting unit, configured to delete the second stored data in the cold data storage disk.
可选的，所述确定单元，包括：Optionally, the determining unit includes:
检测子单元，用于检测存储在所述冷数据存储磁盘中的存储数据的访问次数；a detection subunit, configured to detect the number of accesses of the storage data stored in the cold data storage disk;
第一确定子单元，用于将在预设的时间段内访问次数均为零的所述存储数据确定为第二存储数据；a first determining subunit, configured to determine the stored data whose access times are all zero within a preset time period as the second stored data;
或者，or,
判断子单元，用于判断存储在所述冷数据存储磁盘中的存储数据是否超过了预设时间的存储数据；A judging subunit, configured to judge whether the storage data stored in the cold data storage disk exceeds the storage data of a preset time;
第二确定子单元，用于将超过了预设时间的存储数据确定为第二存储数据。The second determining subunit is configured to determine the stored data exceeding the preset time as the second stored data.
可选的，所述系统还可以包括：Optionally, the system may also include:
第二检测单元，用于周期性的检测存储在冷数据存储磁盘中的第三存储数据的访问次数；The second detection unit is used to periodically detect the number of accesses of the third storage data stored in the cold data storage disk;
第二迁移单元，用于当所述第三存储数据的访问次数大于预设的第二热度阈值时，将所述第三存储数据迁移到第一层存储目录下的热数据存储磁盘中。The second migration unit is configured to migrate the third stored data to a hot data storage disk under the first-tier storage directory when the access times of the third stored data is greater than a preset second heat threshold.
本实施例中，当接收到待存储的数据时，将所述待存储的数据存储到第一层存储目录下的热数据存储磁盘中；周期性的检测存储在所述热数据存储磁盘中的第一存储数据的访问次数；当所述第一存储数据的访问次数小于预设的第一热度阈值时，将所述第一存储数据迁移到第二层存储目录下的冷数据存储磁盘中。通过本实施例的系统，将不同访问量的数据分层存储，不仅控制了磁盘的使用成本，而且简化了分布式文件系统数据存储的流程。In this embodiment, when the data to be stored is received, the data to be stored is stored in the hot data storage disk under the first layer storage directory; the data stored in the hot data storage disk is periodically detected The number of accesses of the first stored data; when the number of accesses of the first stored data is less than the preset first heat threshold, the first stored data is migrated to the cold data storage disk under the second-tier storage directory. Through the system of this embodiment, data with different access volumes are stored in layers, which not only controls the cost of using the disk, but also simplifies the process of data storage in the distributed file system.
需要说明的是，本说明书中的各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似的部分互相参见即可。It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. For the same and similar parts in each embodiment, refer to each other, that is, Can.
对所公开的实施例的上述说明，使本领域专业技术人员能够实现或使用本发明。对这些实施例的多种修改对本领域的专业技术人员来说将是显而易见的，本文中所定义的一般原理可以在不脱离本发明的精神或范围的情况下，在其它实施例中实现。因此，本发明将不会被限制于本文所示的这些实施例，而是要符合与本文所公开的原理和新颖特点相一致的最宽的范围。The above description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention will not be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710385437.5A CN107193500A (en) | 2017-05-26 | 2017-05-26 | A kind of distributed file system Bedding storage method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710385437.5A CN107193500A (en) | 2017-05-26 | 2017-05-26 | A kind of distributed file system Bedding storage method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107193500A true CN107193500A (en) | 2017-09-22 |
Family
ID=59875147
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710385437.5A Pending CN107193500A (en) | 2017-05-26 | 2017-05-26 | A kind of distributed file system Bedding storage method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107193500A (en) |
Cited By (32)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107103203A (en) * | 2017-05-22 | 2017-08-29 | 郑州云海信息技术有限公司 | A kind of storage AUTOMATIC ZONING system and method applied to PACS |
| CN107817950A (en) * | 2017-10-31 | 2018-03-20 | 新华三技术有限公司 | A kind of data processing method and device |
| CN108363813A (en) * | 2018-03-15 | 2018-08-03 | 北京小度信息科技有限公司 | Date storage method, device and system |
| CN108363553A (en) * | 2018-01-31 | 2018-08-03 | 北京兰云科技有限公司 | A kind of data processing method, apparatus and system |
| CN108388406A (en) * | 2018-01-10 | 2018-08-10 | 链家网（北京）科技有限公司 | Data processing method and device |
| CN108519862A (en) * | 2018-03-30 | 2018-09-11 | 百度在线网络技术（北京）有限公司 | Storage method, device, system and the storage medium of block catenary system |
| CN108829350A (en) * | 2018-05-31 | 2018-11-16 | 阿里巴巴集团控股有限公司 | Data migration method and device based on block chain |
| CN109144414A (en) * | 2018-07-27 | 2019-01-04 | 阿里巴巴集团控股有限公司 | The multistage storage method and device of block chain data |
| CN109274752A (en) * | 2018-10-10 | 2019-01-25 | 腾讯科技（深圳）有限公司 | The access method and device, electronic equipment, storage medium of block chain data |
| CN109284069A (en) * | 2018-08-23 | 2019-01-29 | 郑州云海信息技术有限公司 | A distributed storage system and method for storing backup data |
| CN109324759A (en) * | 2018-09-17 | 2019-02-12 | 山东浪潮云投信息科技有限公司 | The processing terminal of big data platform, the method read data and write data |
| CN109634520A (en) * | 2018-11-29 | 2019-04-16 | 南京航空航天大学 | A kind of storage system based on HDFS CD server |
| CN110045925A (en) * | 2019-03-05 | 2019-07-23 | 北京字节跳动网络技术有限公司 | Data processing method, device, electronic equipment and computer readable storage medium |
| CN110309229A (en) * | 2019-05-09 | 2019-10-08 | 北京极数云舟科技有限公司 | The data processing method and distributed system of distributed system |
| CN110308862A (en) * | 2018-03-20 | 2019-10-08 | 纬颖科技服务股份有限公司 | Management method and related storage system |
| CN110532234A (en) * | 2019-08-30 | 2019-12-03 | 杭州迪普科技股份有限公司 | A kind of storage method and device of vulnerability scanning data |
| CN110543279A (en) * | 2018-05-29 | 2019-12-06 | 杭州海康威视数字技术股份有限公司 | Data storage and processing method, device and system |
| CN110673785A (en) * | 2019-08-23 | 2020-01-10 | 上海科技发展有限公司 | Data storage method and system |
| CN110727406A (en) * | 2019-10-10 | 2020-01-24 | 深圳力维智联技术有限公司 | Data storage scheduling method and device |
| CN110781129A (en) * | 2019-09-12 | 2020-02-11 | 苏州浪潮智能科技有限公司 | Resource scheduling method, device and medium in FPGA heterogeneous accelerator card cluster |
| US10592491B2 (en) | 2015-05-04 | 2020-03-17 | Alibaba Group Holding Limited | Distributed data processing |
| CN110908608A (en) * | 2019-11-22 | 2020-03-24 | 苏州浪潮智能科技有限公司 | Storage space saving method and system |
| CN111125047A (en) * | 2019-12-06 | 2020-05-08 | 中盈优创资讯科技有限公司 | Cold and hot data directory identification method and device |
| CN111124304A (en) * | 2019-12-19 | 2020-05-08 | 北京浪潮数据技术有限公司 | Data migration method and device, electronic equipment and storage medium |
| CN111625203A (en) * | 2020-07-31 | 2020-09-04 | 苏州浪潮智能科技有限公司 | Method, system, device and medium for hierarchical storage |
| CN112306901A (en) * | 2020-11-16 | 2021-02-02 | 新华三大数据技术有限公司 | Disk refreshing method and device based on layered storage system, electronic equipment and medium |
| CN113377781A (en) * | 2021-07-12 | 2021-09-10 | 中国工商银行股份有限公司 | Data storage method and device, computer equipment and storage medium |
| CN113835630A (en) * | 2021-09-15 | 2021-12-24 | 联泰集群（北京）科技有限责任公司 | Data storage method, device, data server, storage medium and system |
| CN113918098A (en) * | 2021-10-28 | 2022-01-11 | 北京金山云网络技术有限公司 | Data processing method, device, equipment and medium |
| CN114726791A (en) * | 2022-03-18 | 2022-07-08 | 中国建设银行股份有限公司 | Method and device for determining flow control threshold, electronic equipment and computer storage medium |
| WO2022156649A1 (en) * | 2021-01-19 | 2022-07-28 | 华为技术有限公司 | Capacity adjustment method and related device |
| CN118982617A (en) * | 2024-07-23 | 2024-11-19 | 慧迪恺德(北京)医学科技有限公司 | A method and system for automatically converting two-dimensional slices into three-dimensional data for diagnosis and treatment |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104679661A (en) * | 2013-11-27 | 2015-06-03 | 阿里巴巴集团控股有限公司 | Hybrid memory control method and hybrid memory system |
| CN105892952A (en) * | 2016-04-22 | 2016-08-24 | 深圳市深信服电子科技有限公司 | Hyper-converged system and longitudinal extension method thereof |
-
2017
- 2017-05-26 CN CN201710385437.5A patent/CN107193500A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104679661A (en) * | 2013-11-27 | 2015-06-03 | 阿里巴巴集团控股有限公司 | Hybrid memory control method and hybrid memory system |
| CN105892952A (en) * | 2016-04-22 | 2016-08-24 | 深圳市深信服电子科技有限公司 | Hyper-converged system and longitudinal extension method thereof |
Cited By (48)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10592491B2 (en) | 2015-05-04 | 2020-03-17 | Alibaba Group Holding Limited | Distributed data processing |
| US10872070B2 (en) | 2015-05-04 | 2020-12-22 | Advanced New Technologies Co., Ltd. | Distributed data processing |
| CN107103203A (en) * | 2017-05-22 | 2017-08-29 | 郑州云海信息技术有限公司 | A kind of storage AUTOMATIC ZONING system and method applied to PACS |
| CN107817950A (en) * | 2017-10-31 | 2018-03-20 | 新华三技术有限公司 | A kind of data processing method and device |
| CN107817950B (en) * | 2017-10-31 | 2021-07-23 | 新华三技术有限公司 | Data processing method and device |
| CN108388406A (en) * | 2018-01-10 | 2018-08-10 | 链家网（北京）科技有限公司 | Data processing method and device |
| CN108363553A (en) * | 2018-01-31 | 2018-08-03 | 北京兰云科技有限公司 | A kind of data processing method, apparatus and system |
| CN108363813B (en) * | 2018-03-15 | 2020-06-02 | 北京星选科技有限公司 | Data storage method, device and system |
| CN108363813A (en) * | 2018-03-15 | 2018-08-03 | 北京小度信息科技有限公司 | Date storage method, device and system |
| CN110308862B (en) * | 2018-03-20 | 2023-05-05 | 纬颖科技服务股份有限公司 | Management method and related storage system |
| CN110308862A (en) * | 2018-03-20 | 2019-10-08 | 纬颖科技服务股份有限公司 | Management method and related storage system |
| CN108519862A (en) * | 2018-03-30 | 2018-09-11 | 百度在线网络技术（北京）有限公司 | Storage method, device, system and the storage medium of block catenary system |
| CN110543279A (en) * | 2018-05-29 | 2019-12-06 | 杭州海康威视数字技术股份有限公司 | Data storage and processing method, device and system |
| CN108829350A (en) * | 2018-05-31 | 2018-11-16 | 阿里巴巴集团控股有限公司 | Data migration method and device based on block chain |
| TWI710978B (en) * | 2018-05-31 | 2020-11-21 | 開曼群島商創新先進技術有限公司 | Data migration method and device based on blockchain |
| US10649953B2 (en) | 2018-05-31 | 2020-05-12 | Alibaba Group Holding Limited | Blockchain-based data migration method and apparatus |
| TWI727226B (en) * | 2018-07-27 | 2021-05-11 | 開曼群島商創新先進技術有限公司 | Multi-level storage method and device for blockchain data |
| WO2020023843A1 (en) * | 2018-07-27 | 2020-01-30 | Alibaba Group Holding Limited | Multi-level storage method and apparatus for blockchain data |
| CN109144414A (en) * | 2018-07-27 | 2019-01-04 | 阿里巴巴集团控股有限公司 | The multistage storage method and device of block chain data |
| CN111897498A (en) * | 2018-07-27 | 2020-11-06 | 创新先进技术有限公司 | Multi-level storage method and device for blockchain data |
| CN109144414B (en) * | 2018-07-27 | 2020-07-07 | 阿里巴巴集团控股有限公司 | Multi-level storage method and device for block chain data |
| CN109284069A (en) * | 2018-08-23 | 2019-01-29 | 郑州云海信息技术有限公司 | A distributed storage system and method for storing backup data |
| CN109324759A (en) * | 2018-09-17 | 2019-02-12 | 山东浪潮云投信息科技有限公司 | The processing terminal of big data platform, the method read data and write data |
| CN109274752A (en) * | 2018-10-10 | 2019-01-25 | 腾讯科技（深圳）有限公司 | The access method and device, electronic equipment, storage medium of block chain data |
| CN109634520A (en) * | 2018-11-29 | 2019-04-16 | 南京航空航天大学 | A kind of storage system based on HDFS CD server |
| CN109634520B (en) * | 2018-11-29 | 2021-12-07 | 南京航空航天大学 | Storage system based on HDFS optical disc library |
| CN110045925A (en) * | 2019-03-05 | 2019-07-23 | 北京字节跳动网络技术有限公司 | Data processing method, device, electronic equipment and computer readable storage medium |
| CN110309229A (en) * | 2019-05-09 | 2019-10-08 | 北京极数云舟科技有限公司 | The data processing method and distributed system of distributed system |
| CN110673785A (en) * | 2019-08-23 | 2020-01-10 | 上海科技发展有限公司 | Data storage method and system |
| CN110532234A (en) * | 2019-08-30 | 2019-12-03 | 杭州迪普科技股份有限公司 | A kind of storage method and device of vulnerability scanning data |
| CN110781129B (en) * | 2019-09-12 | 2022-02-22 | 苏州浪潮智能科技有限公司 | Resource scheduling method, device and medium in FPGA heterogeneous accelerator card cluster |
| CN110781129A (en) * | 2019-09-12 | 2020-02-11 | 苏州浪潮智能科技有限公司 | Resource scheduling method, device and medium in FPGA heterogeneous accelerator card cluster |
| CN110727406B (en) * | 2019-10-10 | 2024-05-17 | 深圳力维智联技术有限公司 | Data storage scheduling method and device |
| CN110727406A (en) * | 2019-10-10 | 2020-01-24 | 深圳力维智联技术有限公司 | Data storage scheduling method and device |
| CN110908608A (en) * | 2019-11-22 | 2020-03-24 | 苏州浪潮智能科技有限公司 | Storage space saving method and system |
| CN111125047A (en) * | 2019-12-06 | 2020-05-08 | 中盈优创资讯科技有限公司 | Cold and hot data directory identification method and device |
| CN111125047B (en) * | 2019-12-06 | 2024-03-12 | 中盈优创资讯科技有限公司 | Cold and hot data catalog identification method and device |
| CN111124304A (en) * | 2019-12-19 | 2020-05-08 | 北京浪潮数据技术有限公司 | Data migration method and device, electronic equipment and storage medium |
| CN111625203A (en) * | 2020-07-31 | 2020-09-04 | 苏州浪潮智能科技有限公司 | Method, system, device and medium for hierarchical storage |
| CN112306901B (en) * | 2020-11-16 | 2022-07-29 | 新华三大数据技术有限公司 | Disk refreshing method and device based on layered storage system, electronic equipment and medium |
| CN112306901A (en) * | 2020-11-16 | 2021-02-02 | 新华三大数据技术有限公司 | Disk refreshing method and device based on layered storage system, electronic equipment and medium |
| WO2022156649A1 (en) * | 2021-01-19 | 2022-07-28 | 华为技术有限公司 | Capacity adjustment method and related device |
| CN113377781A (en) * | 2021-07-12 | 2021-09-10 | 中国工商银行股份有限公司 | Data storage method and device, computer equipment and storage medium |
| CN113377781B (en) * | 2021-07-12 | 2024-12-10 | 中国工商银行股份有限公司 | Data storage method, device, computer equipment and storage medium |
| CN113835630A (en) * | 2021-09-15 | 2021-12-24 | 联泰集群（北京）科技有限责任公司 | Data storage method, device, data server, storage medium and system |
| CN113918098A (en) * | 2021-10-28 | 2022-01-11 | 北京金山云网络技术有限公司 | Data processing method, device, equipment and medium |
| CN114726791A (en) * | 2022-03-18 | 2022-07-08 | 中国建设银行股份有限公司 | Method and device for determining flow control threshold, electronic equipment and computer storage medium |
| CN118982617A (en) * | 2024-07-23 | 2024-11-19 | 慧迪恺德(北京)医学科技有限公司 | A method and system for automatically converting two-dimensional slices into three-dimensional data for diagnosis and treatment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107193500A (en) | A kind of distributed file system Bedding storage method and system | |
| US10887393B2 (en) | Data storage device with embedded software | |
| US10289547B2 (en) | Method and apparatus for a network connected storage system | |
| KR101915830B1 (en) | Pooled partition layout and representation | |
| CN105718217B (en) | A kind of method and device of simplify configuration storage pool data sign processing | |
| CN107615261B (en) | Method and apparatus for shared storage among multiple cloud environments | |
| CN104462240A (en) | Method and system for realizing hierarchical storage and management in cloud storage | |
| CN103440208B (en) | A kind of method that data store and device | |
| US10552056B2 (en) | Data storage system tiering accounting for limited write endurance | |
| US20180059949A1 (en) | Tier based data file management | |
| US9189494B2 (en) | Object file system | |
| TW201937361A (en) | Storage device | |
| CN102221981A (en) | Method and apparatus to manage tier information | |
| CN109804359A (en) | For the system and method by write back data to storage equipment | |
| CN107688443A (en) | A kind of method of data storage, system and relevant apparatus | |
| CN111741107A (en) | Hierarchical method and device based on file storage system, and electronic device | |
| CN111741106A (en) | File filing method and device based on file storage system, and electronic equipment | |
| US8775732B1 (en) | Method and system for passing information to disk arrays for efficient storage management | |
| CN105511811B (en) | A kind of method and system improving file system handling capacity | |
| CN109284270B (en) | Deployment optimization method and device for distributed file system storage module | |
| CN115878580A (en) | A log management method and device | |
| CN103797455B (en) | The method and apparatus of storage file | |
| Gupta et al. | Exploring Forensic Implications of the Fusion Drive | |
| WO2015066850A1 (en) | Method and device for storing file |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170922 |