[go: up one dir, main page]

CN111552438B - Method, device, server and storage medium for writing object - Google Patents

Method, device, server and storage medium for writing object Download PDF

Info

Publication number
CN111552438B
CN111552438B CN202010327053.XA CN202010327053A CN111552438B CN 111552438 B CN111552438 B CN 111552438B CN 202010327053 A CN202010327053 A CN 202010327053A CN 111552438 B CN111552438 B CN 111552438B
Authority
CN
China
Prior art keywords
directory
write
written
quota
specified
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010327053.XA
Other languages
Chinese (zh)
Other versions
CN111552438A (en
Inventor
李金阳
郭军
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bigo Technology Pte Ltd
Original Assignee
Guangzhou Baiguoyuan Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Baiguoyuan Information Technology Co Ltd filed Critical Guangzhou Baiguoyuan Information Technology Co Ltd
Priority to CN202010327053.XA priority Critical patent/CN111552438B/en
Publication of CN111552438A publication Critical patent/CN111552438A/en
Application granted granted Critical
Publication of CN111552438B publication Critical patent/CN111552438B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0602Interfaces specially adapted for storage systems specifically adapted to achieve a particular effect
    • G06F3/0604Improving or facilitating administration, e.g. storage management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/10File systems; File servers
    • G06F16/14Details of searching files based on file metadata
    • G06F16/148File search processing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0629Configuration or reconfiguration of storage systems
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/06Digital input from, or digital output to, record carriers, e.g. RAID, emulated record carriers or networked record carriers
    • G06F3/0601Interfaces specially adapted for storage systems
    • G06F3/0628Interfaces specially adapted for storage systems making use of a particular technique
    • G06F3/0638Organizing or formatting or addressing of data
    • G06F3/0643Management of files
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02DCLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Library & Information Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the invention discloses a method, a device, a server and a storage medium for writing an object. Wherein the method comprises the following steps: searching the associated directory of the write directory designated by the object to be written in at each upward directory level; and if the specified writing quantity of the object to be written is smaller than the remaining permission quota of the writing directory and smaller than the remaining permission quota of the associated directory, writing the object to be written into the writing directory. According to the technical scheme provided by the embodiment of the invention, the object to be written can uniformly meet the quota requirements of the appointed writing catalogue and each associated catalogue, the situation that the quota requirements of the writing catalogue are met and exceed the quota requirements of a certain associated catalogue under the upward catalogue level is prevented, the remaining permission quotas of the writing catalogue and the associated catalogue are limited, the excessive writing of different objects to a single catalogue can be effectively avoided, and the suitability of the writing of the objects under different catalogues is improved.

Description

一种对象写入的方法、装置、服务器和存储介质A method, device, server and storage medium for object writing

技术领域technical field

本发明实施例涉及数据处理技术领域,尤其涉及一种对象写入的方法、装置、服务器和存储介质。The embodiments of the present invention relate to the technical field of data processing, and in particular, to a method, device, server and storage medium for object writing.

背景技术Background technique

随着多媒体文件存储量的逐渐增加,Ceph作为Linux系统的文件系统备选之一,通常采用Ceph文件系统(Ceph File System,CephFS)来将客户端操作的各类系统文件存储至不同层级的系统目录下,此时多个客户端向CephFS下的系统目录中写入文件,可能会存在某一目录被过度使用而使其他目录空间浪费的问题,因此文件系统中出现目录配额的要求。With the gradual increase in the storage of multimedia files, Ceph is one of the file system options for the Linux system. Ceph File System (CephFS) is usually used to store various system files operated by the client to different levels of the system. In the directory, when multiple clients write files to the system directory under CephFS, there may be a problem that a certain directory is overused and the space of other directories is wasted, so there is a requirement for directory quotas in the file system.

目前,客户端会采用内核挂载和用户空间文件系统(File System in UserSpace,FUSE)挂载这两种方式来访问CephFS,此时内核挂载时客户端功能会集成到Linux内核里,无法控制CephFS在系统目录下的配额;而FUSE挂载时会直接在FUSE客户端内嵌入相应的配额检查逻辑来控制CephFS下各系统目录的配额,但需要FUSE客户端启动一个用户空间的进程,来提供一个文件访问接口来设置配额,此时系统目录的访问路径较长,导致配额性能较差。At present, the client will access CephFS by means of kernel mounting and user space file system (File System in UserSpace, FUSE) mounting. At this time, when the kernel is mounted, the client function will be integrated into the Linux kernel, which cannot be controlled. The quota of CephFS in the system directory; when FUSE is mounted, the corresponding quota checking logic will be directly embedded in the FUSE client to control the quota of each system directory under CephFS, but the FUSE client needs to start a user space process to provide A file access interface is used to set the quota. At this time, the access path of the system directory is long, resulting in poor quota performance.

发明内容Contents of the invention

本发明实施例提供了一种对象写入的方法、装置、服务器和存储介质,避免不同对象对单个目录的过度写入,提高不同目录下对象写入的适配性。Embodiments of the present invention provide a method, device, server and storage medium for object writing, which avoid excessive writing of different objects to a single directory, and improve the adaptability of object writing in different directories.

第一方面,本发明实施例提供了一种对象写入的方法,该方法包括:In a first aspect, an embodiment of the present invention provides a method for writing an object, the method comprising:

查找出待写入对象指定的写入目录在各上向目录级下的关联目录;Find out the associated directory under each upper directory level of the write directory specified by the object to be written;

如果所述待写入对象的指定写入量小于所述写入目录的剩余许可配额,且小于所述关联目录的剩余许可配额,则将所述待写入对象写入到所述写入目录下。If the specified write amount of the object to be written is less than the remaining quota of the write directory and is smaller than the remaining quota of the associated directory, write the object to be written into the write directory Down.

第二方面,本发明实施例提供了一种对象写入的装置,该装置包括:In the second aspect, an embodiment of the present invention provides an object writing device, which includes:

目录查找模块,用于查找出待写入对象指定的写入目录在各上向目录级下的关联目录;The directory search module is used to find out the associated directory under each directory level of the write directory specified by the object to be written;

对象写入模块,用于如果所述待写入对象的指定写入量小于所述写入目录的剩余许可配额,且小于所述关联目录的剩余许可配额,则将所述待写入对象写入到所述写入目录下。An object writing module, configured to write the object to be written if the specified write amount of the object to be written is less than the remaining allowance quota of the write directory and less than the remaining allowance quota of the associated directory. Enter the write directory.

第三方面,本发明实施例提供了一种服务器,该服务器包括:In a third aspect, an embodiment of the present invention provides a server, and the server includes:

一个或多个处理器;one or more processors;

存储装置,用于存储一个或多个程序;storage means for storing one or more programs;

当所述一个或多个程序被所述一个或多个处理器执行,使得所述一个或多个处理器实现本发明任意实施例所述的对象写入的方法。When the one or more programs are executed by the one or more processors, the one or more processors implement the object writing method described in any embodiment of the present invention.

第四方面,本发明实施例提供了一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时实现本发明任意实施例所述的对象写入的方法。In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the object writing method described in any embodiment of the present invention is implemented.

本发明实施例提供的一种对象写入的方法、装置、服务器和存储介质,首先查找出待写入对象当前指定的写入目录在各个上向目录级下的关联目录,只有在待写入对象的指定写入量既小于写入目录的剩余许可配额,且小于每一关联目录的剩余许可配额时,才将待写入对象写入到该写入目录下,使得待写入对象能够统一满足所指定的写入目录和各个关联目录的配额要求,防止出现满足写入目录的配额要求,而超出上向目录级下某个关联目录的配额要求的情况,同时限定写入目录和关联目录的剩余许可配额,能够有效避免不同对象对单个目录的过度写入,提高不同目录下对象写入的适配性。A method, device, server, and storage medium for writing an object provided by an embodiment of the present invention firstly find out the associated directories under each upper directory level of the writing directory currently specified by the object to be written. When the specified write volume of the object is less than the remaining quota of the write directory and the remaining quota of each associated directory, the object to be written is written to the write directory, so that the objects to be written can be unified Meet the quota requirements of the specified write directory and each associated directory, prevent the situation that the quota requirements of the write directory are met, but exceed the quota requirements of an associated directory under the upper directory level, and limit the write directory and associated directories at the same time The remaining license quota can effectively avoid excessive writing of different objects to a single directory, and improve the adaptability of writing objects in different directories.

附图说明Description of drawings

通过阅读参照以下附图所作的对非限制性实施例所作的详细描述,本发明的其它特征、目的和优点将会变得更明显:Other characteristics, objects and advantages of the present invention will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

图1A为本发明实施例一提供的一种对象写入的方法的流程图;FIG. 1A is a flowchart of a method for writing an object according to Embodiment 1 of the present invention;

图1B为本发明实施例一提供的对象写入过程的原理示意图;FIG. 1B is a schematic diagram of the principle of the object writing process provided by Embodiment 1 of the present invention;

图2A为本发明实施例二提供的一种对象写入的方法的流程图;FIG. 2A is a flowchart of a method for writing an object according to Embodiment 2 of the present invention;

图2B为本发明实施例二提供的方法中确定写入目录和各个关联目录的剩余许可配额的原理示意图;FIG. 2B is a schematic diagram of the principle of determining the remaining license quota of the write directory and each associated directory in the method provided by Embodiment 2 of the present invention;

图3A为本发明实施例三提供的一种对象写入的方法的流程图;FIG. 3A is a flowchart of a method for writing an object according to Embodiment 3 of the present invention;

图3B为本发明实施例三提供的对象写入过程的原理示意图;FIG. 3B is a schematic diagram of the principle of the object writing process provided by Embodiment 3 of the present invention;

图4为本发明实施例四提供的一种对象写入的装置的结构示意图;FIG. 4 is a schematic structural diagram of an object writing device provided in Embodiment 4 of the present invention;

图5为本发明实施例五提供的一种服务器的结构示意图。FIG. 5 is a schematic structural diagram of a server provided in Embodiment 5 of the present invention.

具体实施方式Detailed ways

下面结合附图和实施例对本发明作进一步的详细说明。可以理解的是,此处所描述的具体实施例仅仅用于解释本发明,而非对本发明的限定。另外还需要说明的是,为了便于描述,附图中仅示出了与本发明相关的部分而非全部结构。此外,在不冲突的情况下,本发明中的实施例及实施例中的特征可以相互组合。The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, only some structures related to the present invention are shown in the drawings but not all structures. In addition, the embodiments and the features in the embodiments of the present invention can be combined with each other under the condition of no conflict.

本发明实施例主要针对在现有的CephFS系统中,客户端执行对应的写入操作(如在CephFS系统中写入新的文件、创建新的目录或创建文件软硬链接等)时,由于不同挂载方式下对系统目录设置配额时存在相应配额缺陷,而导致在某一目录下过度写入的问题,直接通过CephFS系统中的元数据服务端(Metadata Server,MDS)来为CephFS系统下的相应目录设置对应的配额信息,无需对客户端做任何改动,进而只有在待写入对象的指定写入量小于当前指定的写入目录的剩余许可配额,且小于该写入目录在各上向目录级下的关联目录的剩余许可配额时,才将待写入对象写入到该写入目录下,使得待写入对象能够统一满足写入目录和各个关联目录的配额要求,有效避免不同对象对单个目录的过度写入,提高不同目录下对象写入的适配性。The embodiment of the present invention is mainly aimed at the existing CephFS system, when the client performs the corresponding write operation (such as writing a new file in the CephFS system, creating a new directory or creating a file soft and hard link, etc.), due to different There is a corresponding quota defect when setting quotas for the system directory in the mount mode, which leads to the problem of excessive writing in a certain directory. The corresponding quota information is set for the corresponding directory without any changes to the client, and only when the specified write volume of the object to be written is less than the remaining allowed quota of the currently specified write directory, and is less than the write directory in each direction When the remaining quota of the associated directory at the directory level is exceeded, the object to be written is written to the write directory, so that the object to be written can uniformly meet the quota requirements of the write directory and each associated directory, effectively avoiding different objects. Excessive writing to a single directory improves the adaptability of object writing in different directories.

实施例一Embodiment one

图1A为本发明实施例一提供的一种对象写入的方法的流程图,本实施例可适用于在各类文件系统的系统目录下写入任意对象的情况中。本实施例提供的对象写入的方法可以由本发明实施例提供的对象写入的装置来执行,该装置可以通过软件和/或硬件的方式来实现,并集成在执行本方法的服务器中,该服务器可以是文件系统内用于处理各类对象元数据的元数据服务器。FIG. 1A is a flow chart of a method for writing an object according to Embodiment 1 of the present invention. This embodiment is applicable to writing any object under the system directory of various file systems. The object writing method provided in this embodiment can be executed by the object writing device provided in the embodiment of the present invention, and the device can be realized by means of software and/or hardware, and integrated in the server executing the method. The server may be a metadata server for processing metadata of various objects in the file system.

具体的,参考图1A,该方法可以包括如下步骤:Specifically, referring to FIG. 1A, the method may include the following steps:

S110,查找出待写入对象指定的写入目录在各上向目录级下的关联目录。S110, find out the associated directory under each upper directory level of the write directory specified by the object to be written.

具体的,本实施例主要针对客户端在CephFS系统的各个系统目录下执行对应的写入操作,此时待写入对象为客户端在某一目录下指定写入的任一目标,例如新创建的目录、文件和文件软硬链接等;同时,在初始化设置CephFS系统时,会根据不同对象的写入需求预先设置相应的目录级,不同目录级下创建有对应的系统目录,例如CephFS系统下的整个对象存储区域为第一级目录,在第一级目录下创建出多个子目录,作为第二级目录,并在每一第二级目录下再次创建出多个子目录,作为第三级目录,以此类推,使得CephFS系统下的各个目录之间存在对应的上下级关系;此时,客户端可以指示在CephFS系统下的任一特定目录下写入相应对象,该特定目录作为本实施例中的写入目录。Specifically, this embodiment is mainly aimed at the client performing corresponding write operations in each system directory of the CephFS system. At this time, the object to be written is any target that the client specifies to write in a certain directory, such as a newly created At the same time, when initializing the CephFS system, the corresponding directory levels will be preset according to the writing requirements of different objects, and corresponding system directories will be created under different directory levels, such as CephFS system The entire object storage area is the first-level directory, create multiple subdirectories under the first-level directory as the second-level directory, and create multiple subdirectories under each second-level directory as the third-level directory , and so on, so that there is a corresponding upper-lower relationship between the various directories under the CephFS system; at this time, the client can indicate to write the corresponding object under any specific directory under the CephFS system, and this specific directory is used as the in the write directory.

可选的,客户端在CephFS系统的某一目录下执行对应的写入操作后,MDS服务端首先会分析出客户端本次操作指定的待写入对象,如新创建的目录、文件或者文件软硬链接等,如图1B所示,同时在CephFS系统的各个目录下查找出待写入对象当前指定的写入目录,进而从该写入目录所在的目录级开始,上向依次排查各个目录级,以在该写入目录的每一上向目录级下分别查找出包含有该写入目录的目标目录,作为本实施例中写入目录在各个上向目录级下的关联目录。Optionally, after the client performs a corresponding write operation in a certain directory of the CephFS system, the MDS server will first analyze the object to be written specified by the client in this operation, such as a newly created directory, file or file Soft and hard links, etc., as shown in Figure 1B, at the same time find out the currently designated writing directory of the object to be written in each directory of the CephFS system, and then start from the directory level where the writing directory is located, and check each directory upwards in sequence level, so as to search for the target directory containing the write directory under each upward directory level of the write directory, as the associated directory of the write directory under each upward directory level in this embodiment.

示例性的,若客户端当前指示在CephFS系统的第三级目录下的某一特定目录下写入一份新文件,此时该新文件为本实施例中的待写入对象,该特定目录为本实施例中的写入目录,在第一级目录和第二级目录下包含有该特定目录的目录为本实施例中的关联目录。Exemplarily, if the client currently indicates to write a new file under a specific directory under the third-level directory of the CephFS system, the new file is the object to be written in this embodiment, and the specific directory It is the writing directory in this embodiment, and the directory containing the specific directory under the first-level directory and the second-level directory is the associated directory in this embodiment.

S120,如果待写入对象的指定写入量小于写入目录的剩余许可配额,且小于关联目录的剩余许可配额,则将待写入对象写入到写入目录下。S120, if the specified writing amount of the object to be written is less than the remaining quota of the writing directory and is smaller than the remaining quota of the associated directory, write the object to be written into the writing directory.

可选的,为了避免客户端在CephFS系统的某个目录下的过度写入,本实施例会预先为不同目录级下的各个目录设定相应的配额,以要求该目录下写入的对象数据量不能超出该目录下设定的配额;此时,不同目录级下的各个目录根据该目录下的配额和已写入的对象数据量会匹配有相应的剩余许可配额,客户端在向某一目录写入相应对象时,首先需要分析当前写入的数据量是否满足对应目录下剩余许可配额的要求,以防止某一目录的过度写入。Optionally, in order to avoid excessive writing by the client in a certain directory of the CephFS system, this embodiment will pre-set corresponding quotas for each directory under different directory levels, so as to require the amount of object data written in this directory The quota set under this directory cannot be exceeded; at this time, each directory under different directory levels will match the corresponding remaining license quota according to the quota under the directory and the amount of object data written. When writing the corresponding object, it is first necessary to analyze whether the amount of data currently written meets the requirements of the remaining license quota under the corresponding directory, so as to prevent excessive writing of a certain directory.

具体的,为了提高不同对象在各个目录下写入的适配性,MDS服务端在查找出待写入对象指定的写入目录在各上向目录级下的关联目录后,首先会计算出待写入对象当前指定的写入目录和所查找出的各个关联目录在当前时刻下的剩余许可配额,此时由于待写入对象指定的写入目录属于各个关联目录下的一个子目录,因此如果后续将待写入对象写入到该写入目录时,会使该写入目录和各个关联目录下已写入的数据量均产生相应的增加量,因此在将待写入对象写入到当前指定的写入目录之前,为了避免某一目录的过度写入,首先分析本次操作的待写入对象的指定写入量,并依次判断该待写入对象的指定写入量是否小于写入目录和各个关联目录的剩余许可配额剩余,也就是判断如果后续将该待写入对象写入到写入目录后,该写入目录和各个关联目录中是否存在某个目录下已写入的数据量超出所设定的配额,使得该目录过度写入对象;如果待写入对象的指定写入量既小于写入目录的剩余许可配额,且小于各个关联目录的剩余许可配额,说明该写入目录和各个关联目录均存在剩余空间来写入该待写入对象,不存在造成某一目录下过度写入的问题,则直接将待写入对象写入到当前指定的写入目录下。Specifically, in order to improve the adaptability of different objects written in each directory, the MDS server will first calculate the associated directories under the directory level specified by the object to be written after finding out the The writing directory currently specified by the input object and the remaining quota of each associated directory found at the current moment. At this time, since the writing directory specified by the object to be written belongs to a subdirectory under each associated directory, if the subsequent When the object to be written is written to the write directory, the amount of written data in the write directory and each associated directory will increase accordingly. Therefore, when the object to be written is written to the currently specified Before writing to the directory, in order to avoid excessive writing of a certain directory, first analyze the specified write amount of the object to be written in this operation, and sequentially determine whether the specified write amount of the object to be written is less than the write directory and the remaining license quota of each associated directory, that is, to judge whether there is an amount of data written in a certain directory in the write directory and each associated directory if the object to be written is subsequently written to the write directory Exceeding the set quota makes the directory excessively write objects; if the specified write volume of the object to be written is less than the remaining license quota of the write directory and the remaining license quota of each associated directory, it means that the write directory There is remaining space in each associated directory to write the object to be written. If there is no problem of overwriting in a certain directory, the object to be written is directly written to the currently specified write directory.

此外,本实施例在依次判断该待写入对象的指定写入量是否小于写入目录和各个关联目录的剩余许可配额剩余时,还存在另一种情况:如果待写入对象的指定写入量大于等于写入目录的剩余许可配额,或者大于等于关联目录的剩余许可配额,则禁止在写入目录下写入待写入对象。In addition, when this embodiment sequentially judges whether the specified write amount of the object to be written is less than the remaining quota of the write directory and each associated directory, there is another situation: if the specified write amount of the object to be written If the amount is greater than or equal to the remaining quota of the write directory, or greater than or equal to the remaining quota of the associated directory, it is forbidden to write the object to be written in the write directory.

具体的,如果待写入对象的指定写入量大于等于写入目录的剩余许可配额,或者大于等于某一个关联目录的剩余许可配额,说明如果将待写入对象写入到该写入目录下后,会使写入目录或者某一关联目录下已写入的数据量超出所设定的配额,从而造成写入目录或者某一关联目录过度写入的问题,因此此时禁止在当前指定的写入目录下写入该待写入对象,有效避免不同对象对单个目录的过度写入。Specifically, if the specified write amount of the object to be written is greater than or equal to the remaining quota of the write directory, or greater than or equal to the remaining quota of an associated directory, it means that if the object to be written is written to the write directory After that, the amount of data written in the write directory or an associated directory will exceed the set quota, which will cause the problem of excessive writing in the write directory or an associated directory. Write the object to be written in the write directory, effectively avoiding excessive writing of different objects to a single directory.

本实施例提供的技术方案,首先查找出待写入对象当前指定的写入目录在各个上向目录级下的关联目录,只有在待写入对象的指定写入量既小于写入目录的剩余许可配额,且小于每一关联目录的剩余许可配额时,才将待写入对象写入到该写入目录下,使得待写入对象能够统一满足所指定的写入目录和各个关联目录的配额要求,防止出现满足写入目录的配额要求,而超出上向目录级下某个关联目录的配额要求的情况,同时限定写入目录和关联目录的剩余许可配额,能够有效避免不同对象对单个目录的过度写入,提高不同目录下对象写入的适配性。In the technical solution provided by this embodiment, firstly find out the associated directory under each upward directory level of the currently designated writing directory of the object to be written, and only when the specified writing amount of the object to be written is less than the remaining amount of the writing directory When the permission quota is less than the remaining permission quota of each associated directory, the object to be written is written to the write directory, so that the object to be written can uniformly meet the quota of the specified write directory and each associated directory Requirements to prevent the situation that the quota requirements of the write directory are met, but exceed the quota requirements of an associated directory under the upward directory level. At the same time, the remaining license quotas of the write directory and the associated directory are limited, which can effectively prevent different objects from using a single directory. Excessive writing, improve the adaptability of object writing in different directories.

实施例二Embodiment two

图2A为本发明实施例二提供的一种对象写入的方法的流程图,图2B为本发明实施例二提供的方法中确定写入目录和各个关联目录的剩余许可配额的原理示意图。本实施例是在上述实施例的基础上进行优化。具体的,如图2A所示,本实施例对于待写入对象当前指定的写入目录和各个关联目录下的剩余许可配额的具体确定过程进行详细的解释说明。FIG. 2A is a flow chart of a method for writing an object according to Embodiment 2 of the present invention, and FIG. 2B is a schematic diagram of the principle of determining the write-in directory and the remaining license quota of each associated directory in the method according to Embodiment 2 of the present invention. This embodiment is optimized on the basis of the foregoing embodiments. Specifically, as shown in FIG. 2A , this embodiment explains in detail the specific determination process of the currently designated writing directory of the object to be written and the remaining license quota under each associated directory.

可选的,如图2A所示,本实施例中可以包括如下步骤:Optionally, as shown in Figure 2A, this embodiment may include the following steps:

S210,根据目录配额需求预先为指定目录设置对应的配额上限,并实时记录每一目录的最新已写入总量。S210. Set a corresponding quota upper limit for the specified directory in advance according to the directory quota requirement, and record the latest written total amount of each directory in real time.

在本实施例中,MDS服务端在CephFS系统执行任意文件操作的过程中,可以实时监测当前时刻下管理员针对某一目录下的对象写入是否存在相应的目录配额需求,以便随时为指定目录设置对应的配额上限;例如,如果管理员在当前时刻下要求某几个指定目录下的已写入的数据量不能超出某个上限,否则会造成该目录过度写入的问题时,管理员在客户端会相应执行指定目录下的配额触发操作,MDS服务端在检测到该配额触发操作时,会通过解析该配额触发操作确定出当前的目录配额需求,该目录配额需求中携带了本次存在配额要求的指定目录,以及每一指定目录要求的配额上限,因此MDS服务端在确定出当前的目录配额需求中携带的指定目录和各个指定目录对应的配额上限后,分别为每一指定目录均设置对应的配额上限;同时,为了在后续对象写入过程中,准确判断相应目录的剩余许可配额,还会实时记录CephFS系统下每一目录的最新已写入总量,以便准确分析每一目录的剩余许可配额,保证不同对象在各个目录下写入的适配准确性。In this embodiment, when the CephFS system executes arbitrary file operations, the MDS server can monitor in real time whether there is a corresponding directory quota requirement for the administrator to write objects in a certain directory, so that the specified directory can be allocated at any time. Set the corresponding quota upper limit; for example, if the administrator requires that the amount of written data in certain specified directories cannot exceed a certain upper limit at the current moment, otherwise it will cause the problem of excessive writing of the directory, the administrator can The client will execute the quota trigger operation in the specified directory accordingly. When the MDS server detects the quota trigger operation, it will determine the current directory quota requirement by analyzing the quota trigger operation. The specified directory required by the quota, and the quota upper limit required by each specified directory. Therefore, after determining the specified directory carried in the current directory quota requirement and the quota limit corresponding to each specified directory, the MDS server sets the quota limit for each specified directory. Set the corresponding quota upper limit; at the same time, in order to accurately judge the remaining license quota of the corresponding directory during the subsequent object writing process, it will also record the latest written total amount of each directory under the CephFS system in real time, so as to accurately analyze each directory The remaining license quota to ensure the adaptation accuracy of different objects written in each directory.

S220,查找出待写入对象指定的写入目录在各上向目录级下的关联目录。S220, find out the associated directory under each upper directory level of the write directory specified by the object to be written.

S230,针对写入目录和关联目录中的每一目录,根据该目录的配额上限和最新已写入总量确定该目录的剩余许可配额。S230. For each directory in the write directory and associated directories, determine the remaining permitted quota of the directory according to the quota upper limit of the directory and the latest written total amount.

可选的,在查找出待写入对象指定的写入目录在各上向目录级下的关联目录后,由于待写入对象指定的写入目录属于各个关联目录下的一个子目录,因此如果后续将待写入对象写入到该写入目录时,会使该写入目录和各个关联目录下已写入的数据量均产生相应的增加量,因此在将待写入对象写入到当前指定的写入目录之前,为了避免某一目录的过度写入,需要分别分析本次操作的待写入对象的指定写入量,与写入目录和各个关联目录中的每一目录的剩余许可配额之间的大小;此时,如图2B所示,本实施例针对待写入对象指定的写入目录和该写入目录在各个上向目录级下的关联目录中的每一目录,首先查找出预先为该目录所设置的配额上限以及该目录下已写入对象占用的最新已写入总量,进而分析该目录下在最新已写入总量的基础上,距离达到该目录的配额上限还剩余的允许对象继续写入的量,作为本实施例中该目录的剩余许可配额,后续通过分许待写入对象的指定写入量与该目录的剩余许可配额之间的大小,判断该目录是否能够继续写入该待写入对象,避免该目录下的过度写入。Optionally, after finding out the associated directory under each upward directory level of the write directory specified by the object to be written, since the write directory specified by the object to be written belongs to a subdirectory under each associated directory, if When the object to be written is subsequently written to the write directory, the amount of written data in the write directory and each associated directory will increase accordingly. Therefore, when the object to be written is written to the current Before the specified write directory, in order to avoid excessive writing of a certain directory, it is necessary to analyze the specified write amount of the object to be written in this operation, and the remaining permissions of each directory in the write directory and each associated directory The size between the quotas; at this time, as shown in Figure 2B, the present embodiment is directed at the write directory specified by the object to be written and each directory in the associated directory of the write directory under each upward directory level, first Find out the upper limit of the quota set in advance for the directory and the latest written total amount occupied by the written objects in the directory, and then analyze the distance to reach the quota of the directory on the basis of the latest written total amount in the directory The amount of the allowed object to continue to be written in the upper limit is used as the remaining allowed quota of the directory in this embodiment, and subsequently judged by dividing the specified writing amount of the object to be written and the remaining allowed quota of the directory Whether the directory can continue to write the object to be written to avoid excessive writing in the directory.

示例性的,由于在根据目录配额需求预先为指定目录设置对应的配额上限时,可能只单独设定某几个指定目录的配额上限,而对其他目录的配额不作要求,此时本实施例待写入对象当前指定的写入目录和各个关联目录的配额上限可能为空,因此在本实施例中,根据该目录的配额上限和最新已写入总量确定该目录的剩余许可配额,具体可以包括:如果该目录的配额上限为空,则确定该目录的剩余许可配额无上限;如果该目录的配额上限为非空,则将该目录的配额上限和最新已写入总量的差值作为该目录的剩余许可配额。Exemplarily, when the corresponding quota upper limit is set for the specified directory in advance according to the directory quota requirement, the quota upper limit of some specified directories may be set separately, and the quota of other directories is not required. The quota upper limit of the writing directory currently specified by the writing object and each associated directory may be empty, so in this embodiment, the remaining quota of the directory is determined according to the quota upper limit of the directory and the latest total amount of written data. Specifically, it can be Including: if the quota upper limit of the directory is empty, determine that the remaining licensed quota of the directory has no upper limit; if the quota upper limit of the directory is not empty, then use the difference between the upper quota limit of the directory and the latest total written amount as The remaining license quota for this directory.

具体的,针对待写入对象当前指定的写入目录和该写入目录在各个上向目录级下的关联目录中的每一目录,如果该目录的配额上限为空,说明当前对该目录的对象写入量不作要求,因此可以直接确定该目录的剩余许可配额无上限,也就是可以随意写入任意量的对象;而如果该目录的配额上限为非空,说明当前对该目录的对象写入量存在一定的限制要求,此时分析该目录的最新已写入总量,进而将该目录的配额上限和最新已写入总量的差值作为该目录的剩余许可配额,避免该目录所写入的对象数据量超出该目录的配额上限要求;也就是后续在判断待写入对象的指定写入量是否小于写入目录和各个关联目录的剩余许可配额时,直接跳过写入目录和各个关联目录中配额上限为空的目录,仅依次判断待写入对象的指定写入量是否小于写入目录和各个关联目录中配额上限为非空的各个目录下的剩余许可配额。Specifically, for each directory in the currently specified writing directory of the object to be written and the associated directory under each upward directory level, if the quota upper limit of the directory is empty, it means that the current quota for the directory is There is no requirement for the number of objects written, so it can be directly determined that the remaining quota of the directory has no upper limit, that is, any number of objects can be written at will; and if the upper limit of the quota of the directory is not empty, it means that the current object writes to the directory There are certain restrictions on the amount of input. At this time, analyze the latest total amount written in the directory, and then use the difference between the quota upper limit of the directory and the latest total amount written in the directory as the remaining license quota of the directory to avoid the directory. The amount of object data written exceeds the upper quota requirement of the directory; that is, when determining whether the specified write amount of the object to be written is less than the remaining quota of the write directory and each associated directory, the write directory and the associated directory are directly skipped. For the directory whose quota upper limit is empty in each associated directory, it is only judged in turn whether the specified writing amount of the object to be written is less than the remaining permitted quota under the writing directory and each directory whose quota upper limit is not empty in each associated directory.

S240,如果待写入对象的指定写入量小于写入目录的剩余许可配额,且小于关联目录的剩余许可配额,则将待写入对象写入到写入目录下。S240, if the specified writing amount of the object to be written is less than the remaining quota of the writing directory and is smaller than the remaining quota of the associated directory, write the object to be written into the writing directory.

S250,更新写入目录和关联目录的最新已写入总量。S250, updating the latest written total amount of the written directory and the associated directory.

可选的,在待写入对象的指定写入量既小于写入目录的剩余许可配额,且小于各个关联目录的剩余许可配额,从而将待写入对象写入到写入目录下后,该写入目录和该写入目录在各上向目录级下的关联目录的最新已写入总量会相应增加,因此需要更新该写入目录和所查找出的各个关联目录的最新已写入总量,如在该写入目录和所查找出的各个关联目录下已有的已写入总量的基础上分别加上待写入对象的指定写入量,作为对应目录的最新已写入总量。Optionally, after the specified write amount of the object to be written is less than the remaining quota of the write directory and the remaining quota of each associated directory, so that the object to be written is written into the write directory, the The latest written totals of the written directory and the associated directories under each upward directory level will increase accordingly, so it is necessary to update the latest written totals of the written directory and each associated directory found. amount, such as adding the specified write amount of the object to be written on the basis of the written total amount under the written directory and each associated directory found, respectively, as the latest written total amount of the corresponding directory. quantity.

S260,如果待写入对象的指定写入量大于等于写入目录的剩余许可配额,或者大于等于关联目录的剩余许可配额,则禁止在写入目录下写入待写入对象。S260. If the specified writing amount of the object to be written is greater than or equal to the remaining quota of the writing directory, or is greater than or equal to the remaining quota of the associated directory, prohibit writing the object to be written in the writing directory.

本实施例提供的技术方案,通过预先为指定目录设置对应的配额上限,来限定待写入对象指定的写入目录和该写入目录在各个上向目录级下的关联目录的剩余许可配额,能够有效避免不同对象对单个目录的过度写入,同时在待写入对象的指定写入量既小于写入目录的剩余许可配额,且小于每一关联目录的剩余许可配额时,才将待写入对象写入到该写入目录下,使得待写入对象能够统一满足所指定的写入目录和各个关联目录的配额要求,防止出现满足写入目录的配额要求,而超出上向目录级下某个关联目录的配额要求的情况,提高不同目录下对象写入的适配性,同时在将待写入对象写入到该写入目录下后,及时更新写入目录和关联目录的最新已写入总量,以保证写入目录和关联目录的剩余许可配额的准确性,提高对象写入的适配准确度。In the technical solution provided by this embodiment, by setting the corresponding quota upper limit for the specified directory in advance, the remaining quota of the write directory specified by the object to be written and the associated directory under each upward directory level is limited, It can effectively avoid excessive writing of different objects to a single directory, and at the same time, only when the specified writing volume of the object to be written is less than the remaining quota of the writing directory and the remaining quota of each associated directory, the The input objects are written to the write directory, so that the objects to be written can uniformly meet the quota requirements of the specified write directory and each associated directory, preventing the occurrence of meeting the quota requirements of the write directory and exceeding the quota requirements of the upper directory level. The quota requirements of an associated directory improve the adaptability of writing objects in different directories. At the same time, after the object to be written is written to the writing directory, the latest updated information of the writing directory and the associated directory is updated in time. The total amount of writing, to ensure the accuracy of the remaining license quota of the writing directory and associated directories, and improve the adaptation accuracy of object writing.

实施例三Embodiment Three

图3A为本发明实施例三提供的一种对象写入的方法的流程图,图3B为本发明实施例三提供的对象写入过程的原理示意图。本实施例是在上述实施例的基础上进行优化。具体的,如图3B所示,本实施例对于对象写入过程中存在的其他处理过程进行详细的解释说明。FIG. 3A is a flow chart of an object writing method provided by Embodiment 3 of the present invention, and FIG. 3B is a schematic diagram of the principle of an object writing process provided by Embodiment 3 of the present invention. This embodiment is optimized on the basis of the foregoing embodiments. Specifically, as shown in FIG. 3B , this embodiment explains in detail other processing procedures existing in the object writing process.

可选的,如图3A所示,本实施例中可以包括如下步骤:Optionally, as shown in Figure 3A, this embodiment may include the following steps:

S310,根据不同目录之间的级联状态,生成对应的目录树。S310. Generate a corresponding directory tree according to the cascading state between different directories.

可选的,在初始化设置CephFS系统时,会根据不同对象的写入需求预先设置相应的目录级,不同目录级下创建有对应的系统目录,此时如图3B所示,为了确保各个目录所在目录级之间的关联性,MDS服务端可以通过分析CephFS系统内各个目录之间的级联状态,确定出整体范围下的第一级目录,以及在该第一级目录下所创建出的多个子目录,作为第二级目录,以此类推,根据不同目录之间的级联状态,生成对应的目录树,后续可以直接根据该目录树中各个目录之间的级联情况直观地查找出待写入对象当前指定的写入目录在各个上向目录级下的关联目录,保证关联目录的全面性。Optionally, when initializing the CephFS system, corresponding directory levels will be preset according to the writing requirements of different objects, and corresponding system directories will be created under different directory levels, as shown in Figure 3B. The correlation between the directory levels, the MDS server can determine the first-level directory in the overall range and the multiple created under the first-level directory by analyzing the cascading status of the various directories in the CephFS system. subdirectory, as the second-level directory, and so on, according to the cascading state between different directories, generate the corresponding directory tree, and then you can directly find out the waiting list directly according to the cascading situation between the various directories in the directory tree. The currently specified write directory of the write object is associated with each directory level under each upward directory, ensuring the comprehensiveness of the associated directory.

S320,接收待写入对象的写入命令。S320. Receive a write command of an object to be written.

具体的,客户端在CephFS系统的某一目录下执行对应的写入操作,会生成对应的写入命令,该写入命令中携带有待写入对象指定的写入目录;MDS服务端在接收到该写入命令后,首先会分析该写入命令指定的待写入对象以及该待写入对象当前指定的写入目录,以便执行相应的对象写入操作。Specifically, the client executes the corresponding write operation in a certain directory of the CephFS system, and generates a corresponding write command, which carries the write directory specified by the object to be written; the MDS server receives the After the write command, the object to be written specified by the write command and the write directory currently specified by the object to be written will be analyzed first, so as to perform the corresponding object write operation.

S330,根据目录树查找出待写入对象指定的写入目录在各上向目录级下的关联目录。S330, find out, according to the directory tree, associated directories under each upper directory level of the writing directory specified by the object to be written.

S340,针对写入目录和关联目录中的每一目录,根据待写入对象的指定写入量中的指定写入对象数量和该目录的剩余许可配额中的剩余许可写入对象数量的第一比对结果,以及待写入对象的指定写入量中的指定写入内存量和该目录的剩余许可配额中的剩余许可写入内存量的第二比对结果,判断待写入对象的指定写入量是否小于该目录的剩余许可配额。S340. For each directory in the write directory and the associated directory, according to the number of specified write objects in the specified write amount of the objects to be written and the remaining permitted write object quantity in the remaining permitted quota of the directory, the first The comparison result, and the second comparison result of the specified write memory amount in the specified write amount of the object to be written and the remaining allowable write memory amount in the remaining license quota of the directory, determine the designation of the object to be written Whether the write volume is less than the remaining licensed quota for the directory.

可选的,在根据目录配额需求预先为指定目录设置对应的配额上限时,可以从对象数量和占用内存量两方面分别设置指定目录的配额,此时在写入目录写入待写入对象之前,也需要从对象数量和占用内存量两方面来判断待写入对象是否满足写入目录和各个关联目录下的配额要求,因此在根据目录树查找出待写入对象指定的写入目录在各上向目录级下的关联目录之后,如图3B所示,针对写入目录和各个关联目录中的每一目录,首先根据该目录下设置的配额上限中包含的对象数量上限和已写入对象总数量,计算该目录的剩余许可配额中的剩余许可写入对象数量,同时根据该目录下设置的配额上限中包含的写入内存量上限和已写入内存总量,计算该目录的剩余许可配额中的剩余许可写入内存量,进而确定出待写入对象的指定写入量中的指定写入对象数量和指定写入内存量,并分别确定待写入对象的指定写入对象数量和该目录的剩余许可写入对象数量的第一比对结果,以及待写入对象的指定写入内存量和该目录的剩余许可写入内存量的第二比对结果,只有在待写入对象的指定写入对象数量小于该目录的剩余许可写入对象数量,且待写入对象的指定写入内存量小于该目录的剩余许可写入内存量时,才可以确定待写入对象的指定写入量小于该目录的剩余许可配额,从而实现不同目录在多维度下的配额要求。Optionally, when setting the quota upper limit for the specified directory in advance according to the quota requirements of the directory, you can set the quota of the specified directory in terms of the number of objects and the amount of memory occupied. , it is also necessary to judge whether the object to be written meets the quota requirements under the write directory and each associated directory from the two aspects of the number of objects and the amount of memory occupied. Therefore, the write directory specified by the object to be written is found in each After going up to the associated directory under the directory level, as shown in Figure 3B, for each directory in the write directory and each associated directory, firstly, according to the upper limit of the number of objects contained in the quota upper limit set under the directory and the number of objects written Total quantity, calculate the number of remaining permitted write objects in the remaining permission quota of the directory, and calculate the remaining permission of the directory according to the upper limit of the amount of written memory included in the upper limit of the quota set under the directory and the total amount of written memory The amount of remaining permitted write memory in the quota, and then determine the number of specified write objects and the specified write memory amount in the specified write amount of objects to be written, and determine the specified number of write objects and the specified write amount of objects to be written respectively The first comparison result of the number of objects that are allowed to be written in the directory, and the second comparison result of the specified amount of memory to be written in the object and the amount of memory that is allowed to be written in the directory are the second comparison results. Only when the number of specified write objects is less than the number of remaining permitted write objects of the directory, and the designated write memory of the object to be written is less than the remaining permitted write memory of the directory, can the designated write of the object to be written be determined. The input amount is less than the remaining license quota of the directory, so as to realize the quota requirements of different directories in multiple dimensions.

S350,如果待写入对象的指定写入量小于写入目录的剩余许可配额,且小于关联目录的剩余许可配额,则将待写入对象写入到写入目录下。S350. If the specified writing amount of the object to be written is less than the remaining quota of the writing directory and is smaller than the remaining quota of the associated directory, write the object to be written into the writing directory.

S360,如果待写入对象的指定写入量大于等于写入目录的剩余许可配额,或者大于等于关联目录的剩余许可配额,则禁止在写入目录下写入待写入对象。S360. If the specified writing amount of the object to be written is greater than or equal to the remaining quota of the writing directory, or is greater than or equal to the remaining quota of the associated directory, prohibit writing the object to be written in the writing directory.

本实施例提供的技术方案,从多方面设置不同目录的配额上限,实现不同目录在多维度下的配额要求,在查找出待写入对象当前指定的写入目录在各个上向目录级下的关联目录后,只有在待写入对象的指定写入量既小于写入目录的剩余许可配额,且小于每一关联目录的剩余许可配额时,才将待写入对象写入到该写入目录下,使得待写入对象能够统一满足所指定的写入目录和各个关联目录的配额要求,防止出现满足写入目录的配额要求,而超出上向目录级下某个关联目录的配额要求的情况,同时限定写入目录和关联目录的剩余许可配额,能够有效避免不同对象对单个目录的过度写入,提高不同目录下对象写入的适配性。The technical solution provided by this embodiment sets the quota upper limit of different directories from multiple aspects, realizes the quota requirements of different directories in multiple dimensions, and finds out the current specified writing directory of the object to be written at each upper directory level. After the directory is associated, the object to be written will be written to the write directory only when the specified write amount of the object to be written is less than the remaining license quota of the write directory and the remaining license quota of each associated directory Next, the objects to be written can uniformly meet the quota requirements of the specified write directory and each associated directory, and prevent the situation that the quota requirements of the write directory are met, but the quota requirements of an associated directory under the upward directory level are exceeded , and at the same time limit the remaining permission quota of the write directory and the associated directory, which can effectively avoid excessive writing of different objects to a single directory, and improve the adaptability of object writing in different directories.

实施例四Embodiment Four

图4为本发明实施例四提供的一种对象写入的装置的结构示意图,具体的,如图4所示,该装置可以包括:FIG. 4 is a schematic structural diagram of an object writing device provided in Embodiment 4 of the present invention. Specifically, as shown in FIG. 4, the device may include:

目录查找模块410,用于查找出待写入对象指定的写入目录在各上向目录级下的关联目录;The directory search module 410 is used to find out the associated directory under each upper directory level of the write directory specified by the object to be written;

对象写入模块420,用于如果待写入对象的指定写入量小于写入目录的剩余许可配额,且小于关联目录的剩余许可配额,则将待写入对象写入到写入目录下。The object writing module 420 is configured to write the object to be written into the write directory if the specified writing amount of the object to be written is less than the remaining quota of the write directory and the remaining quota of the associated directory.

本实施例提供的技术方案,首先查找出待写入对象当前指定的写入目录在各个上向目录级下的关联目录,只有在待写入对象的指定写入量既小于写入目录的剩余许可配额,且小于每一关联目录的剩余许可配额时,才将待写入对象写入到该写入目录下,使得待写入对象能够统一满足所指定的写入目录和各个关联目录的配额要求,防止出现满足写入目录的配额要求,而超出上向目录级下某个关联目录的配额要求的情况,同时限定写入目录和关联目录的剩余许可配额,能够有效避免不同对象对单个目录的过度写入,提高不同目录下对象写入的适配性。In the technical solution provided by this embodiment, firstly find out the associated directory under each upward directory level of the currently designated writing directory of the object to be written, and only when the specified writing amount of the object to be written is less than the remaining amount of the writing directory When the permission quota is less than the remaining permission quota of each associated directory, the object to be written is written to the write directory, so that the object to be written can uniformly meet the quota of the specified write directory and each associated directory Requirements to prevent the situation that the quota requirements of the write directory are met, but exceed the quota requirements of an associated directory under the upward directory level. At the same time, the remaining license quotas of the write directory and the associated directory are limited, which can effectively prevent different objects from using a single directory. Excessive writing, improve the adaptability of object writing in different directories.

本实施例提供的对象写入的装置可适用于上述任意实施例提供的对象写入的方法,具备相应的功能和有益效果。The device for writing an object provided in this embodiment can be applied to the method for writing an object provided in any of the above embodiments, and has corresponding functions and beneficial effects.

实施例五Embodiment five

图5为本发明实施例五提供的一种服务器的结构示意图,如图5所示,该服务器包括处理器50、存储装置51和通信装置52;服务器中处理器50的数量可以是一个或多个,图5中以一个处理器50为例;服务器中的处理器50、存储装置51和通信装置52可以通过总线或其他方式连接,图5中以通过总线连接为例。FIG. 5 is a schematic structural diagram of a server provided in Embodiment 5 of the present invention. As shown in FIG. 5 , the server includes a processor 50, a storage device 51 and a communication device 52; the number of processors 50 in the server can be one or more One, one processor 50 is taken as an example in FIG. 5; the processor 50, storage device 51 and communication device 52 in the server can be connected through a bus or in other ways, and the connection through a bus is taken as an example in FIG. 5.

存储装置51作为一种计算机可读存储介质,可用于存储软件程序、计算机可执行程序以及模块,如本发明任意实施例所述的对象写入的方法对应的程序指令/模块。处理器50通过运行存储在存储装置51中的软件程序、指令以及模块,从而执行服务器的各种功能应用以及数据处理,即实现上述对象写入的方法。As a computer-readable storage medium, the storage device 51 can be used to store software programs, computer-executable programs and modules, such as program instructions/modules corresponding to the object writing method described in any embodiment of the present invention. The processor 50 executes various functional applications and data processing of the server by running the software programs, instructions and modules stored in the storage device 51 , that is, realizes the above-mentioned object writing method.

存储装置51可主要包括存储程序区和存储数据区,其中,存储程序区可存储操作系统、至少一个功能所需的应用程序;存储数据区可存储根据终端的使用所创建的数据等。此外,存储装置51可以包括高速随机存取存储器,还可以包括非易失性存储器,例如至少一个磁盘存储器件、闪存器件、或其他非易失性固态存储器件。在一些实例中,存储装置51可进一步包括相对于处理器50远程设置的存储器,这些远程存储器可以通过网络连接至服务器。上述网络的实例包括但不限于互联网、企业内部网、局域网、移动通信网及其组合。The storage device 51 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required by a function; the data storage area may store data created according to the use of the terminal, and the like. In addition, the storage device 51 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage devices. In some examples, the storage device 51 may further include memories that are remotely located relative to the processor 50, and these remote memories may be connected to a server through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

通信装置52可用于实现客户端与服务器之间的网络连接或者移动数据连接。The communication device 52 can be used to implement a network connection or a mobile data connection between the client and the server.

本实施例提供的一种服务器可用于执行上述任意实施例提供的对象写入的方法,具备相应的功能和有益效果。A server provided in this embodiment can be used to execute the object writing method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

实施例六Embodiment six

本发明实施例六还提供了一种计算机可读存储介质,其上存储有计算机程序,该程序被处理器执行时可实现上述任意实施例中的对象写入的方法。该方法具体可以包括:Embodiment 6 of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the object writing method in any of the above embodiments can be implemented. The method may specifically include:

查找出待写入对象指定的写入目录在各上向目录级下的关联目录;Find out the associated directory under each upper directory level of the write directory specified by the object to be written;

如果待写入对象的指定写入量小于写入目录的剩余许可配额,且小于关联目录的剩余许可配额,则将待写入对象写入到写入目录下。If the specified write amount of the object to be written is less than the remaining quota of the write directory and the remaining quota of the associated directory, the object to be written is written to the write directory.

当然,本发明实施例所提供的一种包含计算机可执行指令的存储介质,其计算机可执行指令不限于如上所述的方法操作,还可以执行本发明任意实施例所提供的对象写入的方法中的相关操作。Certainly, a storage medium containing computer-executable instructions provided by an embodiment of the present invention, the computer-executable instructions are not limited to the method operations described above, and may also execute the object writing method provided by any embodiment of the present invention Related operations in .

通过以上关于实施方式的描述,所属领域的技术人员可以清楚地了解到,本发明可借助软件及必需的通用硬件来实现,当然也可以通过硬件实现,但很多情况下前者是更佳的实施方式。基于这样的理解,本发明的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品可以存储在计算机可读存储介质中,如计算机的软盘、只读存储器(Read-Only Memory,ROM)、随机存取存储器(RandomAccess Memory,RAM)、闪存(FLASH)、硬盘或光盘等,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行本发明各个实施例所述的方法。Through the above description about the implementation mode, those skilled in the art can clearly understand that the present invention can be realized by means of software and necessary general-purpose hardware, and of course it can also be realized by hardware, but in many cases the former is a better implementation mode . Based on this understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a floppy disk of a computer , read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), flash memory (FLASH), hard disk or optical disc, etc., including several instructions to make a computer device (which can be a personal computer, A server, or a network device, etc.) executes the methods described in various embodiments of the present invention.

值得注意的是,上述对象写入的装置的实施例中,所包括的各个单元和模块只是按照功能逻辑进行划分的,但并不局限于上述的划分,只要能够实现相应的功能即可;另外,各功能单元的具体名称也只是为了便于相互区分,并不用于限制本发明的保护范围。It is worth noting that, in the embodiment of the above-mentioned object writing device, the included units and modules are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition , the specific names of each functional unit are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the present invention.

以上所述仅为本发明的优选实施例,并不用于限制本发明,对于本领域技术人员而言,本发明可以有各种改动和变化。凡在本发明的精神和原理之内所作的任何修改、等同替换、改进等,均应包含在本发明的保护范围之内。The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims (10)

1.一种对象写入的方法,其特征在于,包括:1. A method for object writing, characterized in that, comprising: 查找出待写入对象指定的写入目录在各上向目录级下的关联目录;Find out the associated directory under each upper directory level of the write directory specified by the object to be written; 如果所述待写入对象的指定写入量小于所述写入目录的剩余许可配额,且小于所述关联目录的剩余许可配额,则将所述待写入对象写入到所述写入目录下;If the specified write amount of the object to be written is less than the remaining quota of the write directory and is smaller than the remaining quota of the associated directory, write the object to be written into the write directory Down; 在查找出待写入对象指定的写入目录在各上向目录级下的关联目录之前,还包括:Before finding out the associated directory under each upward directory level of the write directory specified by the object to be written, it also includes: 根据目录配额需求预先为指定目录设置对应的配额上限,并实时记录每一目录的最新已写入总量,其中,元数据服务端在检测到配额触发操作时,通过解析所述配额触发操作确定出当前的目录配额需求;Set the corresponding quota upper limit for the specified directory in advance according to the directory quota requirements, and record the latest written total amount of each directory in real time. When the metadata server detects the quota trigger operation, it determines by analyzing the quota trigger operation. Display the current directory quota requirements; 所述在查找出待写入对象指定的写入目录在各上向目录级下的关联目录之后,从对象数量和占用内存量两方面来判断待写入对象是否满足所述写入目录和各个所述关联目录下的配额要求;After finding out the associated directory of the writing directory specified by the object to be written in each upward directory level, judge whether the object to be written meets the requirements of the writing directory and each directory from the number of objects and the amount of memory occupied. Quota requirements under the associated directory; 在查找出待写入对象指定的写入目录在各上向目录级下的关联目录之前,还包括:Before finding out the associated directory under each upward directory level of the write directory specified by the object to be written, it also includes: 所述元数据服务端接收所述待写入对象的写入命令,所述写入命令中携带所述待写入对象指定的写入目录;所述元数据服务端在接收到所述写入命令后,分析所述写入命令指定的待写入对象以及该待写入对象当前指定的写入目录。The metadata server receives the write command of the object to be written, and the write command carries the write directory specified by the object to be written; the metadata server receives the write After the command, analyze the object to be written specified by the write command and the write directory currently specified by the object to be written. 2.根据权利要求1所述的方法,其特征在于,在查找出待写入对象指定的写入目录在各上向目录级下的关联目录之后,还包括:2. The method according to claim 1, characterized in that, after finding out the associated directory under each upper directory level of the specified write directory of the object to be written, it also includes: 如果所述待写入对象的指定写入量大于等于所述写入目录的剩余许可配额,或者大于等于所述关联目录的剩余许可配额,则禁止在所述写入目录下写入所述待写入对象。If the specified write amount of the object to be written is greater than or equal to the remaining quota of the write directory, or is greater than or equal to the remaining quota of the associated directory, it is forbidden to write the object to be written under the write directory. write object. 3.根据权利要求1或2所述的方法,其特征在于,在查找出待写入对象指定的写入目录在各上向目录级下的关联目录之后,还包括:3. The method according to claim 1 or 2, further comprising: 针对所述写入目录和所述关联目录中的每一目录,根据该目录的配额上限和最新已写入总量确定该目录的剩余许可配额。For each directory in the write-in directory and the associated directory, determine the remaining permitted quota of the directory according to the upper limit of the quota and the latest total written amount of the directory. 4.根据权利要求3所述的方法,其特征在于,根据该目录的配额上限和最新已写入总量确定该目录的剩余许可配额,包括:4. The method according to claim 3, wherein determining the remaining licensed quota of the directory according to the quota upper limit of the directory and the latest written total amount includes: 如果该目录的配额上限为空,则确定该目录的剩余许可配额无上限;If the quota upper limit of the directory is empty, determine that the remaining licensed quota of the directory is unlimited; 如果该目录的配额上限为非空,则将该目录的配额上限和最新已写入总量的差值作为该目录的剩余许可配额。If the quota upper limit of the directory is not empty, the difference between the upper quota limit of the directory and the latest written total is taken as the remaining licensed quota of the directory. 5.根据权利要求3所述的方法,其特征在于,在将所述待写入对象写入到所述写入目录下之后,还包括:5. The method according to claim 3, characterized in that, after writing the object to be written into the write directory, further comprising: 更新所述写入目录和所述关联目录的最新已写入总量。Updating the latest written totals of the write directory and the associated directory. 6.根据权利要求1或2所述的方法,其特征在于,在查找出待写入对象指定的写入目录在各上向目录级下的关联目录之后,还包括:6. The method according to claim 1 or 2, further comprising: 针对所述写入目录和所述关联目录中的每一目录,根据所述待写入对象的指定写入量中的指定写入对象数量和该目录的剩余许可配额中的剩余许可写入对象数量的第一比对结果,以及所述待写入对象的指定写入量中的指定写入内存量和该目录的剩余许可配额中的剩余许可写入内存量的第二比对结果,判断所述待写入对象的指定写入量是否小于该目录的剩余许可配额。For each directory in the write directory and the associated directory, according to the specified number of write objects in the specified write amount of the objects to be written and the remaining permitted write objects in the remaining permitted quota of the directory The first comparison result of the quantity, and the second comparison result of the specified write-in memory amount in the specified write-in amount of the object to be written and the remaining allowable write-in memory amount in the remaining allowable quota of the directory, determine Whether the specified writing amount of the object to be written is less than the remaining quota of the directory. 7.根据权利要求1或2所述的方法,其特征在于,在查找出待写入对象指定的写入目录在各上向目录级下的关联目录之前,还包括:7. The method according to claim 1 or 2, further comprising: 根据不同目录之间的级联状态,生成对应的目录树;Generate the corresponding directory tree according to the cascading state between different directories; 相应的,所述查找出待写入对象指定的写入目录在各上向目录级下的关联目录,包括:Correspondingly, the searching out the associated directory under each upper directory level of the write directory specified by the object to be written includes: 根据所述目录树查找出待写入对象指定的写入目录在各上向目录级下的关联目录。According to the directory tree, an associated directory under each upper directory level of the write directory specified by the object to be written is found. 8.一种对象写入的装置,其特征在于,包括:8. A device for object writing, characterized in that it comprises: 目录查找模块,用于查找出待写入对象指定的写入目录在各上向目录级下的关联目录;The directory search module is used to find out the associated directory under each directory level of the write directory specified by the object to be written; 对象写入模块,用于如果所述待写入对象的指定写入量小于所述写入目录的剩余许可配额,且小于所述关联目录的剩余许可配额,则将所述待写入对象写入到所述写入目录下;An object writing module, configured to write the object to be written if the specified write amount of the object to be written is less than the remaining allowance quota of the write directory and less than the remaining allowance quota of the associated directory. into the write directory; 在查找出待写入对象指定的写入目录在各上向目录级下的关联目录之前,还包括:Before finding out the associated directory under each upward directory level of the write directory specified by the object to be written, it also includes: 根据目录配额需求预先为指定目录设置对应的配额上限,并实时记录每一目录的最新已写入总量,其中,元数据服务端在检测到配额触发操作时,通过解析所述配额触发操作确定出当前的目录配额需求;According to the quota requirements of the directory, set the corresponding quota upper limit for the specified directory in advance, and record the latest written total amount of each directory in real time. When the metadata server detects the quota trigger operation, it determines by analyzing the quota trigger operation. Display the current directory quota requirements; 所述在查找出待写入对象指定的写入目录在各上向目录级下的关联目录之后,从对象数量和占用内存量两方面来判断待写入对象是否满足所述写入目录和各个所述关联目录下的配额要求;After finding out the associated directory of the writing directory specified by the object to be written in each upward directory level, judge whether the object to be written meets the requirements of the writing directory and each directory from the number of objects and the amount of memory occupied. Quota requirements under the associated directory; 在查找出待写入对象指定的写入目录在各上向目录级下的关联目录之前,还包括:Before finding out the associated directory under each upward directory level of the write directory specified by the object to be written, it also includes: 所述元数据服务端接收所述待写入对象的写入命令,所述写入命令中携带所述待写入对象指定的写入目录;所述元数据服务端在接收到所述写入命令后,分析所述写入命令指定的待写入对象以及该待写入对象当前指定的写入目录。The metadata server receives the write command of the object to be written, and the write command carries the write directory specified by the object to be written; the metadata server receives the write After the command, analyze the object to be written specified by the write command and the write directory currently specified by the object to be written. 9.一种服务器,其特征在于,所述服务器包括:9. A server, characterized in that the server comprises: 一个或多个处理器;one or more processors; 存储装置,用于存储一个或多个程序;storage means for storing one or more programs; 当所述一个或多个程序被所述一个或多个处理器执行,使得所述一个或多个处理器实现如权利要求1-7中任一所述的对象写入的方法。When the one or more programs are executed by the one or more processors, the one or more processors are made to implement the object writing method according to any one of claims 1-7. 10.一种计算机可读存储介质,其上存储有计算机程序,其特征在于,该程序被处理器执行时实现如权利要求1-7中任一所述的对象写入的方法。10. A computer-readable storage medium, on which a computer program is stored, wherein when the program is executed by a processor, the object writing method according to any one of claims 1-7 is implemented.
CN202010327053.XA 2020-04-23 2020-04-23 Method, device, server and storage medium for writing object Active CN111552438B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010327053.XA CN111552438B (en) 2020-04-23 2020-04-23 Method, device, server and storage medium for writing object

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010327053.XA CN111552438B (en) 2020-04-23 2020-04-23 Method, device, server and storage medium for writing object

Publications (2)

Publication Number Publication Date
CN111552438A CN111552438A (en) 2020-08-18
CN111552438B true CN111552438B (en) 2023-07-28

Family

ID=72007634

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010327053.XA Active CN111552438B (en) 2020-04-23 2020-04-23 Method, device, server and storage medium for writing object

Country Status (1)

Country Link
CN (1) CN111552438B (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113626404B (en) * 2021-06-30 2024-02-13 济南浪潮数据技术有限公司 Control method, device and medium for nested directory quota of distributed file system
CN114461571A (en) * 2021-12-24 2022-05-10 天翼云科技有限公司 A kind of quota method and device based on network file system
CN115617747B (en) * 2022-12-16 2023-03-28 新华三信息技术有限公司 File storage method, device and equipment

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103064639B (en) * 2012-12-28 2016-08-03 华为技术有限公司 Data storage method and device
CN104750558B (en) * 2013-12-31 2018-07-03 伊姆西公司 The method and apparatus that resource allocation is managed in quota system is layered
CN105159968A (en) * 2015-08-25 2015-12-16 浪潮(北京)电子信息产业有限公司 Directory management method for file system and client
CN107562386A (en) * 2017-09-13 2018-01-09 郑州云海信息技术有限公司 A kind of method for writing data and device of distributed storage cluster
CN109783467A (en) * 2019-01-12 2019-05-21 郑州云海信息技术有限公司 A kind of nested directory file number quota setting method of distributed file system

Also Published As

Publication number Publication date
CN111552438A (en) 2020-08-18

Similar Documents

Publication Publication Date Title
US8751763B1 (en) Low-overhead deduplication within a block-based data storage
CN110018998B (en) File management method and system, electronic equipment and storage medium
CN111552438B (en) Method, device, server and storage medium for writing object
US10353636B2 (en) Write filter with dynamically expandable overlay
CN108121813B (en) Data management method, device, system, storage medium and electronic equipment
US9307024B2 (en) Efficient storage of small random changes to data on disk
CN109804359A (en) For the system and method by write back data to storage equipment
CN107391038B (en) Data writing method of data storage type flash memory, flash memory and storage medium
CN110471750B (en) Timing task control method, device, medium and computer equipment
US9430492B1 (en) Efficient scavenging of data and metadata file system blocks
CN111198856A (en) File management method and device, computer equipment and storage medium
CN113704027B (en) File aggregation compatible method, apparatus, computer device and storage medium
US20170286442A1 (en) File system support for file-level ghosting
CN104102695A (en) Data processing method and smart device in startup process of smart device
CN109213450B (en) A method, device and device for deleting associated metadata based on flash memory array
US20190034336A1 (en) System and method for hardware-independent memory storage
CN111382011B (en) A file data access method, device and computer-readable storage medium
US9009204B2 (en) Storage system
KR20140042431A (en) Computing system and method for managing data in the system
US8909875B1 (en) Methods and apparatus for storing a new version of an object on a content addressable storage system
CN107704596A (en) A kind of method, apparatus and equipment for reading file
KR102456017B1 (en) Apparatus and method for file sharing between applications
CN109508140B (en) Storage resource management method and device, electronic equipment and system
CN111796972B (en) File hot-repair method, device, equipment and storage medium
US10678453B2 (en) Method and device for checking false sharing in data block deletion using a mapping pointer and weight bits

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20241127

Address after: 31a, 15 / F, building 30, maple mall, bangrang Road, Brazil, Singapore

Patentee after: Baiguoyuan Technology (Singapore) Co.,Ltd.

Country or region after: Singapore

Address before: 5-13 / F, West Tower, building C, 274 Xingtai Road, Shiqiao street, Panyu District, Guangzhou, Guangdong 510000

Patentee before: GUANGZHOU BAIGUOYUAN INFORMATION TECHNOLOGY Co.,Ltd.

Country or region before: China