CN102169497B - Method and device for managing metadata through bitmaps - Google Patents
Method and device for managing metadata through bitmaps Download PDFInfo
- Publication number
- CN102169497B CN102169497B CN 201110092569 CN201110092569A CN102169497B CN 102169497 B CN102169497 B CN 102169497B CN 201110092569 CN201110092569 CN 201110092569 CN 201110092569 A CN201110092569 A CN 201110092569A CN 102169497 B CN102169497 B CN 102169497B
- Authority
- CN
- China
- Prior art keywords
- file
- information
- attribute
- level
- bitmap
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000004048 modification Effects 0.000 claims description 12
- 238000012986 modification Methods 0.000 claims description 12
- 238000007726 management method Methods 0.000 description 28
- 238000005516 engineering process Methods 0.000 description 4
- 230000001052 transient effect Effects 0.000 description 4
- 238000004321 preservation Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- 238000013523 data management Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明披露了一种通过位图方式管理元数据的方法及装置，其中方法包括：将保存元数据的文件通过文件目录级位图记录文件目录信息，通过文件及其属性级位图记录文件信息及文件属性信息；当访问一文件目录下一个文件时，通过文件目录级位图获取指向相应的文件记录的地址的第一指针，并通过该第一指针在文件及其属性级位图中获取相应的文件信息和文件属性信息。本发明通过多级位图分层记录文件目录、文件及其文件属性以及文件更新信息各类元数据，从而实现对文件数据的高效率存储、管理和检索。
The present invention discloses a method and device for managing metadata through a bitmap, wherein the method includes: recording the file directory information of the file storing the metadata through the file directory level bitmap, and recording the file information through the file and its attribute level bitmap and file attribute information; when accessing a file under a file directory, obtain the first pointer pointing to the address of the corresponding file record through the file directory level bitmap, and obtain the file and its attribute level bitmap through the first pointer Corresponding file information and file attribute information. The invention records file directories, files and their file attributes, and various metadata of file update information through multi-level bitmap layers, thereby realizing efficient storage, management and retrieval of file data.
Description
Technical field
The present invention relates to the management method of computer memory system metadata, relate in particular to method and device by bitmap mode management of metadata.
Background technology
Metadata is the data of data of description.In file system, volume management system and fast photographic system, often can use metadata, utilize metadata information to come the feature of data of description.
The form of management of metadata information is various, and metadata has different forms in the different systems.For example in fast photographic system, the quantity of information of metadata is larger, and can increase along with reducing of management data block granularity.But the metadata information of snapshot data is semantic simple, only has a kind of semanteme, so can adopt the mode of bitmap (bitmap) to store and manage such as the such metadata of fast photographic system, this bitmap mode can be saved storage space.By the information that just can describe a data block granularity in the bitmap, namely the change list of each shows the pointer that is indicated to this data block in the bitmap.The bitmap access speed is very fast, and its access is more consistent with the array access mode, need not any traversing operation, and can directly position by the method for calculating.
When the data that are managed are less, can adopt the one-level bitmap to realize the data of additional space are effectively managed; But, take up room when very large when the data that are managed are more, adopt the one-level bitmap to realize will having following two problems to the data management of corresponding very large space:
1) increase of invalid data ratio can cause the waste of storage space.
When data space was very large, some more data may be temporarily invalid concerning calling party, so the ratio of the temporary transient invalid data space of this class will increase greatly; Obviously be these invalid data assignment bit map information in internal memory, the memory source of greatly waste preciousness.
2) access efficiency of bitmap reduces.
At every turn in access during message bit pattern, all need to obtain by the actual value that obtains in the bitmap one pointer of respective data blocks; When the data space that is managed was very large, message bit pattern tended to account for larger memory headroom, so need the support of swapping in and out technology.Therefore only adopt the one-level bitmap to carry out in the situation of metadata management, can do a lot of invalid swapping in and out operations, so that access efficiency reduces greatly.
Summary of the invention
Technical matters to be solved by this invention provides a kind of method and device by bitmap mode management of metadata, can save a large amount of memory headrooms.
In order to solve the problems of the technologies described above, the invention provides a kind of method by bitmap mode management of metadata, comprising:
The file of Preservation Metadata is passed through file directory level bitmap with list structure log file directory information, comprise the unified identifier of first overall situation and first pointer of identification document catalogue; With list structure log file information and file attribute information, this document information comprises the unified identifier of second overall situation of identification document and points to the second pointer of corresponding file attribute information address in the file attribute level bitmap by file and attribute level bitmap thereof; This document attribute information comprises one or more information in title, file size, document creation and/or the modification time of file;
When the next file of access one file directory, obtain the first pointer of the address of pointing to corresponding file record by described file directory level bitmap, and in described file and attribute level bitmap thereof, obtain corresponding fileinfo by this first pointer, obtain the second pointer according to the unified identifier of the overall situation of second in this document information, find corresponding file attribute information according to this second pointer.
Further, the method also comprises:
Know that according to the file attribute information that obtains corresponding file changes, then return the data of the file of change, do not change if know corresponding file, then return the data of this original of not changing.
Further, with tree structure log file changed information, this document changed information records the file data of change by this document attribute level bitmap.
Further, the method also comprises:
Know that according to the file attribute information that obtains corresponding file changes, the file data of the file amendment acquisition of information change by the tree structure record then, and merging in the original returns the file data after the merging; Do not change the original data of then returning not change if know corresponding file.
In order to solve the problems of the technologies described above, the invention provides a kind of device by bitmap mode management of metadata, comprise the file directory level module and file and the attribute level module thereof that connect successively, wherein:
File directory level module is used for the file of Preservation Metadata is passed through file directory level bitmap with list structure log file directory information, comprises the unified identifier of first overall situation and first pointer of identification document catalogue; When the user accesses under the catalogue file, from the file directory information of file directory level figure record, obtain the first pointer of the address of pointing to corresponding file record, and obtain the file record address by this first pointer;
File and attribute level module thereof, be used for by file and attribute level bitmap thereof with list structure log file information and file attribute information, this document information comprises the unified identifier of second overall situation of identification document and points to the second pointer of corresponding file attribute information address in the file attribute level bitmap; This document attribute information comprises one or more information in title, file size, document creation and/or the modification time of file; Obtain corresponding fileinfo according to the file record address from file and attribute level bitmap thereof, obtain the second pointer according to the unified identifier of the overall situation of second in the fileinfo, obtain corresponding file attribute information according to this second pointer.
Further,
File and attribute level module thereof also know that according to the file attribute information that obtains corresponding file changes, then return the file data of change; Do not change if know corresponding file, then return the data of this original of not changing.
Further,
With tree structure log file changed information, this document changed information records the file data of change by file attribute level bitmap.
Further, file and attribute level module thereof comprise file-level unit and the file attribute level unit that connects successively, wherein:
The file-level unit is used for finding corresponding file record by the first pointer at the file-level bitmap, obtains the second pointer according to the unified identifier of second overall situation that records in this document record;
File attribute level unit, the second pointer that is used for obtaining according to the file-level unit obtains corresponding file attribute information from file attribute level bitmap.
Further,
File attribute level unit also knows that by the file attribute information that obtains corresponding file changes, then obtains the file data of change, merges in the original, and returns file data after the merging; Do not change the original data of then returning not change if know corresponding file.
The present invention is by multistage bitmap laminated record file directory, file and file attribute thereof and all kinds of metadata of file update information, thereby realization is to high-level efficiency storage, the management and retrieval of file data.
Description of drawings
Fig. 1 is the framework that the present invention is used for the secondary bitmap embodiment of management of metadata;
Fig. 2 is the embodiment of the method flow process that the present invention is directed to the management of metadata of secondary bitmap shown in Figure 1;
Fig. 3 is the framework that the present invention is used for three grades of bitmap embodiment of management of metadata;
Fig. 4 is the embodiment of the method flow process that the present invention is directed to the management of metadata of three grades of bitmaps shown in Figure 3;
Fig. 5 is the structured flowchart that passes through the device embodiment of bitmap mode management of metadata of the present invention.
Embodiment
Below in conjunction with accompanying drawing and preferred embodiment technical scheme of the present invention is described in detail.The embodiment that below exemplifies only is used for description and interpretation the present invention, and does not consist of the restriction to technical solution of the present invention.
Fig. 1 has represented that the present invention is used for the framework of the secondary bitmap embodiment of management of metadata, comprises file directory level and file and attribute level thereof, wherein:
The file directory level is used for the record directory listing, for example with the mode record of data link table; The information of each catalogue comprises catalogue ID and points to the address that next stage bitmap File is tabulated in the directory listing; Wherein the catalogue ID mode of unifying identifier (GUID, Globally Unique IDentifier) with the overall situation generates, and is unique in whole system to guarantee each catalogue;
File and attribute level thereof are used for the file attribute of listed files and All Files under each catalogue of record directory listing, for example with the mode record of data link table; The information of each file comprises file ID and file attribute information in the listed files, one or more information in title, size, establishment and the modification time of this document attribute information include file; Wherein listed files is with the mode record of data link table, and file ID generates in the mode of GUID equally, is unique in whole system to guarantee each file.
The present invention adopts the mode record of above-mentioned two level frameworks with metadata, and wherein the bitmap capacity of file directory level is less, so can memory-resident; File and attribute level bitmap thereof can be saved in the disk unit.One (forming pointer with other position) in the upper level bitmap describes the data block (data block of log file and attribute level thereof) in the next stage bitmap.
When the shared space of the metadata of management is larger, the above-mentioned way that metadata is divided into the two-stage bitmap management of the present invention, can avoid adopting one-level bitmap management mode user once access can have access to many temporary transient invalid data, but when needing, the user just has access to next stage from upper level, both dwindle the space of visit data, also improved simultaneously the efficient of visit data.
Fig. 2 expresses the secondary bitmap that the present invention is directed to management of metadata shown in Figure 1 and the embodiment of the method flow process of the management of metadata that provides comprises:
110: meta data file is recorded file directory information by file directory level figure, and record file and file attribute information by file and attribute level figure thereof;
The file directory level is with the mode log file directory listing of data link table, and the information of each catalogue comprises catalogue ID and points to the address that next stage bitmap File is tabulated in the directory listing.
120: when the user accesses the next file of a file directory, obtain corresponding file and file attribute information address by file directory level bitmap;
Find corresponding pointer in sensing file and the attribute level bitmap thereof by the catalogue ID that records in the file directory level bitmap, and by the corresponding fileinfo of this pointed address.
130: obtain corresponding file record by file and attribute level bitmap thereof, find corresponding file attribute information according to the file ID in the file record;
In file and attribute level bitmap thereof, find corresponding listed files (being file record) by the pointer that points to the fileinfo address, find corresponding file attribute information according to the file ID in the listed files.
140: return corresponding file data, process ends according to file attribute information.
According to file attribute information (such as file name, size, establishment and modification time etc.), if know the file unmodified, then return the original data; Otherwise return the file data of modification.
When the shared space of metadata of management very large, when causing adopting the secondary bitmap also can appearance can access many temporary transient invalid data, be necessary above-mentioned file and attribute level thereof are further divided into secondary, the framework of three grades of bitmap embodiment as shown in Figure 3, that is except comprising the file directory level, file and attribute fraction thereof are become file-level and file attribute level, wherein:
The file directory level, effect is with described in the above-mentioned secondary bitmap framework, and this repeats no more;
File-level, be used for the mode log file tabulation with data link table, the information of each file that records in the listed files comprises file ID and points to the address of next stage bitmap File attribute that its file ID generates in the mode of GUID, is unique to guarantee each file in whole system;
The file attribute level, be used for respectively log file attribute information and file amendment information, wherein file attribute information comprises one or more information in file name, file size, creation-time and the modification time with the list structure record, file amendment information is with the data of tree structure log file change, and tree structure for example adopts B tree or B+ tree.
Fig. 4 expresses three grades of bitmaps that the present invention is directed to management of metadata shown in Figure 3 and the embodiment of the method flow process of the management of metadata that provides comprises:
210: when the user accesses the next file of a file directory, obtain corresponding file record address by the file directory level;
Find corresponding pointer in the sensing file-level bitmap by the catalogue ID that records in the file directory level, and by the corresponding listed files of this pointed (being file record) address.
220: obtain corresponding file attribute address by corresponding file record in the file-level;
File record File ID by file-level finds corresponding pointer in the sensing file attribute level bitmap, and by the corresponding file attribute of this pointed address.
230: obtain corresponding file attribute information by the file attribute level;
Get access to file attribute information and the file amendment information of record by the file attribute level, wherein file attribute information comprises one or more information in file name, file size, creation-time and the modification time; The data of file amendment information recording/file amendment.
240: judging whether corresponding file is changed, is execution in step 250 then, otherwise execution in step 260;
Know according to file attribute information File creation-time and/or modification time information whether file is changed.
250: obtain the file data of change, merge in the original, return the file data after the merging, process ends;
From the file amendment information of corresponding tree structure, get access to the data of file amendment, merge in the original and return.
260: return the original data, process ends.
Above-mentioned way of the present invention, can further avoid adopting one-level or secondary bitmap management mode user once access can have access to many temporary transient invalid data, but when needing, the user just has access to next stage, can further save thus the space of visit data, also further improve simultaneously the efficient of management and visit data.
The present invention is directed to said method embodiment, the device embodiment by bitmap mode management of metadata correspondingly also is provided, its structure comprises the file directory level module and file and the attribute level module thereof that connect successively as shown in Figure 5, wherein:
File directory level module is used for obtaining corresponding file record address from file directory level bitmap when the user accesses under the catalogue file;
File directory level module finds by the catalogue ID that records in the file directory level bitmap and points to corresponding pointer in file and the attribute level bitmap thereof, and by the corresponding listed files of this pointed address.
File and attribute level module thereof are used for obtaining corresponding file record from file and attribute level bitmap thereof, obtain corresponding file attribute information according to the fileinfo in the file record, and return corresponding file data according to file attribute information.
File and attribute level bitmap Further Division thereof are file-level bitmap and file attribute level bitmap, and file and attribute level module thereof further comprise file-level unit and the file attribute level unit that connects successively, wherein:
The file-level unit is used for finding corresponding pointer in the sensing file attribute level bitmap by the file ID that the file-level bitmap records;
File attribute level unit, the pointer that is used for obtaining according to the file-level unit obtains corresponding file attribute information, knows by the file attribute information that obtains whether corresponding file is changed, if the file data of change is then obtained in change, merge in the original, and return file data after the merging; If the original data of obtaining are not then returned in change.
More than explanation only is the better embodiment of the present invention; but protection scope of the present invention is not limited to this; anyly be familiar with those skilled in the art in the technical scope that the present invention discloses, the variation that can expect easily or replacement all should be encompassed within protection scope of the present invention.Therefore, protection scope of the present invention should be as the criterion with the protection domain of claims.
Claims (9)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110092569 CN102169497B (en) | 2011-04-13 | 2011-04-13 | Method and device for managing metadata through bitmaps |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110092569 CN102169497B (en) | 2011-04-13 | 2011-04-13 | Method and device for managing metadata through bitmaps |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN102169497A CN102169497A (en) | 2011-08-31 |
| CN102169497B true CN102169497B (en) | 2013-04-17 |
Family
ID=44490659
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201110092569 Active CN102169497B (en) | 2011-04-13 | 2011-04-13 | Method and device for managing metadata through bitmaps |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102169497B (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622286B (en) * | 2012-02-23 | 2016-02-03 | 浪潮(北京)电子信息产业有限公司 | A kind of raid-array bitmap backup method and system |
| CN102693286B (en) * | 2012-05-10 | 2014-03-26 | 华中科技大学 | Method for organizing and managing file content and metadata |
| CN103544156B (en) * | 2012-07-10 | 2019-04-09 | 腾讯科技（深圳）有限公司 | File memory method and device |
| CN104598533B (en) * | 2014-12-29 | 2018-08-14 | 曙光云计算集团有限公司 | A kind of storage method of file extent attribute |
| CN105653648A (en) * | 2015-12-28 | 2016-06-08 | 北京金山安全软件有限公司 | File acquisition method and device and electronic equipment |
| CN109445713A (en) * | 2018-11-09 | 2019-03-08 | 郑州云海信息技术有限公司 | A kind of storage state recording method, system and the associated component of metadata volume |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7206450B2 (en) * | 2002-04-25 | 2007-04-17 | Microsoft Corporation | Compression of bi-level images with explicit representation of ink clusters |
| CN101499026B (en) * | 2008-02-01 | 2012-07-04 | 凯迈（洛阳）测控有限公司 | Miniature data backup method of data acquisition system based on FLASH ROM |
| CN101908202B (en) * | 2010-07-09 | 2012-05-09 | 哈尔滨工程大学 | A Quick Display Method of Electronic Nautical Chart |
-
2011
- 2011-04-13 CN CN 201110092569 patent/CN102169497B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN102169497A (en) | 2011-08-31 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102169497B (en) | Method and device for managing metadata through bitmaps | |
| CN102725752B (en) | Method and device for processing dirty data | |
| CN102982151B (en) | Method for merging multiple physical files into one logic file | |
| CN101916290B (en) | Managing method of internal memory database and device | |
| CN102375853A (en) | Distributed database system, method for building index therein and query method | |
| CN103282899A (en) | File system data storage method and access method and device therefor | |
| WO2014110940A1 (en) | A method, apparatus and system for storing, reading the directory index | |
| CN103595797A (en) | Caching method for distributed storage system | |
| CN102693286A (en) | Method for organizing and managing file content and metadata | |
| CN106354805A (en) | Optimization method and system for searching and caching distribution storage system NoSQL | |
| US20150169623A1 (en) | Distributed File System, File Access Method and Client Device | |
| CN112131214A (en) | Method, system, device and storage medium for data writing and data query | |
| CN108319608A (en) | The method, apparatus and system of access log storage inquiry | |
| CN103473337A (en) | Massive catalogs and files oriented processing method in distributed type storage system | |
| JP2008262561A (en) | System and method for aiding file search and file operation by indexing historical file names and locations | |
| CN104598652B (en) | A kind of data base query method and device | |
| WO2020024798A1 (en) | Efficient storage method for time series data | |
| CN104516945A (en) | Hadoop distributed file system metadata storage method based on relational data base | |
| CN110704431A (en) | Hierarchical storage management method for mass data | |
| Cai et al. | The Embedded IoT Time Series Database for Hybrid Solid‐State Storage System | |
| CN108256019A (en) | Database key generation method, device, equipment and its storage medium | |
| CN115718571A (en) | Data management method and device based on multi-dimensional features | |
| CN102831240A (en) | Storage method and storage structure of extensible metadata documents | |
| CN117215477A (en) | Data object storage method, device, computer equipment and storage medium | |
| US9442948B2 (en) | Resource-specific control blocks for database cache |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| TR01 | Transfer of patent right | ||
| TR01 | Transfer of patent right |
Effective date of registration: 20201123 Address after: 215100 No. 1 Guanpu Road, Guoxiang Street, Wuzhong Economic Development Zone, Suzhou City, Jiangsu Province Patentee after: SUZHOU LANGCHAO INTELLIGENT TECHNOLOGY Co.,Ltd. Address before: 100085 Beijing, Haidian District on the road to information on the ground floor, building 2-1, No. 1, C Patentee before: Inspur (Beijing) Electronic Information Industry Co.,Ltd. |