CN101533408B - Processing method and processing device of mass data - Google Patents
Processing method and processing device of mass data Download PDFInfo
- Publication number
- CN101533408B CN101533408B CN200910082496.0A CN200910082496A CN101533408B CN 101533408 B CN101533408 B CN 101533408B CN 200910082496 A CN200910082496 A CN 200910082496A CN 101533408 B CN101533408 B CN 101533408B
- Authority
- CN
- China
- Prior art keywords
- data
- file
- stored
- data object
- address offset
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a processing method and a processing device of mass data. The method comprises the followings: a data file and an index file are set, wherein the data file comprises at least one file data block, and the length of each file data block is equal; the index file corresponds to the data file, comprising the address offset of each data object in the data file; when the data object is accessed, if the free space of the file data block currently appointed by a cursor is not enough to store the data object to be stored, the data file applies one or more new file data blocks to the system after the free space is filled with the data object to be stored, and the remaining data that is not accessed in the data object to be stored is stored in the new file data block, and the address offset of newly accessed data object is recorded in the index file; and the stored data object is read in the data file in a manner that the cursor points to the address offset of the data object to be read. The application of the technical proposal improves the access efficiency for mass data.
Description
Technical field
The present invention relates to data processing field, particularly relate to a kind of disposal route and treating apparatus of mass data.
Background technology
At data processing industries, need data grows to be processed many, data file is increasing.At present, for the access issues of mass data, be generally summed up as following 2 kinds of methods:
1) include mass data in database to manage;
2) data object is set up serializing stored in common file.
Realizing in process of the present invention, inventor finds that in prior art, at least there are the following problems:
Method 1) be now popular Mass Data Management mode, although data base administration large files mass data has certain advantage, but due to the method employing is that database is to carry out data management, for mass data, maximum bottleneck is efficiency, for the system that efficiency requirements is higher, database cannot meet efficient demand;
And method 2) problem of small data quantity can only be solved, cannot the storage demand of satisfying magnanimity data.
So, the quick access of large-data documents is accessed, more and more becomes the technical bottleneck of industry, and traditional requirement being difficult to satisfying magnanimity data large files by I/O mode reading and writing of files.
Summary of the invention
In order to solve the problem, the object of this invention is to provide a kind of disposal route and treating apparatus of mass data, to improve the access efficiency to mass data.
To achieve these goals, the invention provides a kind of disposal route of mass data, wherein, comprising:
Arrange a data file and index file, described data file is for storing data object, and described data file comprises at least one file data blocks, and in described data file, the length of each file data blocks is equal; Described index file is corresponding with described data file, and described index file comprises in described data file the address offset of each data object in described data file stored;
By data object stored in described data file time, judge in described data file, whether the remaining space of the file data blocks of the current sensing of vernier enough store data object to be stored; In this way, then by described data object to be stored stored in described remaining space, and record described newly stored in the address offset of data object in described index file; Otherwise, after described remaining space fills up by described data object to be stored, described data file is to one or more new file data blocks of system application, and by described data object to be stored also not stored in remaining data be deposited in described new file data blocks, and record described newly stored in the address offset of data object in described index file;
The data object stored in described data file is read by the address offset of the data object pointing to described vernier for reading.
Preferably, described disposal route, wherein, the length of the file data blocks in described data file is: the integral multiple of the smallest allocation granularity of default.
Preferably, described disposal route, wherein, described data object is: binary object, or data stream.
Preferably, described disposal route, wherein, described in the data object be stored in described file data blocks comprise: data mode, data length and data content; Described data mode comprises: insert state and remove state.
Preferably, described disposal route, wherein, described method also comprises the step of the data object deleted in described data file, is specially:
According to input, for deleting the call number of data object, from described index file, obtain the described address offset for deleting data object, and the described data mode for deleting data object is set to remove state.
Preferably, described disposal route, wherein, also comprises: traversal reads the step of data object in described data file, is specially:
Step a, is set to data file head by the vernier of described data file;
Step b, moves to the address offset of next data object by described vernier, and obtains the data content of the data object of the current sensing of vernier, data length and data mode;
Step c, judges whether to have read all data objects; In this way, then terminate; Otherwise, proceed to step b.
Preferably, described disposal route, wherein, also comprises: the random step reading data object in described data file, is specially:
According to input, the call number of data object for reading at random, from described index file, obtain the address offset of the described data object for reading at random in described data file;
The vernier of described data file is pointed to the address offset of described acquisition, and read the data content of data object corresponding to this address offset, data length and data mode.
On the other hand, provide a kind for the treatment of apparatus of mass data, wherein, comprising:
File arranges module, and for arranging a data file and index file, described data file is for storing data object, and described data file comprises at least one file data blocks, and in described data file, the length of each file data blocks is equal; Described index file is corresponding with described data file, and described index file comprises in described data file the address offset of each data object in described data file stored;
Memory module, for when data object is arranged the described data file of module installation stored in described file, judge in described data file, whether the remaining space of the file data blocks of the current sensing of vernier enough store data object to be stored, and record described newly stored in the address offset of data object in described index file; In this way, then by described data object to be stored stored in described remaining space; Otherwise, after described remaining space fills up by described data object to be stored, described data file is to one or more new file data blocks of system application, and by described data object to be stored also not stored in remaining data be deposited in described new file data blocks, and record described newly stored in the address offset of data object in described index file;
Read module, the address offset for the data object by pointing to described vernier for reading reads the data object stored in described data file.
Preferably, described treating apparatus, wherein, the length of the file data blocks in described data file is: the integral multiple of the smallest allocation granularity of default.
Preferably, described treating apparatus, wherein, described in the data object be stored in described file data blocks comprise: data mode, data length and data content; Described data mode comprises: insert state and remove state.
At least one technical scheme in technique scheme has following beneficial effect:
The address offset corresponding by each data object in storing data files in setting data file and index file, indexed file and by data file application documents data block, the access mode of File Mapping internal memory can be realized, thus by pointer access disk file, by the file part of disk or be all mapped to particular address range in the process address space, then just can the content of access memory mapped file by pointer, thus I/O operation need not be performed to file, like this concerning big data quantity, access efficiency is high.And due to the file data blocks in data file be all isometric, thus the file data blocks of the every sub-distribution equal length of system.Like this when data file is mapped to internal memory, each File Mapping, all only have mapped the blocks of files of one piece of regular length, thus need not record the file size needing each time to map, and has unified rule, has saved a large amount of internal memories, also improved efficiency.
Accompanying drawing explanation
Fig. 1 is the schematic flow sheet of the disposal route of the mass data of one embodiment of the invention;
Fig. 2 is in one embodiment of the invention, an example of data file and the concrete file structure of index file;
Fig. 3 is in one embodiment of the invention, the schematic flow sheet of update;
Fig. 4 is in one embodiment of the invention, the schematic flow sheet of traversal read operation;
Fig. 5 is in one embodiment of the invention, the schematic flow sheet of a kind of specific implementation logic of traversal read operation;
Fig. 6 is the structural representation of the treating apparatus of the mass data of the embodiment of the present invention.
Embodiment
For making the technical problem to be solved in the present invention, technical scheme and advantage clearly, be described in detail below in conjunction with the accompanying drawings and the specific embodiments.
Fig. 1 is the schematic flow sheet of mass data processing method in the embodiment of the present invention.As Fig. 1, the mass data processing method of one embodiment of the invention comprises:
Step 101, arranges a data file and index file, and this data file is for storing data object, and described data file comprises at least one file data blocks, and in described data file, the length of each file data blocks is equal; Described index file is corresponding with described data file, and described index file comprises in described data file the address offset of each data object in described data file stored;
Step 102, by data object stored in described data file time, judge in described data file, whether the remaining space of the file data blocks of the current sensing of vernier enough store data object to be stored; In this way, then step 103 is performed; Otherwise, perform step 104;
Step 103, by described data object to be stored stored in described remaining space, and records described newly stored in the address offset of data object, process ends in described index file;
Step 104, after described remaining space fills up by described data object to be stored, described data file is to one or more new file data blocks of system application, and by described data object to be stored also not stored in remaining data be deposited in described new file data blocks, and record described newly stored in the address offset of data object in described index file;
Step 105, reads by the address offset of the data object pointing to described vernier for reading the data object stored in described data file.
In embodiments of the invention, data object is: binary object, or data stream arbitrarily.
In the embodiment of the present invention, by the address offset of each data object in recording data files in setting data file and index file, indexed file and by data file application documents data block, the access mode of File Mapping internal memory can be realized, this storage mode provides a kind of memory management feature of uniqueness, it can access disk file by pointer, similarly is access Dram.By the file part of disk or be all mapped to particular address range in the process address space, then just can the content of access memory mapped file by pointer, thus I/O operation need not be performed to file, like this concerning big data quantity, access efficiency is high.And due to the file data blocks in data file be all isometric, thus the file data blocks of the every sub-distribution equal length of system.Like this when data file is mapped to internal memory, each File Mapping, all only have mapped the blocks of files of one piece of regular length, thus need not record the file size needing each time to map, and has unified rule, has saved a large amount of internal memories, also improved efficiency.
Below by an object lesson, the present invention will be described.
In this embodiment, the access of data as mass data is realized by setting data file and index file.Wherein, the physics mode of data file shows as the file data blocks (Block) of one group of fixed length, and the length of each file data blocks is equal, and is the integral multiple of the smallest allocation granularity of default.In this example, utilize the file data blocks of fixed length to store elongated data object.Exemplarily, as in Windows system, the smallest allocation granularity of default is 64k, then in data file, the length of each file data blocks is the integral multiple of 64k.The logical course of data file shows as a series of elongated data object.Each data object is stored in file data blocks in a predetermined manner.Index file is corresponding with data file, for recording each data object address offset in the data file (Offset).Exemplarily, this address offset is the distance of the reference position distance file header of data object.Exemplarily, from this index file, the address offset of corresponding Blob is obtained by the call number of data object.Exemplarily, this call number may correspond in Blob sequence number in the data file.
In embodiments of the invention, the length of the file data blocks of data file is the integral multiple of the smallest allocation granularity of default, thus the blocks of files length of the minimum particle size integral multiple at every turn needing distribution system to set, for the virtual memory system adopted based on page (Page-based), virtual memory management adopts the mode of page to carry out data transmission, the unified deposit of execution in an identical manner read-write operation, decreases read-write number of times, greatly improves literacy.。
Fig. 2 shows in embodiments of the invention, an example of data file and the concrete file structure of index file.In this embodiment, data object is scale-of-two large data objects (Blob).Each Blob comprises: data mode, data length and data content.Data mode comprises: insert state and remove state; Described insert state, for identify described data mode be stored in valid data; Describedly removing state, is deleted invalid data for identifying described data mode.Physically, in data file, deleted data object still exists, but when reading data, when the data mode of the data object read is for removing, can judge that these data are deleted data.As Fig. 2, exemplary but not as restriction ground, in the data file, the recording mode of each Blob can be: the data content of the data length+n Byte of the data mode+4Byte of a byte and 1Byte, n is natural number.Wherein, data mode comprises: insert state and remove state, is the data object or the data object deleted that insert for indicating this data object.In the index file of this example, every 4 byte records address offset, the address offset of different pieces of information object is arranged in order according to data object order in the data file.Certainly, the above-mentioned byte number for recording data mode, data length and address offset is not limited to above-mentioned mentioning, byte number used can more also can be less.
In embodiments of the invention, storage to mass data, deletion and reading can be realized by carrying out above-mentioned data file and/or index file operating.Below above-mentioned several file operation is described respectively:
Update: by the data object for storing as Blob data inserting file.Fig. 3 shows the schematic flow sheet of the update of the embodiment of the present invention.As Fig. 3, this update comprises:
Step 301, after the length of Blob and the data content of Blob of the insertion of input wish, judges whether the remaining space of the Block of the current sensing of vernier in data file enough stores the Blob of insertion; In this way, then step 302 is performed; Otherwise, perform step 303;
Step 302, by this Blob directly stored in this remaining space, and is set to insert state by the state of this Blob, then records the address offset of this Blob in indexed file;
Step 303, first with this Blob, above-mentioned remaining space is filled up, then, data file is to one or more new Block of system application, continue the residue fills of Blob do not inserted in the Block of above-mentioned new application, then the state of this Blob is set to insert state, and after filling action completes, in indexed file, records the address offset of this Blob.
Deletion action: delete an elongated data object in data file.Specifically comprise: input is for deleting the call number of Blob; According to this call number, from index file, obtain these elongated data address offset in the data file, and the state of this Blob is set to the state of removing.
In an embodiment of the present invention, after data object write data file, when deleting data object, physically the data object of this deletion is not removed, and just the state of the data object of this deletion is set to remove state.Such access efficiency that can improve mass data.
Traversal obtains operation: all elongated data object stored in ergodic data file successively.Fig. 4 shows the schematic flow sheet of the traversal read operation of the embodiment of the present invention.As Fig. 4, this operation comprises:
Step 401, resets file vernier, the vernier of data file is set to data file head;
Step 402, moving cursor, to the address offset of next Blob, then obtains the data content of the Blob of the current sensing of vernier, data length and data mode;
Step 403, judges whether to have read all data objects; In this way, then terminate; Otherwise, proceed to and perform step 402, continue to read the Blob not yet read.
Exemplarily, Fig. 5 is the schematic flow sheet of the specific implementation logic that traversal obtains.As Fig. 5, this specific implementation comprises:
Step 501, resets file vernier;
Step 502, moving cursor is to next address;
Step 503, judges whether this address is end-of-file; In this way, then terminate to read; Otherwise, perform step 504;
Step 504, reads the data mode of Blob corresponding to vernier current sensing address, data length and data content, and after having read the content of this Blob, proceeds to and perform step 502.
Random read operation, for obtaining the fixed data object of data document at random.This operation, specifically comprises:
According to input, the call number of data object for reading at random, from described index file, obtain the address offset of the described data object for reading at random in described data file;
The vernier of described data file is pointed to the address offset of described acquisition;
Judge that this address offset is the end-of-file of data file; In this way, then process ends; Otherwise, read the data content of data object corresponding to this address offset, data length and data mode.
Utilize file structure and the file operation of the above embodiment of the present invention, under following test environment, store 2,000 ten thousand object datas, storage time is 1281 milliseconds, and whole reading time 406 milliseconds, file size is 488320KB.
Test environment:
Window XP system
Intel Core2 Duo CPU 2.10HZ
2G internal memory,
Hitachi HTS542516K9A300 disk
Above-mentioned test case is only used to the technical scheme utilizing the embodiment of the present invention is described, substantially increases the access efficiency to mass data, and is not used in and restricts the present invention.The technical scheme of the embodiment of the present invention also can be applicable to other operating system and hardware environment.For different applied environments, as object data, above-mentioned storage time, reading time and file size all may be different.
One of ordinary skill in the art will appreciate that, the all or part of step realized in the method for the above-mentioned fact is that the hardware that can carry out instruction relevant by program has come, described program can be stored in the storage medium that a computing machine can read, and this program, when performing, comprises the steps:
Arrange a data file and index file, described data file is for storing data object, and described data file comprises at least one file data blocks, and in described data file, the length of each file data blocks is equal; Described index file is corresponding with described data file, and described index file comprises in described data file the address offset of each data object in described data file stored;
By data object stored in described data file time, judge in described data file, whether the remaining space of the file data blocks of the current sensing of vernier enough store data object to be stored; In this way, then by described data object to be stored stored in described remaining space, and record described newly stored in the address offset of data object in described index file; Otherwise, after described remaining space fills up by described data object to be stored, described data file is to one or more new file data blocks of system application, and by described data object to be stored also not stored in remaining data be deposited in described new file data blocks, and record described newly stored in the address offset of data object in described index file;
The data object stored in described data file is read by the address offset of the data object pointing to described vernier for reading.
Described storage medium, as ROM/RAM, disk, CD etc.
Present invention also offers a kind for the treatment of apparatus of mass data.As Fig. 6, the data processing equipment of the embodiment of the present invention comprises: file arranges module 601, for arranging a data file and index file, described data file is for storing data object, described data file comprises at least one file data blocks, and in described data file, the length of each file data blocks is equal; Described index file is corresponding with described data file, and described index file comprises in described data file the address offset of each data object in described data file stored; Memory module 602, for when data object is arranged the described data file of module installation stored in described file, judge in described data file, whether the remaining space of the file data blocks of the current sensing of vernier enough store data object to be stored, and record described newly stored in the address offset of data object in described index file; In this way, then by described data object to be stored stored in described remaining space; Otherwise, after described remaining space fills up by described data object to be stored, described data file is to one or more new file data blocks of system application, and by described data object to be stored also not stored in remaining data be deposited in described new file data blocks, and record described newly stored in the address offset of data object in described index file.Read module 603, the address offset for the data object by pointing to described vernier for reading reads the data object stored in described data file.
In the data processing equipment of the embodiment of the present invention, the length of the file data blocks in described data file is: the integral multiple of the smallest allocation granularity of default.
In the data processing equipment of the embodiment of the present invention, described in the data object be stored in described file data blocks comprise: data mode, data length and data content; Described data mode comprises: insert state and remove state.
Utilize the technical scheme of the embodiment of the present invention, greatly can improve the access efficiency to mass data, thus can high speed buffer storage file, persistent repository file be set up for mass data and/or set up magnanimity serializing file for data object.And the data utilizing the technical scheme of the embodiment of the present invention to store, be not easily cracked.
The preferred embodiment of the present invention described in upper; it should be pointed out that for those skilled in the art, under the prerequisite not departing from principle described in the embodiment of the present invention; can also make some improvements and modifications, these improvements and modifications also should be considered as protection scope of the present invention.
Claims (8)
1. a disposal route for mass data, is characterized in that, comprising:
Arrange a data file and index file, described data file is for storing data object, and described data file comprises at least one file data blocks, and in described data file, the length of each file data blocks is equal; Described index file is corresponding with described data file, and described index file comprises in described data file the address offset of each data object in described data file stored;
By data object stored in described data file time, judge in described data file, whether the remaining space of the file data blocks of the current sensing of vernier enough store data object to be stored; In this way, then by described data object to be stored stored in described remaining space, and in described index file record newly stored in the address offset of data object; Otherwise, after described remaining space fills up by described data object to be stored, described data file is to one or more new file data blocks of system application, and by described data object to be stored also not stored in remaining data be deposited in described new file data blocks, and record described newly stored in the address offset of data object in described index file;
The data object stored in described data file is read by the address offset of the data object pointing to described vernier for reading;
The length of the file data blocks in described data file is: the integral multiple of the smallest allocation granularity of default.
2. the disposal route of mass data according to claim 1, is characterized in that, described data object is: binary object, or data stream.
3. the disposal route of mass data according to claim 1, is characterized in that, described in the data object be stored in described file data blocks comprise: data mode, data length and data content; Described data mode comprises: insert state and remove state.
4. the disposal route of mass data according to claim 3, is characterized in that, described method also comprises the step of the data object deleted in described data file, is specially:
According to input, for deleting the call number of data object, from described index file, obtain the described address offset for deleting data object, and the described data mode for deleting data object is set to remove state.
5. the disposal route of the mass data according to claim 3 or 4, is characterized in that, also comprises: traversal reads the step of data object in described data file, is specially:
Step a, is set to data file head by the vernier of described data file;
Step b, moves to the address offset of next data object by described vernier, and obtains the data content of the data object of the current sensing of vernier, data length and data mode;
Step c, judges whether to have read all data objects; In this way, then terminate; Otherwise, proceed to step b.
6. the disposal route of mass data according to claim 4, is characterized in that, also comprises: the random step reading data object in described data file, is specially:
According to input, the call number of data object for reading at random, from described index file, obtain the address offset of the described data object for reading at random in described data file;
The vernier of described data file is pointed to the address offset of described acquisition, and read the data content of data object corresponding to this address offset, data length and data mode.
7. a treating apparatus for mass data, is characterized in that, comprising:
File arranges module, and for arranging a data file and index file, described data file is for storing data object, and described data file comprises at least one file data blocks, and in described data file, the length of each file data blocks is equal; Described index file is corresponding with described data file, and described index file comprises in described data file the address offset of each data object in described data file stored;
Memory module, for when data object is arranged the described data file of module installation stored in described file, judge in described data file, whether the remaining space of the file data blocks of the current sensing of vernier enough store data object to be stored, and in described index file record newly stored in the address offset of data object; In this way, then by described data object to be stored stored in described remaining space; Otherwise, after described remaining space fills up by described data object to be stored, described data file is to one or more new file data blocks of system application, and by described data object to be stored also not stored in remaining data be deposited in described new file data blocks, and record described newly stored in the address offset of data object in described index file;
Read module, the address offset for the data object by pointing to described vernier for reading reads the data object stored in described data file;
The length of the file data blocks in described data file is: the integral multiple of the smallest allocation granularity of default.
8. treating apparatus according to claim 7, is characterized in that, described in the data object be stored in described file data blocks comprise: data mode, data length and data content; Described data mode comprises: insert state and remove state.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910082496.0A CN101533408B (en) | 2009-04-21 | 2009-04-21 | Processing method and processing device of mass data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN200910082496.0A CN101533408B (en) | 2009-04-21 | 2009-04-21 | Processing method and processing device of mass data |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101533408A CN101533408A (en) | 2009-09-16 |
| CN101533408B true CN101533408B (en) | 2015-02-04 |
Family
ID=41104014
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN200910082496.0A Active CN101533408B (en) | 2009-04-21 | 2009-04-21 | Processing method and processing device of mass data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101533408B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111090396A (en) * | 2019-12-10 | 2020-05-01 | 联想(北京)有限公司 | File processing method and device and electronic equipment |
Families Citing this family (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102193917B (en) * | 2010-03-01 | 2014-03-26 | 中国移动通信集团公司 | Method and device for processing and querying data |
| CN102193873B (en) * | 2010-03-04 | 2013-06-05 | 中国科学院微电子研究所 | Method for quickly reading and writing physical layout data |
| CN102810092B (en) * | 2011-05-31 | 2017-07-07 | 腾讯科技（深圳）有限公司 | data read-write method and system |
| CN102306194B (en) * | 2011-09-08 | 2013-05-22 | 北京掌城科技有限公司 | Methods for storing and querying GPS (global positioning system) data |
| CN103186537B (en) * | 2011-12-27 | 2018-01-12 | 腾讯科技（北京）有限公司 | A kind of data read-write method and device based on index order |
| CN103257831B (en) | 2012-02-20 | 2016-12-07 | 深圳市腾讯计算机系统有限公司 | The read/writing control method of memorizer and the memorizer of correspondence |
| CN103294710B (en) * | 2012-02-28 | 2016-08-31 | 北京新媒传信科技有限公司 | A kind of data access method and device |
| CN104102552A (en) * | 2013-04-15 | 2014-10-15 | 深圳中兴网信科技有限公司 | Message processing method and device |
| CN103488710B (en) * | 2013-09-10 | 2018-04-24 | 广州巨杉软件开发有限公司 | The non-fixed-length data method of efficient storage in big data page |
| CN104462141B (en) * | 2013-09-24 | 2018-05-22 | 中国移动通信集团重庆有限公司 | Method, system and the storage engines device of a kind of data storage and inquiry |
| CN105630779A (en) * | 2014-10-27 | 2016-06-01 | 杭州海康威视系统技术有限公司 | Hadoop distributed file system based small file storage method and apparatus |
| CN104504001B (en) * | 2014-12-04 | 2017-08-08 | 西北工业大学 | Towards the vernier building method of magnanimity distributed relational database |
| CN104834604B (en) * | 2015-04-22 | 2018-02-16 | 上海微小卫星工程中心 | A kind of IO addresses are to I/O object quick mapping method and system |
| CN106021536A (en) * | 2016-05-27 | 2016-10-12 | 成都索贝数码科技股份有限公司 | Data insertion method and system based on storage of FICS objects |
| CN106980685B (en) * | 2017-03-31 | 2020-04-17 | 广东神马搜索科技有限公司 | Data processing method and data processing device |
| CN107704202B (en) * | 2017-09-18 | 2021-09-07 | 北京京东尚科信息技术有限公司 | Method and device for quickly reading and writing data |
| CN110597762A (en) * | 2018-05-25 | 2019-12-20 | 杭州海康威视系统技术有限公司 | File processing method, device, equipment and storage medium |
| CN111026890A (en) * | 2019-11-28 | 2020-04-17 | 天脉聚源(杭州)传媒科技有限公司 | Picture data storage method, system, device and storage medium based on index table |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101039278A (en) * | 2007-03-30 | 2007-09-19 | 华为技术有限公司 | Data management method and system |
-
2009
- 2009-04-21 CN CN200910082496.0A patent/CN101533408B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101039278A (en) * | 2007-03-30 | 2007-09-19 | 华为技术有限公司 | Data management method and system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111090396A (en) * | 2019-12-10 | 2020-05-01 | 联想(北京)有限公司 | File processing method and device and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101533408A (en) | 2009-09-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101533408B (en) | Processing method and processing device of mass data | |
| CN107168657B (en) | Virtual disk hierarchical cache design method based on distributed block storage | |
| US9684462B2 (en) | Method and apparatus utilizing non-uniform hash functions for placing records in non-uniform access memory | |
| US7761648B2 (en) | Caching method for NAND flash translation layer | |
| CN100498740C (en) | Data cache processing method, system and data cache device | |
| CN113094003B (en) | Data processing method, data storage device and electronic equipment | |
| US20120047324A1 (en) | Sequential access storage and data de-duplication | |
| CN102169460A (en) | Method and device for managing variable length data | |
| US20170357462A1 (en) | Method and apparatus for improving performance of sequential logging in a storage device | |
| WO2010125574A1 (en) | Description | |
| CN107741947B (en) | Method for storing and acquiring random number key based on HDFS file system | |
| CN109213423A (en) | Concurrent I/O command is handled without lock based on address barrier | |
| CN103309619A (en) | NAND Flash memory data storage method | |
| CN115878027A (en) | Storage object processing method and device, terminal and storage medium | |
| CN101783814A (en) | Metadata storing method for mass storage system | |
| CN112395260B (en) | Data storage method and medium | |
| CN107894875A (en) | A kind of RTU date storage methods | |
| CN112596949A (en) | High-efficiency SSD (solid State disk) deleted data recovery method and system | |
| CN104252415B (en) | Method and system for redistributing data | |
| CN103488772A (en) | Method, system and equipment for caching files through external storage | |
| CN108958657A (en) | A kind of date storage method, storage equipment and storage system | |
| TWI710893B (en) | Data storage device and control method for non-volatile memory | |
| KR101056909B1 (en) | Flash memory device and memory management method in flash memory device | |
| CN113721839B (en) | Computing system and storage hierarchy method for processing graph data | |
| CN111324307A (en) | Satellite-borne NAND FLASH type solid-state memory and method for distributing storage space |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |