CN102890677A - Reliable storage method of non-document system - Google Patents
Reliable storage method of non-document system Download PDFInfo
- Publication number
- CN102890677A CN102890677A CN2011102027043A CN201110202704A CN102890677A CN 102890677 A CN102890677 A CN 102890677A CN 2011102027043 A CN2011102027043 A CN 2011102027043A CN 201110202704 A CN201110202704 A CN 201110202704A CN 102890677 A CN102890677 A CN 102890677A
- Authority
- CN
- China
- Prior art keywords
- data
- file
- write
- offset address
- reading
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A reliable storage method of a non-document system comprises a writing process of data, a reading process of the data and a deleting process of the data, wherein in the writing process of the data, a local database is opened by an operator, and an offset address of a current write disk is acquired; then a hard-disk device is opened, and a file pointer is deviated to a write position; when the data is written, a write mark is established in the local database, and the data begins to write; in the reading process of the data, the record of the file in the local database is searched according to a search condition transmitted by an application file, an offset address of the file in the hard-disk record is acquired from the record, data is read from the offset address, and the data is returned to the application layer in a callback form; and in the deleting process of the data, the file in the local database is marked as deleted by the operator.
Description
Technical field
The present invention relates to a kind of data efficient, reliable memory technology, be applied to the industry that Video Surveillance Industry, IPTV etc. need the mass data storage.
Background technology
At present, store mostly all based on file system based on the data of hard disk.From the FAT file system of windows to NTFS, the data non-volatile memory that the hard disk management function that all being based on operating system provides is carried out, ext2/3 that comprises Linux etc. also is like this.Based on the data of file system store brought to our exploitation great easy, but its efficient and reliability are called the focus that people disagree always.This focus has also expedited the emergence of more, better file system.Yet how file system upgrades again, also is the general product based on the general application habit of people.For some special industries, as: video monitoring, IPTV.First characteristics of the data of these industries are exactly huge, and monitoring camera, its size of motion picture film be hundreds of million easily.Second characteristic is exactly that file size is substantially fixing, that is to say: can not mix in the storage data in these industries and store some less files, such as the interim webpage of preserving etc. that can not mix.The 3rd characteristics are wished the file ordered arrangement exactly, and namely file storage can be carried out record according to date or certain fixing arrangement mode in hard disk.The 4th characteristic are exactly reading of file and write very frequently, like this general file are write the great challenge that debris management just brings.
Summary of the invention
The object of the invention is to: A: realize the date storage method based on non-file system; B: the separate, stored that realizes file storage and fileinfo index; C: the basic synchronization that realizes file and file index information.
For achieving the above object, the present invention is by the following technical solutions:
The present invention includes the ablation process of data, the process that reads of data, the delete procedure of data,
In the ablation process of described data, the operator opens local data base, obtains the current offset address that writes disk; Then open hard disc apparatus, file pointer is displaced to writing position; Fashionable when there being data to write, write sign and beginning data writing in local data base foundation;
Reading in the process of described data, according to the search condition that application layer is imported into, retrieve the local database record of file, from record, obtain the offset address of file in hard disk recording, from this offset address, with data reading, return to application layer with the readjustment form;
In the delete procedure of described data, the operator is designated deletion to this document in local data base.
Described local data base adopts Berkeley DB as the prototype data storehouse.
In reading in the process of data, the reading times of these data of local database record.
Adopt the present invention of technique scheme, well that file storage and fileinfo index is independent, improved the efficient of storage.In addition, when being arranged, the informational needs deletion just in local data base, corresponding record is carried out mark, rather than the deletion of the file of physics.This has guaranteed minimum disk is operated in the process, has prolonged the mission life of disk.In the method data are carried out separate, stored, corresponding data message then enters database and carries out record.When the needs read data conducts interviews, at first database is inquired about the disk address that obtains data and directly read.Reduce to greatest extent traversal and search to catalogue, the efficient that provides data to read reduces the unnecessary operation of hard disk.Data are write fashionable, directly write from the magnetic head current location.After data write and finish with the information write into Databasce.Finish writing synchronously of data, be i.e. write synchronously record after data storage is finished data are described.
Description of drawings
Fig. 1 is the process flow diagram that hard disk obtains among the present invention.
Fig. 2 is the process flow diagram that video data obtains among the present invention.
Fig. 3 is the process flow diagram that video data writes among the present invention.
Embodiment
In general, the storing process of non-file system comprises the ablation process of data, the process that reads of data, the delete procedure of data.
Before data are carried out various operations, at first local data base is operated: because in the system, have a large amount of recorded informations that needs real-time storage.It is the prototype data storehouse that the present invention has adopted Berkeley DB, and the information in the system is stored.Relate to opening, close, read, write, retrieving of database.Because this database do not adopt the class SQL database, but abandon the SQL statement support for performance, and then promoted database performance.Berkeley DB is the database of an opening, and its interface is that open, disclosed.
Secondly, carry out opening with pointer of hard disk and locate: hard disk is a physical equipment in operating system, it is shown as/dev/sd** take Linux as the example kernel.This is exactly the equipment that we will operate, if only have a hard disk in the system then for sda.The like be sdb, sdc, sdd, sde ..., at the subregion of each hard disk then be: sda1, sda2, sda3(are take the different subregions of first hard disk as example) etc.The equipment that we need to open is hard disc apparatus rather than certain concrete subregion.Namely we should open sdb, sdc, sdd, sde ... equipment, equipment are opened the offset address of rear this hard disk according to recording in the database, and the operation file pointer of hard disk is offset.Data write the access ready state, wait for that application data arrives.
Such as Fig. 1, shown in Figure 3, the writing of data: when having data to write business, at first open local data base, obtain and currently should write the disk offset address.Then open hard disc apparatus, file pointer is displaced to should writing position.Fashionable when there being data to write, write sign and beginning data writing in local data base foundation.When one piece of data write complete after, deposit complete information in database.As long as just thinking, side existing problems have this record or file and unusually should carry out rollback in the process.
As shown in Figure 2, reading of data: the search condition of importing into according to application layer.Retrieve the data-base recording of file, from record, obtain the offset address of file in hard disk recording.Successively with data reading, return to application layer with the readjustment form from this address.And record these data and read counting, so that statistics file reads probability.For follow-up raising file hit rate provides foundation.
The deletion of data: this process and when reading, database obtains basically identical.Different is that this function is from database file identification to be deletion.Be not directly with physical file deletion, reduce evolutionary operator probability to hard disk, the mission life that has prolonged disk with this.
Claims (3)
1. the reliable storage method of a non-file system, it comprises the ablation process of data, the process that reads of data, the delete procedure of data, it is characterized in that:
In the ablation process of described data, the operator opens local data base, obtains the current offset address that writes disk; Then open hard disc apparatus, file pointer is displaced to writing position; Fashionable when there being data to write, write sign and beginning data writing in local data base foundation;
Reading in the process of described data, according to the search condition that application layer is imported into, retrieve the local database record of file, from record, obtain the offset address of file in hard disk recording, from this offset address, with data reading, return to application layer with the readjustment form;
In the delete procedure of described data, the operator is designated deletion to this document in local data base.
2. the reliable storage method of non-file system according to claim 1 is characterized in that: described local data base adopts Berkeley DB as the prototype data storehouse.
3. the reliable storage method of non-file system according to claim 1 is characterized in that: in reading in the process of data, the reading times of these data of local database record.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011102027043A CN102890677A (en) | 2011-07-20 | 2011-07-20 | Reliable storage method of non-document system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2011102027043A CN102890677A (en) | 2011-07-20 | 2011-07-20 | Reliable storage method of non-document system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102890677A true CN102890677A (en) | 2013-01-23 |
Family
ID=47534184
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2011102027043A Pending CN102890677A (en) | 2011-07-20 | 2011-07-20 | Reliable storage method of non-document system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102890677A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107154013A (en) * | 2016-03-02 | 2017-09-12 | 阿里巴巴集团控股有限公司 | Additional card, content delivery network server and execution method for image procossing |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5408652A (en) * | 1990-08-31 | 1995-04-18 | Fujitsu Limited | Method and apparatus for heterogenous database access by generating different access procedures for different database data structures |
| CN101039278A (en) * | 2007-03-30 | 2007-09-19 | 华为技术有限公司 | Data management method and system |
-
2011
- 2011-07-20 CN CN2011102027043A patent/CN102890677A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5408652A (en) * | 1990-08-31 | 1995-04-18 | Fujitsu Limited | Method and apparatus for heterogenous database access by generating different access procedures for different database data structures |
| CN101039278A (en) * | 2007-03-30 | 2007-09-19 | 华为技术有限公司 | Data management method and system |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107154013A (en) * | 2016-03-02 | 2017-09-12 | 阿里巴巴集团控股有限公司 | Additional card, content delivery network server and execution method for image procossing |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103080896B (en) | Always searched to reduce the time to tape-shaped medium's to access rearrangement | |
| CN100512409C (en) | Data processing apparatus and data processing method | |
| US9348517B2 (en) | Using a migration threshold and a candidate list for cache management of sequential write storage | |
| US8898428B2 (en) | Mapping locations of logical volume records on a physical stacked volume | |
| US11250888B1 (en) | Flash memory and method for storing and retrieving embedded audio video data | |
| US10621065B2 (en) | Concurrent logging of data layers within a tape storage device | |
| CN111651127B (en) | Monitoring data storage method and device based on shingled magnetic recording disk | |
| CN104281651A (en) | Method and system for searching large volume of video data | |
| US10346368B2 (en) | Method and apparatus of per-block-group journaling for ordered mode journaling file system | |
| US10838944B2 (en) | System and method for maintaining a multi-level data structure | |
| KR101674176B1 (en) | Method and apparatus for fsync system call processing using ordered mode journaling with file unit | |
| US20150149415A1 (en) | Rollback Mechanism for Linear Tape File Systems | |
| US10423580B2 (en) | Storage and compression of an aggregation file | |
| CN105068761B (en) | A kind of video interception storage method and system convenient for retrieval | |
| US9851927B2 (en) | File management in a file system | |
| US8656094B2 (en) | Locating host data records on a physical stacked volume | |
| CN113553325B (en) | A method and system for synchronizing aggregate objects in an object storage system | |
| CN102890677A (en) | Reliable storage method of non-document system | |
| CN107621924A (en) | A kind of method and apparatus based on high in the clouds cleaning solid state hard disc data | |
| US11100056B2 (en) | Life cycle data set repository | |
| US10769125B2 (en) | Ordering records for timed meta-data generation in a blocked record environment | |
| CN108140047B (en) | Data processing apparatus and method, and data container structure | |
| JP6390196B2 (en) | Storage system, storage method, and program | |
| US10388304B1 (en) | Storing data on unused areas of a storage medium | |
| CN110196785A (en) | Backup data management method, apparatus and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20130123 |