CN103955413A - File snapshot reduction method based on file local modification - Google Patents
File snapshot reduction method based on file local modification Download PDFInfo
- Publication number
- CN103955413A CN103955413A CN201410150960.6A CN201410150960A CN103955413A CN 103955413 A CN103955413 A CN 103955413A CN 201410150960 A CN201410150960 A CN 201410150960A CN 103955413 A CN103955413 A CN 103955413A
- Authority
- CN
- China
- Prior art keywords
- file
- snapshot
- source document
- paging
- page
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a snapshot reduction method based on file local modification and relates to the technical field of disk reduction. The snapshot reduction method comprises the following steps: paging a to-be-snapshot primary file so as to divide the primary file into multiple pages with uniform size; paging a temporary change file with page size consistent with that of the primary file; when a write operation on the pages of the primary file is performed, creating a new page in the temporary change file, turning the write operation of the primary file to an offset position in a corresponding page in the new page of the temporary change file; creating an memory list structure, connecting deviations in change pages in the temporary change file in series, and combining with the corresponding pages of the primary files so as to form a virtual snapshot file. The file snapshot reduction method based on file local modification has the beneficial effects that a paging mode is adopted for performing snapshot, so that the addressing speed is improved; an indexing relation is established, so that the memory consumption is reduced; snapshot file copy is reduced, data redundancy is reduced, and the snapshot processing speed based on file local modification is improved.
Description
Technical field
The present invention relates to a kind of computer documents storage backup technology, particularly a kind of based on the local File Snapshot method of reducing of revising of file.
Background technology
Along with cloud computing, in the world universal of the cloud memory technology that particularly cloud computing relates to, a lot of aspects of daily life work all relate to cloud memory technology.The requirement of the SAN storage area network technology therefore cloud memory technology being related to is also more and more higher, SAN storage area network plays an important role in cloud storage as architecture, storage area network often need to do snapshot by virtual disk at time point arbitrarily, virtual disk involves in row Disk Backup restoring operation fast.
When disk carries out snapshot, traditional method is copy-on-write mode (Copy-On-Write), and this copy-on-write mode need to copy data integrity to temporary needs complete foundation in internal memory by parent page.Once the huge change of snapshot, can cause larger resource consumption to Installed System Memory, finally affect system response time.More serious meeting causes the load too high of system and produces abnormal collapse.
As shown in Figure 1, the ultimate principle of traditional disk snapshot is when source document starts to occur to revise, source document slavish copying is a, basis as snapshot document, then all new changes of file are all created on snapshot document, such as content modification, file content growth, file content reduction etc. all operate completely on snapshot document.After all operations of snapshot document is completed, for snapshot document, can there be two kinds of modes of operation: " merging " operation, snapshot document is replaced to source document, data are just upgraded completely like this; " abandon " operation, snapshot document is deleted, all the same as never occurring to the file change that source document is all, original file content is recovered reservation.Traditional File Snapshot needs first source document to be copied, then on the file copying, carry out retouching operation, again the snapshot document replacement source document of retouching operation or deletion snapshot document are retained to source document, cause like this data redundancy, take storage space large.
In sum, be necessary to provide a kind of consumption with disk space to alleviate memory headroom consumption, based on the local snapshot method of reducing of revising of file, can reduce data redundancy, improve the local processing speed of revising snapshot of file.
Summary of the invention
In view of this, the object of the present invention is to provide a kind of snapshot method of reducing based on the local modification of file, reduce copying of source document, reduce data redundancy, improve the processing speed of the local modification of file snapshot.
The present invention solves the problems of the technologies described above by following technological means:
The present invention includes a kind of snapshot method of reducing based on the local modification of file, specifically comprise the following steps:
Treat the source document of doing snapshot and carry out paging, be divided into a plurality of pages of the same size;
Interim change file is carried out to paging, and the size of paging is consistent with the page size of described source document;
When the page of described source document is carried out to write operation, in described interim change file, create a new page, the write operation of described source document is redirect to the corresponding page bias internal position in the new page of described interim change file;
Create internal memory list structure, change page bias internal in described interim change file is together in series, the page corresponding to source document merges becomes virtual snapshot file;
When needs are modified to described original file data, described virtual snapshot file is carried out to union operation;
Maybe, when needs return to described original file content, described virtual snapshot file is abandoned to operation.
Further, described in treat the source document of doing snapshot and carry out paging, be divided into the data block that each page is 1024 bytes;
Further, described in treat the source document of doing snapshot and carry out paging, be divided into the data block that each page is 2048 bytes;
Further, the described step that virtual snapshot file is merged is:
Find the interim change file paging corresponding with described source document paging;
Find after the interim change file paging corresponding with described source document paging, with described interim change file, cover described source document;
Described interim change file covers after described source document, empties described interim change file, and the part of described source document is revised and replaced described source document;
Further, the described step that virtual snapshot file is abandoned is:
Find the interim change file paging corresponding with described source document paging;
Find after the interim change file paging corresponding with described source document paging, the page bias internal internal storage structure of described interim change file is deleted one by one;
Delete described interim change file, the destructing of virtual snapshot file, the local modification of described source document lost, and described source document reverts to original contents.
Beneficial effect of the present invention: the present invention adopts the consumption of disk space to alleviate memory headroom consumption, adopts paging mode to carry out snapshot and can improve addressing speed, set up index relative minimizing memory consumption.The present invention has reduced reading and copying of snapshot document, reduces data redundancy, and internal memory operation replaces disk operating, has improved the local processing speed of revising snapshot of file.
Accompanying drawing explanation
Below in conjunction with drawings and Examples, the invention will be further described.
Fig. 1 is traditional magnetic disk snapshot principle schematic;
Fig. 2 is the principle schematic of snapshot reduction of the present invention;
Fig. 3 is workflow diagram of the present invention.
Embodiment
Below with reference to accompanying drawing, the present invention is described in detail, as shown in Figure 2:
Of the present invention a kind of based on the local snapshot method of reducing of revising of file, its ultimate principle is when source document will occur to revise, all changes are write in an interim change file, utilize internal storage data structure that the addition that temporarily changes file and source document " virtual " is set up to snapshot, to system, form the abstract file of a virtual snapshot file.Such as content modification, file content growth, file content reduction etc. all operate completely in interim change file and internal storage data structure, can there is not to source document the change operation of any reality.After snapshot document all operations is completed, for snapshot document, there are two kinds of modes of operation: in the time need to modifying to the data of source document, carry out " merging " operation, by source document and interim change file realistic at the auxiliary lower virtual snapshot file of internal storage data structure be actual file, data are with regard to renewal completely; When needs return to original file content, carry out " abandoning " operation, only need to change file and internal storage data structure is deleted temporarily, all file change of source document is all the same as never occurring, and original file content is recovered reservation.
As shown in Figure 3, based on the local snapshot method of reducing of revising of file, specifically comprise the following steps:
Will make the source document of snapshot with a certain fixed size paging, each page is the data block of 1024 bytes or 2048 byte-sized, changes file temporarily and also adopts the paging consistent with source document.Interim change file paging mode allocation space, in the arbitrary concrete paging of interim change file, to changing and write incremental data by page bias internal mode in the page of source document.Internal storage data structure will change file paging incremental data temporarily and change skew merging mutually with the corresponding source document page, offer the visual field of a complete virtual snapshot file of upper strata snapshot read-write program.The effect of paging is to be convenient to set up the index corresponding relation between the page number of source document writing position and the page number of interim change file.
When certain page of source document produces write operation, just in new page of interim change document creation, by the corresponding page bias internal position write operation of source document being redirect in interim change file new page.
With interim fluctuation zone Jian Wei unit, be change page creation internal memory list structure, will change file paging incremental data temporarily and change skew merging mutually with the corresponding source document page, the page corresponding to source document is merged into virtual snapshot file.
In the time need to modifying to the data of source document, " merging " operation occurs, and the source document paging of take is foundation with corresponding interim change file paging virtual snapshot file, and virtual snapshot file covers source document, and empty interim change file, virtual snapshot Piece file mergence end of operation.
When needs return to original file content, " abandoning " operation occurs, by source document paging, be clue with corresponding interim change file Paged Memory index corresponding relation, the page bias internal internal storage structure that temporarily changes file is deleted one by one, finally delete interim change file, virtual snapshot file destructing, the local modification of source document lost, reduction original contents.
Finally explanation is, above embodiment is only unrestricted in order to technical scheme of the present invention to be described, although the present invention is had been described in detail with reference to preferred embodiment, those of ordinary skill in the art is to be understood that, can modify or be equal to replacement technical scheme of the present invention, and not departing from aim and the scope of technical solution of the present invention, it all should be encompassed in the middle of claim scope of the present invention.
Claims (5)
1. based on the local snapshot method of reducing of revising of file, it is characterized in that: comprise the following steps:
Treat the source document of doing snapshot and carry out paging, be divided into a plurality of pages of the same size;
Interim change file is carried out to paging, and the size of paging is consistent with the page size of described source document;
When the page of described source document is carried out to write operation, in described interim change file, create a new page, the write operation of described source document is redirect to the corresponding page bias internal position in the new page of described interim change file;
Create internal memory list structure, change page bias internal in described interim change file is together in series, the page corresponding to source document merges becomes virtual snapshot file;
When needs are modified to described original file data, described virtual snapshot file is carried out to union operation;
Maybe, when needs return to described original file content, described virtual snapshot file is abandoned to operation.
2. as claimed in claim 1 based on the local snapshot method of reducing of revising of file, it is characterized in that: described in treat the source document of doing snapshot and carry out paging, be divided into the data block that each page is 1024 bytes.
3. as claimed in claim 1 based on the local snapshot method of reducing of revising of file, it is characterized in that: described in treat the source document of doing snapshot and carry out paging, be divided into the data block that each page is 2048 bytes.
4. as claimed in claim 1 based on the local snapshot method of reducing of revising of file, it is characterized in that: the described step that virtual snapshot file is merged is:
Find the interim change file paging corresponding with described source document paging;
Find after the interim change file paging corresponding with described source document paging, with described interim change file, cover described source document;
Described interim change file covers after described source document, empties described interim change file, and the part of described source document is revised and replaced described source document.
5. as claimed in claim 1 based on the local snapshot method of reducing of revising of file, it is characterized in that: the described step that virtual snapshot file is abandoned is:
Find the interim change file paging corresponding with described source document paging;
Find after the interim change file paging corresponding with described source document paging, the page bias internal internal storage structure of described interim change file is deleted one by one;
Delete described interim change file, the destructing of virtual snapshot file, the local modification of described source document lost, and described source document reverts to original contents.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410150960.6A CN103955413A (en) | 2014-04-11 | 2014-04-11 | File snapshot reduction method based on file local modification |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410150960.6A CN103955413A (en) | 2014-04-11 | 2014-04-11 | File snapshot reduction method based on file local modification |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103955413A true CN103955413A (en) | 2014-07-30 |
Family
ID=51332688
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410150960.6A Pending CN103955413A (en) | 2014-04-11 | 2014-04-11 | File snapshot reduction method based on file local modification |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103955413A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107506344A (en) * | 2017-10-12 | 2017-12-22 | 郑州云海信息技术有限公司 | A kind of file online editing method and apparatus |
| CN112347037A (en) * | 2020-10-13 | 2021-02-09 | 北京天融信网络安全技术有限公司 | Snapshot processing method, device, medium and electronic equipment for storage volume |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101661415A (en) * | 2009-09-21 | 2010-03-03 | 中兴通讯股份有限公司 | Method for memorizing snapshot data and system for memorizing snapshot |
| CN102096613A (en) * | 2010-12-30 | 2011-06-15 | 上海爱数软件有限公司 | Method and device for generating snapshot |
| CN102239480A (en) * | 2008-12-17 | 2011-11-09 | Lsi公司 | Storage system snapshot assisted by ssd technology |
| CN102289510A (en) * | 2011-08-31 | 2011-12-21 | 成都市华为赛门铁克科技有限公司 | Method and device for writing data to production volume |
-
2014
- 2014-04-11 CN CN201410150960.6A patent/CN103955413A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102239480A (en) * | 2008-12-17 | 2011-11-09 | Lsi公司 | Storage system snapshot assisted by ssd technology |
| CN101661415A (en) * | 2009-09-21 | 2010-03-03 | 中兴通讯股份有限公司 | Method for memorizing snapshot data and system for memorizing snapshot |
| CN102096613A (en) * | 2010-12-30 | 2011-06-15 | 上海爱数软件有限公司 | Method and device for generating snapshot |
| CN102289510A (en) * | 2011-08-31 | 2011-12-21 | 成都市华为赛门铁克科技有限公司 | Method and device for writing data to production volume |
Non-Patent Citations (1)
| Title |
|---|
| 来去黑暗中: "《blog.csdn.net/runningya/article/details/9621717》", 29 July 2013 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107506344A (en) * | 2017-10-12 | 2017-12-22 | 郑州云海信息技术有限公司 | A kind of file online editing method and apparatus |
| CN112347037A (en) * | 2020-10-13 | 2021-02-09 | 北京天融信网络安全技术有限公司 | Snapshot processing method, device, medium and electronic equipment for storage volume |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US8108446B1 (en) | Methods and systems for managing deduplicated data using unilateral referencing | |
| CN103970585A (en) | Method and device for creating virtual machine | |
| CN104537050B (en) | A kind of method of the quick establishment file system metadata of batch and data | |
| CN101577735B (en) | Method, device and system for taking over fault metadata server | |
| CN103154950B (en) | Repeated data deleting method and device | |
| US9678966B2 (en) | Swat command and API for atomic swap and trim of LBAs | |
| CN104375877A (en) | A virtual machine online snapshot generation and deletion method | |
| CN107291889A (en) | A kind of date storage method and system | |
| CN102770841A (en) | Method and apparatus for generating minimum boot image | |
| CN103544045A (en) | HDFS-based virtual machine image storage system and construction method thereof | |
| CN105493043A (en) | System and method for virtual machine conversion | |
| CN103713941B (en) | Virtual machine image cloning process and system based on qcow2 forms | |
| US20070233753A1 (en) | Database system management method, database system, database device, and backup program | |
| CN104461685A (en) | Virtual machine processing method and virtual computer system | |
| CN105718530A (en) | File storage system and file storage control method thereof | |
| CN103500089A (en) | Small file storage system suitable for Mapreduce calculation model | |
| CN103761159A (en) | Method and system for processing incremental snapshot | |
| CN103049539A (en) | Method and device for storing file data in file system | |
| KR20150122533A (en) | Method for generating secondary index and apparatus for storing secondary index | |
| CN105045850B (en) | Junk data recovery method in cloud storage log file system | |
| CN104484283A (en) | Method for reducing write amplification of solid state disk | |
| CN102541691A (en) | Log check point recovery method applied to memory data base OLTP (online transaction processing) | |
| CN106445421A (en) | Storage system data migration method and system | |
| CN103793475A (en) | Distributed file system data migration method | |
| CN103034591A (en) | Memory sharing method and device for virtual machine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20140730 |