CN104679883A - Method for rapidly archiving data and reducing storage space by utilizing partition replacement - Google Patents
Method for rapidly archiving data and reducing storage space by utilizing partition replacement Download PDFInfo
- Publication number
- CN104679883A CN104679883A CN201510113968.XA CN201510113968A CN104679883A CN 104679883 A CN104679883 A CN 104679883A CN 201510113968 A CN201510113968 A CN 201510113968A CN 104679883 A CN104679883 A CN 104679883A
- Authority
- CN
- China
- Prior art keywords
- data
- subregion
- space
- swap
- file
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to the technical field of data storage, in particular to a method for rapidly archiving data and reducing storage space by utilizing partition replacement. The invention fully utilizes the data rapid filing scheme of the database table partition exchange technology and the table space transmission technology aiming at the characteristics of low efficiency, long time consumption and the like of the data partition storage of the large data volume data table in the conventional data filing mode, and more perfectly solves the problems of rapid filing of the large data volume data table and effective reduction of the storage space.
Description
Technical field
The present invention relates to technical field of data storage, particularly a kind ofly utilize subregion to replace to realize data and file the method for falling storage space fast.
Background technology
In today of information Industry Quick Development, information data has penetrated into each industry current and operation function field, becomes the important factor of production.Along with the arrival of large data age, in the database of the production information system of every profession and trade, ubiquity the tables of data with mass data.
The tables of data of these big data quantities has features:
1) mode of employing table subregion stores the business datum of magnanimity, subregion condition is usually based on date, employing table subregion, except can significantly improving the manageability of data, query performance and availability, can also be conducive to business datum and regularly file process;
2) mass data table is huge owing to taking storage space, need often to carry out data filing, to discharge storage space in time, reduce the storage pressure of database, but when adopting usual manner to carry out data filing to Volume data table, there is the problem of inefficiency length consuming time, when the tables of data (adopt table partitioned mode store) of such as oracle database export order to TB rank is filed, need cost just can complete for about 2 hours, during data filing, the performance of database itself is posed a big pressure simultaneously.
Therefore, current in the urgent need to a kind of brand-new data filing mechanism, solve Volume data table and adopt routine data archiving method inefficiency length consuming time, effectively can not reduce a difficult problem for storage space.
Summary of the invention
In order to solve the problem of prior art, the invention provides and a kind ofly utilize subregion to replace to realize data and file the method for falling storage space fast, it is applicable to the Volume data table adopting partitioned storage, can realize the storage space that data are filed fast and effectively reduced tables of data.
The technical solution adopted in the present invention is as follows:
Utilize subregion displacement to realize data and file a method of falling storage space fast, comprise the following steps:
A: create the table space being used for exchanges data;
B: create swap table on the table space created;
C, the corresponding subregion place table space of the traffic table needing to carry out archival back-up is set to read only pattern;
D, the table space being used for exchanging also is set to read only pattern;
E, table subregion and the swap table of archival back-up will be needed in traffic table, carry out subregion exchange, the storage space location swap of table subregion and swap table after exchanging;
F, by exchange after the table space that store partition data, with export order, back up out by transportable tablespace pattern;
G, the table space at swap table place is set to read write pattern, the normal use in recovery table space;
H, filed for traffic table subregion place table space is re-set as read write pattern, recovers normally to use;
I, the dmp file of being derived by filing and data file to be copied, to move on other alternate devices filing and preserve.
In step B, the structure of swap table is completely the same with the structure of the traffic table that will exchange, and comprise literary name section and table index, swap table can only be structurally common table.
In step I, if traffic table does not want the subregion retaining archival back-up, directly can delete subregion.
The present invention is directed to the tables of data of big data quantity, the routine data archiving method inefficiency length consuming time faced, effectively can not reduce a difficult problem for storage space fast, in conjunction with the employing table partitioned mode storage usually of Volume data table, and the feature using the date as subregion condition, provide a kind of utilize database table subregion switching technology and transportable tablespace technology realize the mechanism that data file fast, by spatially creating at free list with the on all four swap table of the structure of large data volume table, (swap table can only be common table, can not be partition table), then Oracle is utilized to show subregion switching technology, the a certain table subregion of large data volume table and swap table are carried out subregion exchange, data after having exchanged under this table subregion are for empty, and under swap table, be the legacy data of table subregion, whole subregion exchange process is exceedingly fast (level second).In like manner, if want to carry out subregion exchange to multiple table subregions of large data volume table, just need to be pre-created multiple swap table, accomplish one_to_one corresponding, after all subregions have exchanged, utilized by the table space at swap table place the transmission table space technology of Oracle to back up derivation at a high speed, its data backup speed is equal to the speed of physical file copy, is only limited to the bottleneck of system I/O.As can be seen here, the employing usual manner that this mechanism perfectly solves Volume data table cannot be filed fast and effectively reduce a difficult problem for storage space.
The present invention utilizes subregion to replace and realizes exchanges data, thus realizes data and file fast, and it utilizes table space physical transfer to realize the quick derivation of table space data, thus the quick backup realizing table space data is derived.Volume data table stores data with partitioned mode, using the date as subregion condition.Before carrying out subregion displacement, need to set up swap table space, and spatially create swap table at swap table, the structure of swap table must be consistent with data list structure.
The beneficial effect that the technical scheme that the embodiment of the present invention provides is brought is:
The present invention, by adopting the measure such as database table subregion switching technology and table space physical transmission techniques, effectively can solve large data volume table and adopt a usual manner filing inefficiency longer difficult problem consuming time.
Notebook data filing mechanism, for the filing inefficiency problem of large data volume table, utilizes the advantage of oracle database table subregion switching technology, transportable tablespace technology, realizes filing fast and effectively reduces storage space of database.
Data filing based on subregion switching technology and transportable tablespace technology is machine-processed, and the advantage had is:
1) utilize table subregion switching technology, the quick exchange that can realize partition data is derived, and data filing process is exceedingly fast, more efficiently than the usual manner being deleted data by such as order line, does not almost affect database performance.
2) utilizing transportable tablespace technology, can realize the quick backup of table space data, by deriving the data file of dmp file and table space, backing up by file copy mode, backup rate is fast, wide adaptability.
3) the fast quick-recovery of filing data can be realized, first by filed dmp file and table space data file, copy in Database Systems, then transportable tablespace technology is utilized, realize the fast quick-recovery of table space, and then the filed table subregion in the data of the table space recovered and traffic table is carried out subregion exchange, just can realize the fast quick-recovery of filing data, date restoring speed is quickly.
Accompanying drawing explanation
In order to be illustrated more clearly in the technical scheme in the embodiment of the present invention, below the accompanying drawing used required in describing embodiment is briefly described, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is schematic diagram before table subregion of the present invention exchanges.
Fig. 2 is that table subregion of the present invention exchanges schematic diagram.
Fig. 3 is schematic diagram after table subregion of the present invention exchanges.
Fig. 4 is concrete treatment step process flow diagram of the present invention.
Fig. 5 is that the data partition of the performance management system that the present invention is directed to communication network exchanges the embodiment of filing fast.
Embodiment
For making the object, technical solutions and advantages of the present invention clearly, below in conjunction with accompanying drawing, embodiment of the present invention is described further in detail.
Embodiment one
The present embodiment is for solving its technical matters, the technical scheme adopted is as Fig. 1, shown in Fig. 2 and Fig. 3, traffic table is partition table, if want by the data backup under certain subregion of traffic table out, need spatially to create swap table on all four with the structure of traffic table at swap table in advance, then by swap table space, table subregion place table space arranges read only pattern, carry out table subregion again to exchange, the table storage pointer of subregion and the storage pointer of swap table are exchanged, thus reach the object of data interchange, the data of such table subregion are just transformed into swap table and have descended, as can be seen from Figure 3, the net result that this subregion exchanges is the location swap in the table storage list space of subregion and the storage list space of swap table, store the table space of table partition data, exchange to swap table to have descended, these comprise the table space of filing data, meet the requirement of transportable tablespace completely, can according to transmission table space manner filing process, after filing, to will be set to the table space of read only pattern, be re-set as read write pattern.
Concrete execution step as shown in Figure 4, is described as follows:
Step 101: create and be used for the table space of exchanges data, very little (being no more than 10M) that the initial memory of table space can be arranged so that arrive exchange after the total storage space of database is reduced object.
Step 102: the table space created in 101 steps creates swap table, the structure of swap table is completely the same with the structure of the traffic table that will exchange, and comprise literary name section and table index etc., swap table can only be structurally common table, can not be partition table.
Step 103: the corresponding subregion place table space of the traffic table needing to carry out archival back-up being set to read only pattern, preventing carrying out between subregion commutation period, due to the renewal to partition data, causes the situation of swap fault to occur.
Step 104: the table space being used for exchanging also is set to read only pattern, prevents subregion from exchanging because Data Update leads to the failure.
Step 105: table subregion and the swap table of archival back-up will be needed in traffic table, and carry out subregion exchange, the storage space location swap of table subregion and swap table after exchanging.
Step 106: by the table space that store partition data after exchange, with export order, back up out by transportable tablespace pattern.
Step 107: the table space at swap table place is set to read write pattern, the normal use in recovery table space.
Step 108: filed for traffic table subregion place table space is re-set as read write pattern, recovers normally to use.
Step 109: the dmp file of being derived by filing and data file are copied, move to filing on other alternate devices to preserve, in addition, if traffic table does not want the subregion retaining archival back-up, directly can delete subregion, to save the storage space of database.
For the performance management system of field of telecommunications Integrated Network Management (PM system), embody rule of the present invention is described.Performance management system is the management system based on the network load management for carrying communication network the whole network.Organize content comprises the flux and flow direction management of network load, the efficiency management of the network operation, the configuration service condition management of Internet resources, the analysis of causes etc. of call failure, relate to the collection of the business datum of magnanimity, gather, store and the link such as inquiry, wherein, performance management system is to the storage of business datum, employing table partitioned mode stores, subregion condition is the business date, these business datums need regularly to file simultaneously, to reduce the storage space taken, meet subregion completely and replace the feature realizing data and file fast.
As shown in Figure 5, the embodiment for the quick filing of data partition exchange of the performance management system of communication network is as follows:
Step 201: before enforcement subregion replaces quick archival solution, first want ready work, create the swap table space being used for exchanges data in the database of operation system, this table space need not be very large; Create archive file storing directory, to be left concentratedly by archive file and to manage.
Step 202: spatially create swap table at swap table, the data structure of swap table is completely the same with the structure of the traffic table that will file, and has how many traffic table, just creates how many swap tables, accomplish one_to_one corresponding.
Step 203: dispose filing procedure in Database Systems, can by the process implementation automatic business processing of data filing by filing procedure, the parameter such as simultaneously configure that filing procedure needs to use swap table space, swap table, traffic table, archive file save contents.
Step 204: the executive plan arranging filing procedure, can the carrying out of property performance period call filing procedure, realize filing fast the robotization of business datum.
By embodiment above, just complete utilizing subregion to replace to realize the enforcement that data file fast, once set the executive plan of filing procedure, then can realizing the automated execution of filing fast, without the need to manually take part in.
The foregoing is only preferred embodiment of the present invention, not in order to limit the present invention, within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (3)
1. utilize subregion displacement to realize data and file a method of falling storage space fast, comprise the following steps:
A: create the table space being used for exchanges data;
B: create swap table on the table space created;
C, the corresponding subregion place table space of the traffic table needing to carry out archival back-up is set to read only pattern;
D, the table space being used for exchanging also is set to read only pattern;
E, table subregion and the swap table of archival back-up will be needed in traffic table, carry out subregion exchange, the storage space location swap of table subregion and swap table after exchanging;
F, by exchange after the table space that store partition data, with export order, back up out by transportable tablespace pattern;
G, the table space at swap table place is set to read write pattern, the normal use in recovery table space;
H, filed for traffic table subregion place table space is re-set as read write pattern, recovers normally to use;
I, the dmp file of being derived by filing and data file to be copied, to move on other alternate devices filing and preserve.
2. according to claim 1ly a kind ofly utilize subregion to replace to realize data and file the method for falling storage space fast, it is characterized in that, in described step B, the structure of swap table is completely the same with the structure of the traffic table that will exchange, comprise literary name section and table index, swap table can only be structurally common table.
3. according to claim 1ly a kind ofly utilize subregion to replace to realize data and file the method for falling storage space fast, it is characterized in that, in described step I, if traffic table does not want the subregion retaining archival back-up, directly can delete subregion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510113968.XA CN104679883A (en) | 2015-03-17 | 2015-03-17 | Method for rapidly archiving data and reducing storage space by utilizing partition replacement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510113968.XA CN104679883A (en) | 2015-03-17 | 2015-03-17 | Method for rapidly archiving data and reducing storage space by utilizing partition replacement |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104679883A true CN104679883A (en) | 2015-06-03 |
Family
ID=53314925
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510113968.XA Pending CN104679883A (en) | 2015-03-17 | 2015-03-17 | Method for rapidly archiving data and reducing storage space by utilizing partition replacement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104679883A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109901965A (en) * | 2017-12-08 | 2019-06-18 | 英业达科技有限公司 | Storage resource processing system and its method |
| CN110399354A (en) * | 2019-07-29 | 2019-11-01 | 锐捷网络股份有限公司 | The subregion of database exchanges method and device |
| CN114925402A (en) * | 2022-05-17 | 2022-08-19 | 江苏方天电力技术有限公司 | Electric power marketing data archiving and integrity protection method |
| CN115391307A (en) * | 2022-07-27 | 2022-11-25 | 上海乾臻信息科技有限公司 | Database optimization method, device, electronic device, and computer-readable storage medium |
| CN119938412A (en) * | 2025-04-08 | 2025-05-06 | 数翊科技(北京)有限公司武汉分公司 | A table partition hot unloading and hot loading method based on openGauss |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101404667A (en) * | 2007-10-05 | 2009-04-08 | 普罗斯特系统公司 | Methods for implementation of WORM enforcement in a storage system |
| US20130024427A1 (en) * | 2010-07-20 | 2013-01-24 | Zte Corporation | Method and Apparatus for Recovering Database Backup |
| CN103106271A (en) * | 2013-02-05 | 2013-05-15 | 广东全通教育股份有限公司 | Database backup and recovery method and system based on mass data |
| CN103150517A (en) * | 2013-02-06 | 2013-06-12 | 杭州电子科技大学 | Secure secret electronic file storage and archiving method and method for checking matching of user permission and file open permission |
| US8484163B1 (en) * | 2010-12-16 | 2013-07-09 | Netapp, Inc. | Cluster configuration backup and recovery |
-
2015
- 2015-03-17 CN CN201510113968.XA patent/CN104679883A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101404667A (en) * | 2007-10-05 | 2009-04-08 | 普罗斯特系统公司 | Methods for implementation of WORM enforcement in a storage system |
| US20130024427A1 (en) * | 2010-07-20 | 2013-01-24 | Zte Corporation | Method and Apparatus for Recovering Database Backup |
| US8484163B1 (en) * | 2010-12-16 | 2013-07-09 | Netapp, Inc. | Cluster configuration backup and recovery |
| CN103106271A (en) * | 2013-02-05 | 2013-05-15 | 广东全通教育股份有限公司 | Database backup and recovery method and system based on mass data |
| CN103150517A (en) * | 2013-02-06 | 2013-06-12 | 杭州电子科技大学 | Secure secret electronic file storage and archiving method and method for checking matching of user permission and file open permission |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109901965A (en) * | 2017-12-08 | 2019-06-18 | 英业达科技有限公司 | Storage resource processing system and its method |
| CN110399354A (en) * | 2019-07-29 | 2019-11-01 | 锐捷网络股份有限公司 | The subregion of database exchanges method and device |
| CN110399354B (en) * | 2019-07-29 | 2022-03-18 | 锐捷网络股份有限公司 | Partition exchange method and device for database |
| CN114925402A (en) * | 2022-05-17 | 2022-08-19 | 江苏方天电力技术有限公司 | Electric power marketing data archiving and integrity protection method |
| CN115391307A (en) * | 2022-07-27 | 2022-11-25 | 上海乾臻信息科技有限公司 | Database optimization method, device, electronic device, and computer-readable storage medium |
| CN119938412A (en) * | 2025-04-08 | 2025-05-06 | 数翊科技(北京)有限公司武汉分公司 | A table partition hot unloading and hot loading method based on openGauss |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109493076B (en) | Kafka message unique consumption method, system, server and storage medium | |
| CN104679772B (en) | Method, apparatus, equipment and the system of file are deleted in Distributed Data Warehouse | |
| CN102880531B (en) | Database backup system and backup method and slave database server of database backup system | |
| CN103116661B (en) | A kind of data processing method of database | |
| EP2998881A1 (en) | A computer implemented method for dynamic sharding | |
| CN104679883A (en) | Method for rapidly archiving data and reducing storage space by utilizing partition replacement | |
| CN102035865B (en) | Data storing and addressing methods, systems and equipment | |
| CN104462185B (en) | A kind of digital library's cloud storage system based on mixed structure | |
| CN102750283A (en) | Massive data synchronization system and method | |
| US11221785B2 (en) | Managing replication state for deleted objects | |
| KR102038529B1 (en) | System for processing real-time data modification of in-memory database | |
| CN105512240A (en) | Data copying method and system based on different database | |
| CN103473239A (en) | Method and device for updating data of non relational database | |
| US10628298B1 (en) | Resumable garbage collection | |
| CN101692226A (en) | Storage method of mass filing stream data | |
| US12259848B2 (en) | Synchronous object placement for information lifecycle management | |
| CN101916290A (en) | In-memory database management method and device | |
| CN104424052A (en) | Automatic redundant distributed storage system and method | |
| US11175993B2 (en) | Managing data storage system | |
| CN111143113A (en) | Method, electronic device and computer program product for copying metadata | |
| US10915513B2 (en) | Archival of data in a relational database management system using block level copy | |
| CN114063883B (en) | Data storage method, electronic device and computer program product | |
| US10289685B2 (en) | Information lifecycle governance | |
| US9483200B2 (en) | System and method for optical cold storage wherein plurality of first and second chunks are encoded and placed on different optical disks | |
| CN103399943A (en) | Communication method and communication device for parallel query of clustered databases |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20150603 |
|
| WD01 | Invention patent application deemed withdrawn after publication |