[go: up one dir, main page]

CN108090095B - Method and device for reconstructing database in batches - Google Patents

Method and device for reconstructing database in batches Download PDF

Info

Publication number
CN108090095B
CN108090095B CN201611044272.7A CN201611044272A CN108090095B CN 108090095 B CN108090095 B CN 108090095B CN 201611044272 A CN201611044272 A CN 201611044272A CN 108090095 B CN108090095 B CN 108090095B
Authority
CN
China
Prior art keywords
database
information
server
rebuilt
size
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201611044272.7A
Other languages
Chinese (zh)
Other versions
CN108090095A (en
Inventor
赵春猛
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Gridsum Technology Co Ltd
Original Assignee
Beijing Gridsum Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Gridsum Technology Co Ltd filed Critical Beijing Gridsum Technology Co Ltd
Priority to CN201611044272.7A priority Critical patent/CN108090095B/en
Publication of CN108090095A publication Critical patent/CN108090095A/en
Application granted granted Critical
Publication of CN108090095B publication Critical patent/CN108090095B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/22Indexing; Data structures therefor; Storage structures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

本发明公开了一种批量重建数据库的方法和装置。其中,该方法包括:获取分布式部署的一个或多个服务器上的数据库信息,其中,数据库信息至少包括:数据库的名称、数据库中存储的文件的大小、数据库中存储的数据表的总行数;根据数据库中存储的文件的大小和/或表的总行数确定需要重建的数据库;对需要重建的数据库进行重建。本发明解决了现有技术无法实现将分布式部署的多个服务器上的数据库进行批量重建的技术问题。

Figure 201611044272

The invention discloses a method and a device for batch rebuilding a database. Wherein, the method includes: acquiring database information on one or more servers deployed in a distributed manner, wherein the database information at least includes: the name of the database, the size of the file stored in the database, and the total number of rows of the data table stored in the database; Determine the database that needs to be rebuilt according to the size of the file stored in the database and/or the total number of rows in the table; rebuild the database that needs to be rebuilt. The invention solves the technical problem that the prior art cannot realize the batch reconstruction of databases on multiple servers deployed in a distributed manner.

Figure 201611044272

Description

批量重建数据库的方法和装置Method and device for batch rebuilding database

技术领域technical field

本发明涉及计算机网络技术领域,具体而言,涉及一种批量重建数据库的方法和装置。The present invention relates to the technical field of computer networks, in particular, to a method and apparatus for batch rebuilding a database.

背景技术Background technique

在本领域,数据库技术的出现,实现了现代信息科学系统中对大量计算机数据进行统一分析、管理和共享的目的。然而,在将计算机数据存储至数据库(例如,SQL-server数据库)的过程中,通常很难通过一个步骤便将需要进行多维度剖析的数据实现分门别类地存储。为了实现数据库中的数据存储,通常采用的方案是增加一个处理环节,先将计算机获取的原始数据存储到一个过程数据库(例如:receiver数据库)中,然后进一步处理这个过程数据库中存储的数据,从而生成符合剖析条件的最终的数据库(例如:DataWarehouse数据库)。在这个过程中,随着过程数据不断的累积,该过程数据库中存储的数据会一直增加,尤其是存储了很多中间过程数据库-receiver数据库,无疑会给服务器的硬盘造成巨大的压力。In this field, the emergence of database technology has achieved the purpose of unified analysis, management and sharing of large amounts of computer data in modern information science systems. However, in the process of storing computer data in a database (for example, a SQL-server database), it is usually difficult to store the data that needs to be analyzed in multiple dimensions in one step. In order to realize the data storage in the database, the solution usually adopted is to add a processing link, first store the original data obtained by the computer into a process database (for example: receiver database), and then further process the data stored in the process database, thereby Generate the final database (eg DataWarehouse database) that meets the profiling conditions. In this process, with the continuous accumulation of process data, the data stored in the process database will continue to increase, especially the storage of many intermediate process databases-receiver databases will undoubtedly cause huge pressure on the hard disk of the server.

为了避免上述过程数据库中存储的数据持续增加,需要定期采用模板数据库将这类过程数据库进行重建。由于模板数据库中仅仅包含数据库的正确的表结构等信息,没有任何数据,因此数据量和占用磁盘空间都比较小。现有技术采用的方案是通过查看服务器本地磁盘文件大小的方式来获取数据库中存储的数据库文件的大小,从而根据数据库文件大小确定需要是否将数据库重建。对于分布式数据库,由于采用多个分布式部署的服务器来将数据库分布式存储,难以保证数据库中存储的数据库文件存储的路径一致,因而,采用查看服务器本地磁盘文件大小的方式只能实现对单个数据库进行重建,或者利用脚本程序等实现对单个服务器上的数据库进行重建,无法对分布式部署的所有服务器上的多个数据库进行批量重建。In order to avoid the continuous increase of the data stored in the above-mentioned process database, it is necessary to periodically use a template database to rebuild such a process database. Because the template database only contains information such as the correct table structure of the database, without any data, the amount of data and the occupied disk space are relatively small. The solution adopted in the prior art is to obtain the size of the database file stored in the database by checking the file size of the local disk of the server, so as to determine whether the database needs to be rebuilt according to the size of the database file. For a distributed database, since multiple distributed servers are used to store the database in a distributed manner, it is difficult to ensure that the storage paths of the database files stored in the database are consistent. The database is reconstructed, or the database on a single server is reconstructed by using a script program, etc., and multiple databases on all servers deployed in a distributed manner cannot be reconstructed in batches.

针对上述现有技术无法实现将分布式部署的多个服务器上的数据库进行批量重建的问题,目前尚未提出有效的解决方案。Aiming at the problem that the above-mentioned prior art cannot realize batch reconstruction of databases on multiple servers deployed in a distributed manner, no effective solution has been proposed yet.

发明内容SUMMARY OF THE INVENTION

本发明实施例提供了一种批量重建数据库的方法和装置,以至少解决现有技术无法实现将分布式部署的多个服务器上的数据库进行批量重建的技术问题。Embodiments of the present invention provide a method and apparatus for batch rebuilding a database, so as to at least solve the technical problem that the prior art cannot implement batch rebuilding of databases on multiple servers deployed in a distributed manner.

根据本发明实施例的一个方面,提供了一种批量重建数据库的方法,包括:获取分布式部署的一个或多个服务器上的数据库信息,其中,数据库信息至少包括:数据库的名称、数据库中存储的文件的大小、数据库中存储的数据表的总行数;根据数据库中存储的文件的大小和/或表的总行数确定需要重建的数据库;对需要重建的数据库进行重建。According to an aspect of the embodiments of the present invention, there is provided a method for rebuilding a database in batches, including: acquiring database information on one or more servers deployed in a distributed manner, wherein the database information at least includes: the name of the database, the data stored in the database The size of the file and the total number of rows of the data table stored in the database; determine the database to be rebuilt according to the size of the file stored in the database and/or the total number of rows of the table; rebuild the database to be rebuilt.

进一步地,获取分布式部署的一个或多个服务器上的数据库信息,包括:通过数据库管理系统调用服务器上的数据库系统表;从数据库系统表获取数据库信息。Further, acquiring database information on one or more servers deployed in a distributed manner includes: calling a database system table on the server through a database management system; and acquiring database information from the database system table.

进一步地,在获取分布式部署的一个或多个服务器上的数据库信息之后,上述方法还包括:获取分布式部署的一个或多个服务器的标识信息;将标识信息和服务器上的数据库信息存储至数据库临时表中。Further, after acquiring database information on one or more servers deployed in a distributed manner, the above method further includes: acquiring identification information on one or more servers deployed in a distributed manner; storing the identification information and the database information on the servers in a database temporary table.

进一步地,根据数据库中存储的文件的大小和/或数据表的总行数确定需要重建的数据库包括:判断数据库是否满足预设条件,其中,预设条件包括以下任意之一:数据库中存储的文件的大小大于预设阈值、数据库中存储的表的总行数大于预设行数;从数据库临时表中筛选出满足预设条件的数据库。Further, determining the database to be rebuilt according to the size of the file stored in the database and/or the total number of rows in the data table includes: judging whether the database satisfies a preset condition, wherein the preset condition includes any one of the following: the file stored in the database The size of the database is greater than the preset threshold, and the total number of rows of the table stored in the database is greater than the preset number of rows; the database that meets the preset condition is filtered from the database temporary table.

进一步地,对需要重建的数据库进行重建包括:获取数据库临时表中需要重建的数据库;删除需要重建的数据库;利用模板数据库对需要重建的数据库进行重建。Further, rebuilding the database that needs to be rebuilt includes: obtaining the database that needs to be rebuilt in the temporary table of the database; deleting the database that needs to be rebuilt; and using the template database to rebuild the database that needs to be rebuilt.

进一步地,在删除需要重建的数据库之前,方法包括:将需要重建的数据库备份至本地磁盘或备份服务器中。Further, before deleting the database that needs to be rebuilt, the method includes: backing up the database that needs to be rebuilt to a local disk or a backup server.

进一步地,在利用模板数据库对需要重建的数据库进行重建之后,方法还包括:对重建的数据库进行校验;在校验成功的情况下,删除本地磁盘或备份服务器中备份的数据库。Further, after using the template database to rebuild the database to be rebuilt, the method further includes: verifying the rebuilt database; and deleting the database backed up in the local disk or the backup server when the verification is successful.

根据本发明实施例的另一方面,还提供了一种批量重建数据库的装置,包括:第一获取模块,用于获取分布式部署的一个或多个服务器上的数据库信息,其中,数据库信息至少包括:数据库的名称、数据库中存储的文件的大小、数据库中存储的数据表的总行数;确定模块,用于根据数据库中存储的文件的大小和/或数据表的总行数确定需要重建的数据库;创建模块,用于对需要重建的数据库进行重建。According to another aspect of the embodiments of the present invention, an apparatus for rebuilding databases in batches is further provided, including: a first obtaining module configured to obtain database information on one or more servers deployed in a distributed manner, wherein the database information is at least Including: the name of the database, the size of the file stored in the database, the total number of rows of the data table stored in the database; the determining module is used to determine the database to be rebuilt according to the size of the file stored in the database and/or the total number of rows of the data table ; Create a module for rebuilding the database that needs to be rebuilt.

进一步地,上述第一获取模块包括:调用模块,用于通过数据库管理系统调用服务器上的数据库系统表;第二获取模块,用于从数据库系统表获取数据库信息。Further, the above-mentioned first obtaining module includes: a calling module for calling the database system table on the server through the database management system; and a second obtaining module for obtaining database information from the database system table.

进一步地,上述装置还包括:第三获取模块,用于获取分布式部署的一个或多个服务器的标识信息;第一存储模块,用于将标识信息和服务器上的数据库信息存储至数据库临时表中。Further, the above-mentioned device also includes: a third acquisition module for acquiring the identification information of one or more servers deployed in a distributed manner; a first storage module for storing the identification information and database information on the server in a temporary database table middle.

进一步地,上述确定模块还包括:判断数据库是否满足预设条件,其中,预设条件至少包括以下任意之一:数据库中存储的文件的大小大于预设阈值、数据库中存储的表的总行数大于预设行数;从数据库临时表中筛选出满足预设条件的数据库。Further, the above-mentioned determination module further includes: judging whether the database satisfies a preset condition, wherein the preset condition includes at least any one of the following: the size of the file stored in the database is greater than the preset threshold, and the total number of rows of the table stored in the database is greater than The preset number of rows; the database that meets the preset conditions is filtered from the database temporary table.

进一步地,上述创建模块还包括:第四获取模块,用于获取数据库临时表中需要重建的数据库;第一删除模块,用于删除需要重建的数据库;处理模块,用于利用模板数据库对需要重建的数据库进行重建。Further, the above-mentioned creation module also includes: a fourth acquisition module, for acquiring the database that needs to be rebuilt in the temporary table of the database; the first deletion module, for deleting the database that needs to be rebuilt; The database is rebuilt.

进一步地,上述装置还包括:第二存储模块,用于将需要重建的数据库备份至本地磁盘或备份服务器中。Further, the above-mentioned apparatus further includes: a second storage module for backing up the database that needs to be rebuilt to a local disk or a backup server.

进一步地,上述装置还包括:校验模块,用于对重建的数据库进行校验;第二删除模块,用于在校验成功的情况下,删除本地磁盘或备份服务器中备份的数据库。Further, the above device further includes: a verification module for verifying the reconstructed database; a second deletion module for deleting the database backed up in the local disk or the backup server when the verification is successful.

在本发明实施例中,通过获取分布式部署的一个或多个服务器上的数据库信息,其中,数据库信息至少包括:数据库的名称、数据库中存储的文件的大小、数据库中存储的数据表的总行数;根据数据库中存储的文件的大小和/或表的总行数确定需要重建的数据库;对需要重建的数据库进行重建,达到了对分布式部署的一个或多个服务器上的数据库进行批量处理的目的,从而实现了对不同服务器上的数据库进行批量重建的技术效果,进而解决了现有技术无法实现将分布式部署的多个服务器上的数据库进行批量重建的技术问题。In this embodiment of the present invention, by acquiring database information on one or more servers deployed in a distributed manner, the database information at least includes: the name of the database, the size of the file stored in the database, and the total row of the data table stored in the database. determine the database to be rebuilt according to the size of the files stored in the database and/or the total number of rows in the table; rebuild the database that needs to be rebuilt to achieve batch processing of databases on one or more servers deployed in a distributed manner The purpose is to achieve the technical effect of batch reconstruction of databases on different servers, thereby solving the technical problem that the prior art cannot achieve batch reconstruction of databases on multiple servers deployed in a distributed manner.

附图说明Description of drawings

此处所说明的附图用来提供对本发明的进一步理解,构成本申请的一部分,本发明的示意性实施例及其说明用于解释本发明,并不构成对本发明的不当限定。在附图中:The accompanying drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the attached image:

图1是根据本发明实施例的一种批量重建数据库的方法流程图;1 is a flowchart of a method for batch rebuilding a database according to an embodiment of the present invention;

图2是根据本发明实施例的一种可选的批量重建数据库的方法流程图;2 is a flowchart of an optional method for batch rebuilding a database according to an embodiment of the present invention;

图3是根据本发明实施例的一种可选的批量重建数据库的方法流程图;3 is a flowchart of an optional method for batch rebuilding a database according to an embodiment of the present invention;

图4是根据本发明实施例的一种可选的批量重建数据库的方法流程图;4 is a flowchart of an optional method for batch rebuilding a database according to an embodiment of the present invention;

图5是根据本发明实施例的一种可选的批量重建数据库的方法流程图;5 is a flowchart of an optional method for batch rebuilding a database according to an embodiment of the present invention;

图6是根据本发明实施例的一种可选的批量重建数据库的方法流程图;6 is a flowchart of an optional method for batch rebuilding a database according to an embodiment of the present invention;

图7是根据本发明实施例的一种优选的在SQL Server中批量重建数据库的方法流程图;以及7 is a flowchart of a preferred method for batch rebuilding a database in SQL Server according to an embodiment of the present invention; and

图8是根据本发明实施例的一种批量重建数据库的装置示意图。FIG. 8 is a schematic diagram of an apparatus for batch rebuilding a database according to an embodiment of the present invention.

具体实施方式Detailed ways

为了使本技术领域的人员更好地理解本发明方案,下面将结合本发明实施例中的附图,对本发明实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本发明一部分的实施例,而不是全部的实施例。基于本发明中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都应当属于本发明保护的范围。In order to make those skilled in the art better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only Embodiments are part of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

需要说明的是,本发明的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象,而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换,以便这里描述的本发明的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外,术语“包括”和“具有”以及他们的任何变形,意图在于覆盖不排他的包含,例如,包含了一系列步骤或单元的过程、方法、系统、产品或设备不必限于清楚地列出的那些步骤或单元,而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。It should be noted that the terms "first", "second" and the like in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that the data so used may be interchanged under appropriate circumstances such that the embodiments of the invention described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

实施例1Example 1

根据本发明实施例,提供了一种批量重建数据库的方法实施例,需要说明的是,在附图的流程图示出的步骤可以在诸如一组计算机可执行指令的计算机系统中执行,并且,虽然在流程图中示出了逻辑顺序,但是在某些情况下,可以以不同于此处的顺序执行所示出或描述的步骤。According to an embodiment of the present invention, an embodiment of a method for batch rebuilding a database is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings may be executed in a computer system such as a set of computer-executable instructions, and, Although a logical order is shown in the flowcharts, in some cases steps shown or described may be performed in an order different from that herein.

图1是根据本发明实施例的一种批量重建数据库的方法流程图,如图1所示,该方法包括如下步骤:FIG. 1 is a flowchart of a method for batch rebuilding a database according to an embodiment of the present invention. As shown in FIG. 1 , the method includes the following steps:

步骤S102,获取分布式部署的一个或多个服务器上的数据库信息,其中,数据库信息至少包括:数据库的名称、数据库中存储的文件的大小、数据库中存储的数据表的总行数。Step S102: Obtain database information on one or more servers deployed in a distributed manner, where the database information at least includes: the name of the database, the size of the file stored in the database, and the total number of rows of the data table stored in the database.

具体地,在上述步骤中,上述服务器可以是用于对数据库中的数据进行存储与管理服务的数据库服务器;通过互联网连接在同一个域内的分布式部署的一个或多个服务器,可以提供分布式数据存储与管理服务;数据库是由多个数据表构成的,数据表是数据库的实质内容;在对分布式部署的一个或多个服务器上的数据库重建之前,首先要获取这些服务器上的数据库名称、数据库中存储的文件的大小、数据库中存储的数据表信息(例如,数据表的总行数)等。Specifically, in the above steps, the above server may be a database server for storing and managing data in the database; one or more distributed servers connected to the same domain through the Internet can provide distributed Data storage and management service; the database is composed of multiple data tables, and the data table is the essence of the database; before rebuilding the database on one or more servers deployed in a distributed manner, first obtain the database name on these servers , the size of the file stored in the database, the data table information stored in the database (for example, the total number of rows in the data table), etc.

一种可选的实施例中,上述数据库可以为数据库进行数据存储过程中的过程数据库。In an optional embodiment, the above-mentioned database may be a process database in the data storage process of the database.

步骤S104,根据数据库中存储的文件的大小和/或表的总行数确定需要重建的数据库。Step S104: Determine the database to be rebuilt according to the size of the file stored in the database and/or the total number of rows in the table.

具体地,在上述步骤中,上述文件可以为数据库中存储的文件,例如,数据库存储的数据;上述文件的大小可以为数据库中存储的文件所占磁盘空间的大小,可以用于表征数据库中存储的数据量;上述数据表可以为数据库中存放数据的多个表,在获取分布式部署的一个或多个服务器上的数据库信息之后,根据数据库中存储的文件大小和/或数据库中存储的多个表的总行数,来确定需要重建的数据库。Specifically, in the above-mentioned steps, the above-mentioned file may be a file stored in a database, for example, data stored in a database; the size of the above-mentioned file may be the size of the disk space occupied by the file stored in the database, which can be used to represent the storage in the database. The above data table can be a plurality of tables storing data in the database, after obtaining the database information on one or more servers of distributed deployment, according to the file size stored in the database and/or the number of stored data in the database The total number of rows in each table to determine the database that needs to be rebuilt.

此处需要说明的是,数据库中存储的数据量太大会影响数据库的访问效率,而数据库中表的行数会也会影响数据库的响应速度,因此,本发明采用数据库中存储的文件大小和表的总行数两个指标来确定需要重建的数据库,对需要重建的数据库重建,从而提高数据的访问效率和响应速度。It should be noted here that the large amount of data stored in the database will affect the access efficiency of the database, and the number of rows in the table in the database will also affect the response speed of the database. Therefore, the present invention adopts the file size and table stored in the database. There are two indicators of the total number of rows to determine the database that needs to be rebuilt, and rebuild the database that needs to be rebuilt, thereby improving data access efficiency and response speed.

一种优选的实施例中,当数据库中存储的文件的大小和表的总行数同时满足某一条件时,才对该数据库进行重建,从而提高准确度。In a preferred embodiment, the database is reconstructed only when the size of the file stored in the database and the total number of rows of the table satisfy a certain condition, thereby improving the accuracy.

步骤S106,对需要重建的数据库进行重建。Step S106, rebuild the database that needs to be rebuilt.

具体地,在上述步骤中,在根据数据库中存储的文件大小和数据库中存储的多个表的总行数,来确定需要重建的数据库之后,基于模板数据对确定的需要重建的数据库进行重建。Specifically, in the above steps, after the database to be rebuilt is determined according to the file size stored in the database and the total number of rows of multiple tables stored in the database, the determined database to be rebuilt is rebuilt based on template data.

此处需要说明的是,为了避免数据库在长时间存储数据过程中,过程数据库中积累的数据持续增加,不仅占用磁盘空间,而且影响数据库的访问速度,因而,需要定期对采用模板数据来对数据库进行重建,重建的过程是首先将数据库里表的数据存放到备份空间中,删除原有的表和索引,重建空的表和索引后,在将数据导入到新表或索引中,通过数据库重建,可以清除旧的数据,从而减少磁盘的占用空间,提高数据库的访问速度。It should be noted here that in order to avoid the continuous increase of data accumulated in the process database in the process of storing data for a long time in the database, it not only occupies disk space, but also affects the access speed of the database. To rebuild, the process of rebuilding is to first store the data of the table in the database in the backup space, delete the original table and index, rebuild the empty table and index, import the data into the new table or index, and rebuild the database through the database. , which can clear the old data, thereby reducing the space occupied by the disk and improving the access speed of the database.

由上可知,在本申请上述实施例中,通过获取同一个域内的不同服务器上的数据库中存储的文件大小和表的行数,并根据数据库中存储的文件大小和表的行数两个指标来确定需要重建的数据库,最后基于模板数据,将需要重建的数据库进行重建。容易注意,本申请上述实施例公开的方案,未使用本次磁盘获取数据库占用磁盘空间的文件大小,而是从数据库管理系统中获取多个不同服务器上的数据库信息,对不同服务器上数据库的文件大小和表的行数进行分析,来确定需要重建的数据库,达到了对分布式部署的一个或多个服务器上的数据库进行批量处理的目的,从而实现了对不同服务器上的数据库进行批量重建的技术效果,进而解决了现有技术无法实现将分布式部署的多个服务器上的数据库进行批量重建的技术问题。It can be seen from the above that in the above-mentioned embodiment of the present application, by obtaining the file size and the number of rows of the table stored in the databases on different servers in the same domain, and according to the two indicators of the size of the file and the number of rows of the table stored in the database to determine the database that needs to be rebuilt, and finally rebuild the database that needs to be rebuilt based on the template data. It is easy to note that the solutions disclosed in the above-mentioned embodiments of the present application do not use the current disk to obtain the file size of the database occupied disk space, but obtain the database information on multiple different servers from the database management system. The size and the number of rows in the table are analyzed to determine the database that needs to be rebuilt, which achieves the purpose of batch processing the databases on one or more servers deployed in a distributed manner, thereby realizing the batch reconstruction of databases on different servers. The technical effect further solves the technical problem that the existing technology cannot realize the batch reconstruction of the databases on multiple servers deployed in a distributed manner.

作为一种可选的实施例,如图2所示,上述步骤S102中获取分布式部署的一个或多个服务器上的数据库信息,可以包括如下步骤:As an optional embodiment, as shown in FIG. 2 , in the foregoing step S102, obtaining database information on one or more servers deployed in a distributed manner may include the following steps:

步骤S202,通过数据库管理系统调用服务器上的数据库系统表;Step S202, calling the database system table on the server through the database management system;

步骤S204,从数据库系统表获取数据库信息。Step S204, acquiring database information from the database system table.

具体地,在上述步骤中,上述数据库管理系统可以用于操纵和管理上述分布式部署的一个或多个服务器上的数据库的大型软件,一种可选的实施例中,该数据库管理系统可以为SQL Server关系型数据库管理系统,上述数据库系统表可以为上述分布式部署的不同服务器上的数据系统表,存储有该服务器上存储的所有的数据库的信息,例如,用户在数据库中创建的各种表信息,数据库中存储的文件大小等。基于上述步骤S202至S204,利用分布式数据库中的数据库管理系统(例如,SQL Server)调用分布式部署的一个或多个服务器上的数据库系统表,根据不同服务器上的数据库系统表记载的数据库信息,可以获取各个服务器上的数据库信息。Specifically, in the above steps, the above-mentioned database management system may be used for large-scale software for manipulating and managing the databases on one or more servers deployed in the above-mentioned distribution. In an optional embodiment, the database management system may be SQL Server relational database management system, the above-mentioned database system table can be the data system table on different servers of the above-mentioned distributed deployment, and stores the information of all the databases stored on the server, for example, various types of data created by the user in the database Table information, file size stored in the database, etc. Based on the above steps S202 to S204, the database management system (for example, SQL Server) in the distributed database is used to call the database system tables on one or more servers deployed in a distributed manner, according to the database information recorded in the database system tables on different servers , you can obtain database information on each server.

一种可选的实施例中,上述数据库可以为SQL Sever数据库,上述数据库系统表可以为SQL Sever数据库中的系统表,例如Sysdatabases、Sysxlogins、Systypes、Sysusers和Sysobjects,其中,系统表Sysdatabases存在于SQL Sever的Master数据库中,用于存储有当前系统所有的数据库,系统表Sysxlogins,用于存储所有能登陆SQL Sever的帐号;系统表Systypes,存在于每个数据库中,用于存储系统中默认的用户类型和用户自定义的用户类型,系统表Sysusers用于存储有每个数据库中的用户和角色,系统表Sysobjects用于存储在数据库内创建的每个对象(约束、默认、日志、规则、存储过程)。In an optional embodiment, the above-mentioned database can be a SQL Sever database, and the above-mentioned database system table can be a system table in the SQL Sever database, such as Sysdatabases, Sysxlogins, Systypes, Sysusers, and Sysobjects, wherein the system table Sysdatabases exists in SQL Sever. In Sever's Master database, it is used to store all the databases of the current system, the system table Sysxlogins is used to store all accounts that can log in to SQL Sever; the system table Systypes, which exists in each database, is used to store the default users in the system Types and user-defined user types, the system table Sysusers is used to store users and roles in each database, and the system table Sysobjects is used to store each object (constraint, default, log, rule, stored procedure) created in the database ).

此处需要说明的是,现有技术通过查看生产服务器本地磁盘文件大小的方式获得SQL Server数据库中存储的文件大小,但是当服务器比较多的情况下,很难保证SQLServer数据库在每个服务器上存储的路径一致。通过上述实施例,实现了获取多个分布式部署的服务器上的存储的数据库信息。It should be noted here that the existing technology obtains the file size stored in the SQL Server database by checking the file size of the local disk of the production server, but when there are many servers, it is difficult to ensure that the SQL Server database is stored on each server. path is the same. Through the above embodiments, the database information stored on a plurality of distributed deployment servers can be obtained.

作为一种可选的实施例,如图3所示,在获取分布式部署的一个或多个服务器上的数据库信息之后,上述方法还可以包括如下步骤:As an optional embodiment, as shown in FIG. 3 , after acquiring database information on one or more servers deployed in a distributed manner, the above method may further include the following steps:

步骤S302,获取分布式部署的一个或多个服务器的标识信息;Step S302, obtaining identification information of one or more servers deployed in a distributed manner;

步骤S304,将标识信息和服务器上的数据库信息存储至数据库临时表中。Step S304, the identification information and the database information on the server are stored in the database temporary table.

具体地,在上述步骤中,上述服务器可以为分布式部署的一个或多个服务器;上述标识信息可以为用于标识分布式部署的每个服务器的IP地址;在获取分布式部署的一个或多个服务器上的数据库信息之后,可以在数据库中建立一个临时表用于存储获取到的不同服务器上的数据库信息,同时获取相应的服务器的标识信息存储于临时表中,用以区分数据库来自哪个服务器。Specifically, in the above steps, the above-mentioned server may be one or more servers deployed in a distributed manner; the above-mentioned identification information may be an IP address used to identify each server in the distributed deployment; After the database information on each server, a temporary table can be established in the database to store the obtained database information on different servers, and the identification information of the corresponding server can be obtained and stored in the temporary table to distinguish which server the database comes from. .

可选地,可以利用windows身份依次获得同一个域内不同服务器上数据库信息。Optionally, the database information on different servers in the same domain can be obtained sequentially by using the Windows identity.

通过上述实施例,建立数据库临时表,用于将从分布式部署的多个服务器上获取的数据库信息进行汇总记录,达到了将不同服务器上的数据库批量分析的目的。Through the above-mentioned embodiment, a temporary database table is established for summarizing and recording database information obtained from multiple servers deployed in a distributed manner, so as to achieve the purpose of batch analysis of databases on different servers.

在一种可选的实施例中,如图4所示,上述步骤S104中根据数据库中存储的文件的大小和/或表的总行数确定需要重建的数据库,可以包括如下步骤:In an optional embodiment, as shown in FIG. 4 , in the above step S104, the database to be rebuilt is determined according to the size of the file stored in the database and/or the total number of rows in the table, which may include the following steps:

步骤S402,判断数据库是否满足预设条件,其中,预设条件至少包括以下任意之一:数据库中存储的文件的大小大于预设阈值、数据库中存储的表的总行数大于预设行数;Step S402, judging whether the database satisfies a preset condition, wherein the preset condition includes at least any one of the following: the size of the file stored in the database is greater than a preset threshold, and the total number of rows of the table stored in the database is greater than the preset number of rows;

步骤S404,从数据库临时表中筛选出满足预设条件的数据库。Step S404: Screen out the database that meets the preset condition from the database temporary table.

具体地,在上述步骤中,上述数据库可以为分布式部署一个或多个服务器上的数据库;上述预设条件可以是为了对数据库进行处理预先设置的任何条件,一种可选的实例中,上述预设条件可以为数据库中存储的文件的大小,和/或,数据库中存储的表的总行数,根据分布式部署的一个或多个服务器上的数据库中存储的文件大小和表的总行数,判断数据库临时表中的数据库是否满足预设条件,将数据库临时表中满足预设条件的数据库进行过滤、筛选,从而得到需要重建的数据库。Specifically, in the above-mentioned steps, the above-mentioned database may be a database deployed on one or more servers in a distributed manner; the above-mentioned preset condition may be any condition preset for processing the database. In an optional example, the above-mentioned The preset condition may be the size of the file stored in the database, and/or the total number of rows of the table stored in the database, according to the size of the file stored in the database on one or more servers deployed in a distributed manner and the total number of rows of the table, It is judged whether the database in the database temporary table satisfies the preset condition, and the database that satisfies the preset condition in the database temporary table is filtered and screened, so as to obtain the database that needs to be rebuilt.

一种优选的实施例中,上述预设条件为同时满足数据库中存储的文件的大小大于预设阈值、数据库中存储的表的总行数大于预设行数。In a preferred embodiment, the above preset conditions are simultaneously satisfied that the size of the file stored in the database is greater than a preset threshold, and the total number of rows of the table stored in the database is greater than the preset number of rows.

通过上述实施例,可以实现将分布式部署的一个或多个服务器上的数据库中需要重建的数据库全部筛选的目的。Through the above embodiments, the purpose of screening all databases that need to be rebuilt among the databases on one or more servers deployed in a distributed manner can be achieved.

在一种可选的实施例中,如图5所示,上述步骤S106中对需要重建的数据库进行重建,可以包括如下步骤:In an optional embodiment, as shown in FIG. 5 , the rebuilding of the database to be rebuilt in the foregoing step S106 may include the following steps:

步骤S502,获取数据库临时表中需要重建的数据库;Step S502, obtaining the database that needs to be rebuilt in the database temporary table;

步骤S504,删除需要重建的数据库;Step S504, delete the database that needs to be rebuilt;

步骤S506,利用模板数据库对需要重建的数据库进行重建。Step S506, using the template database to rebuild the database that needs to be rebuilt.

具体地,在上述步骤中,在根据数据库中存储的文件的大小和/或数据表的总行数,在数据库临时表中筛选出需要重建的数据库后,删除需要重建的数据库,利用空白的、没有任何数据的模板数据库对需要重建的数据库进行重建。Specifically, in the above steps, after filtering out the database that needs to be rebuilt in the temporary table of the database according to the size of the file stored in the database and/or the total number of rows in the data table, delete the database that needs to be rebuilt, use the blank, no A template database of any data rebuilds the database that needs to be rebuilt.

可选地,基于上述实施例,在删除需要重建的数据库之前,上述方法还包括:将需要重建的数据库备份至本地磁盘或备份服务器中。Optionally, based on the foregoing embodiment, before deleting the database that needs to be rebuilt, the foregoing method further includes: backing up the database that needs to be rebuilt to a local disk or a backup server.

在一种可选的实施例中,如图6所示,在利用模板数据库对需要重建的数据库进行重建之后,上述方法还可以包括如下步骤:In an optional embodiment, as shown in FIG. 6, after using the template database to rebuild the database to be rebuilt, the above method may further include the following steps:

步骤S602,对重建的数据库进行校验;Step S602, verifying the reconstructed database;

步骤S604,在校验成功的情况下,删除本地磁盘或备份服务器中备份的数据库。Step S604, in the case that the verification is successful, delete the database backed up in the local disk or the backup server.

作为一种优选的实施例,可以结合图7所示的SQL Server中批量重建数据的方法流程图来说明本申请上述实施例,如图7所示,该方法包括如下步骤:As a preferred embodiment, the above-mentioned embodiment of the present application can be described with reference to the flowchart of the method for batch rebuilding data in SQL Server shown in FIG. 7 . As shown in FIG. 7 , the method includes the following steps:

步骤S702,利用SQL查询不同服务器上的数据库信息。Step S702, using SQL to query database information on different servers.

在上述步骤中,上述数据库信息可以包括:数据库中存储的文件的大小和表的总行数,通过执行数据库系统表,可以获取分布式部署的多个服务器上的数据库的数据库文件、日志文件、表的行数等详细信息。In the above-mentioned steps, the above-mentioned database information may include: the size of the files stored in the database and the total number of rows in the table, and by executing the database system table, the database files, log files, and tables of the databases on multiple servers of distributed deployment can be obtained. details such as the number of rows.

一种可选的实施例中,可以通过下面的程序来实现查询不同服务器上的数据库信息。程序示例:In an optional embodiment, the following program can be used to query database information on different servers. Program example:

Select name,convert(float,size)*(8192.0/1024.0)/1024/1024.FromWebDissectorReceiver_800003.Select name, convert(float, size)*(8192.0/1024.0)/1024/1024.FromWebDissectorReceiver_800003.

Select Count(*)表行数Select Count(*)Number of table rows

FROM[WebDissectorReceiver_800003].[E_Session].FROM[WebDissectorReceiver_800003].[E_Session].

上述程序的输出结果为:The output of the above program is:

Figure BDA0001160358540000081
Figure BDA0001160358540000081

Figure BDA0001160358540000091
Figure BDA0001160358540000091

表行数table row count 456153456153

由上述输出结果可以看出,查询得到的数据库名称为WebDissectorReceiver,数据库文件的大小为1.12890625,数据库中表的总行数为456153。As can be seen from the above output results, the database name obtained by the query is WebDissectorReceiver, the size of the database file is 1.12890625, and the total number of rows in the table in the database is 456153.

步骤S704,将获取到的数据库信息存储到新的数据表中,存储时增加一个字符串来记录服务器的IP地址。Step S704: Store the acquired database information in a new data table, and add a character string to record the IP address of the server during storage.

在上述步骤中,通过使用循环语句就可以将整个生产环境的所有服务器上所有的数据库的信息存储到新的数据库表中,存储时增加一个字符串字段用来记录服务器的IP地址。In the above steps, the information of all databases on all servers in the entire production environment can be stored in a new database table by using a loop statement, and a string field is added to record the IP address of the server during storage.

一种可选的实施例中,可以通过下面的程序来实现将获取到的数据库信息存储到新的数据表中。部分程序示例:In an optional embodiment, the following program can be used to realize storing the acquired database information in a new data table. Some program examples:

Figure BDA0001160358540000092
Figure BDA0001160358540000092

输出结果为:The output is:

IDID DbnameDbname mdfmdf ldfldf TableCountTableCount ServerServer 00 receiverreceiver 1.1289061.128906 0.299370.29937 2345623456 172.1.1.1172.1.1.1

在上述实施例中,Dbname表示数据名称,mdf表示数据库文件,ldf表示日志文件,TableCount表示数据库的表行数,Server表示服务器。的由上述输出结果可以看出,将receiver数据库存储至的新的数据表后,新的数据表中,该数据库的表ID号为0,数据库的名称为receiver,数据库文件的大小为1.128906,日志文件大小为0.29937,表的行数为23456,该数据库所在服务器的IP地址为172.1.1.1。In the above embodiment, Dbname represents the data name, mdf represents the database file, ldf represents the log file, TableCount represents the number of table rows in the database, and Server represents the server. It can be seen from the above output results that after storing the receiver database to a new data table, in the new data table, the table ID number of the database is 0, the name of the database is receiver, the size of the database file is 1.128906, and the log The file size is 0.29937, the number of rows in the table is 23456, and the IP address of the server where the database is located is 172.1.1.1.

步骤S706,备份数据库到本地磁盘或者冷备服务器。Step S706, back up the database to a local disk or a cold standby server.

在上述步骤中,将分布式部署的一个或多个服务器上的数据库备份至本地磁盘或备份服务器(例如冷备服务器)中。In the above steps, the database on one or more servers deployed in a distributed manner is backed up to a local disk or a backup server (for example, a cold standby server).

一种可选的实施例中,将多个服务器上的数据库批量备份可以通过以下的代码实现,部分程序示例为:In an optional embodiment, the batch backup of databases on multiple servers can be implemented by the following code, and some program examples are:

Figure BDA0001160358540000101
Figure BDA0001160358540000101

Figure BDA0001160358540000111
Figure BDA0001160358540000111

Figure BDA0001160358540000121
Figure BDA0001160358540000121

一种可选的实施例中,将上述脚本代码复制到GWD目标数据库服务器SQL ServerManagement Studio中,根据实际情况更改执行参数,使用DEBUG模式查看Messages中输出的脚本是否正确,确认无误后更改@ExecMode。In an optional embodiment, the above script code is copied to the GWD target database server SQL Server Management Studio, the execution parameters are changed according to the actual situation, the DEBUG mode is used to check whether the script output in Messages is correct, and @ExecMode is changed after confirming that it is correct.

此处需要说明的是,批量备份数据库不包括AS库,AS库可以通过AS程序的j:d的方式基于模板创建,因此不需要备份。It should be noted here that the batch backup database does not include the AS library. The AS library can be created based on the template in the j:d method of the AS program, so backup is not required.

优选地,上述备份服务器可以是冷备服务器。Preferably, the above-mentioned backup server may be a cold backup server.

步骤S708,根据数据库中存储的文件的大小和表的总行数,确定需要重建的数据库。Step S708: Determine the database to be rebuilt according to the size of the file stored in the database and the total number of rows in the table.

在上述步骤中,在将获取到的数据库信息存储到新的数据表中后,根据数据库中存储的文件的大小和表的总行数,确定需要重建的数据库。In the above steps, after the acquired database information is stored in a new data table, the database to be rebuilt is determined according to the size of the file stored in the database and the total number of rows in the table.

步骤S710,删除旧的数据库。Step S710, delete the old database.

在上述步骤中,在确定需要重建的数据库后,将分布式部署的多个服务器上的这些数据库删除。In the above steps, after determining the databases that need to be rebuilt, delete these databases on multiple servers deployed in a distributed manner.

步骤S712,创建新的数据库。Step S712, create a new database.

在上述步骤中,在将分布式部署的多个服务器上的这些数据库删除后,利用模板数据,在这些服务器上创建新的数据库。In the above steps, after the databases on the multiple servers deployed in a distributed manner are deleted, new databases are created on these servers by using the template data.

步骤S714,校验后删除备份的数据库。Step S714, delete the backup database after verification.

本申请上述实施例公开的方案,不使用window本地磁盘获取数据库占用磁盘文件大小,转而从SQL Server中使用该数据库的系统表得到占用磁盘文件的大小,既准确又便于导入到数据库表中,而且方便使用SQL语句过滤条件进行筛选,筛选成功后通过脚本删除后统一重建,避免因sql-server部署路径造成的不同服务器的文件路径不一致。The solutions disclosed in the above embodiments of the present application do not use the window local disk to obtain the size of the disk file occupied by the database, but instead use the system table of the database in SQL Server to obtain the size of the occupied disk file, which is accurate and easy to import into the database table, Moreover, it is convenient to use SQL statement filter conditions for filtering. After the filtering is successful, the script is deleted and then rebuilt uniformly, so as to avoid inconsistent file paths of different servers caused by the sql-server deployment path.

实施例2Example 2

根据本发明实施例,还提供了一种批量重建数据库的装置实施例。本发明实施例1中的批量重建数据库的方法可以在本发明实施例2的装置中执行。According to an embodiment of the present invention, an embodiment of an apparatus for batch rebuilding a database is also provided. The method for batch rebuilding a database in Embodiment 1 of the present invention may be executed in the apparatus of Embodiment 2 of the present invention.

图8是根据本发明实施例的一种批量重建数据库的装置示意图,如图8所示,该装置包括:获取模块801、确定模块803和创建模块805。FIG. 8 is a schematic diagram of an apparatus for batch rebuilding a database according to an embodiment of the present invention. As shown in FIG. 8 , the apparatus includes: an acquisition module 801 , a determination module 803 , and a creation module 805 .

其中,获取模块801,用于获取分布式部署的一个或多个服务器上的数据库信息,其中,数据库信息至少包括:数据库的名称、数据库中存储的文件的大小、数据库中存储的数据表的总行数;确定模块803,用于根据文件的大小和/或数据表的总行数确定需要重建的数据库;创建模块805,用于对需要重建的数据库进行重建。Wherein, the obtaining module 801 is used to obtain database information on one or more servers of distributed deployment, wherein the database information at least includes: the name of the database, the size of the file stored in the database, the total row of the data table stored in the database The determining module 803 is used for determining the database that needs to be rebuilt according to the size of the file and/or the total number of rows in the data table; the creating module 805 is used for rebuilding the database that needs to be rebuilt.

由上可知,在本申请上述实施例中,通过获取同一个域内的不同服务器上的数据库中存储的文件大小和表的行数,并根据数据库中存储的文件大小和表的行数两个指标来确定需要重建的数据库,最后基于模板数据,将需要重建的数据库进行重建。容易注意,本申请上述实施例公开的方案,未使用本次磁盘获取数据库占用磁盘空间的文件大小,而是从数据库管理系统中获取多个不同服务器上的数据库信息,对不同服务器上数据库的文件大小和表的行数进行分析,来确定需要重建的数据库,达到了对分布式部署的一个或多个服务器上的数据库进行批量处理的目的,从而实现了对不同服务器上的数据库进行批量重建的技术效果,进而解决了现有技术无法实现将分布式部署的多个服务器上的数据库进行批量重建的技术问题。It can be seen from the above that in the above-mentioned embodiment of the present application, by obtaining the file size and the number of rows of the table stored in the databases on different servers in the same domain, and according to the two indicators of the size of the file and the number of rows of the table stored in the database to determine the database that needs to be rebuilt, and finally rebuild the database that needs to be rebuilt based on the template data. It is easy to note that the solutions disclosed in the above-mentioned embodiments of the present application do not use this disk to obtain the file size of the disk space occupied by the database, but obtain database information on multiple different servers from the database management system. The size and the number of rows in the table are analyzed to determine the database that needs to be rebuilt, so as to achieve the purpose of batch processing the databases on one or more servers deployed in a distributed manner, thereby realizing the batch reconstruction of databases on different servers. The technical effect further solves the technical problem that the existing technology cannot realize the batch reconstruction of the databases on multiple servers deployed in a distributed manner.

在一种可选的实施例中,上述第一获取模块包括:调用模块,用于通过数据库管理系统调用服务器上的数据库系统表;第二获取模块,用于从数据库系统表获取数据库信息。In an optional embodiment, the above-mentioned first obtaining module includes: a calling module for calling a database system table on the server through a database management system; and a second obtaining module for obtaining database information from the database system table.

在一种可选的实施例中,上述装置还包括:第三获取模块,用于获取服务器的标识信息;存储模块,用于将标识信息和服务器的数据库信息存储至数据库临时表中。In an optional embodiment, the above-mentioned apparatus further includes: a third obtaining module, configured to obtain the identification information of the server; and a storage module, configured to store the identification information and the database information of the server in a temporary database table.

在一种可选的实施例中,上述确定模块还包括:判断数据库是否满足预设条件,其中,预设条件包括以下任意之一:文件的大小大于预设阈值、数据表的总行数大于预设行数;从数据库临时表中筛选出满足预设条件的数据库。In an optional embodiment, the above determination module further includes: judging whether the database satisfies a preset condition, wherein the preset condition includes any one of the following: the size of the file is greater than a preset threshold, and the total number of rows in the data table is greater than a preset Set the number of rows; filter out the database that meets the preset conditions from the database temporary table.

在一种可选的实施例中,上述创建模块还包括:第四获取模块,用于获取数据库临时表中需要重建的数据库;第一删除模块,用于删除需要重建的数据库中的原有数据;处理模块,用于利用模板数据对需要重建的数据库进行重建。In an optional embodiment, the above-mentioned creation module further includes: a fourth acquisition module, used for acquiring the database that needs to be rebuilt in the temporary table of the database; a first deletion module, used for deleting the original data in the database that needs to be rebuilt ; The processing module is used to rebuild the database that needs to be rebuilt by using the template data.

在一种可选的实施例中,上述装置还包括:第二存储模块,用于将需要重建的数据库中存储的原有数据备份至本地磁盘或备份服务器中。In an optional embodiment, the above-mentioned apparatus further includes: a second storage module, configured to back up the original data stored in the database that needs to be rebuilt to a local disk or a backup server.

在一种可选的实施例中,上述装置还包括:校验模块,用于对重建的数据库进行校验;第二删除模块,用于在校验成功的情况下,删除本地磁盘或备份服务器中备份的数据库。In an optional embodiment, the above-mentioned apparatus further includes: a verification module for verifying the reconstructed database; a second deletion module for deleting the local disk or the backup server when the verification is successful database backed up in .

上述本发明实施例序号仅仅为了描述,不代表实施例的优劣。The above-mentioned serial numbers of the embodiments of the present invention are only for description, and do not represent the advantages or disadvantages of the embodiments.

在本发明的上述实施例中,对各个实施例的描述都各有侧重,某个实施例中没有详述的部分,可以参见其他实施例的相关描述。In the above-mentioned embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

在本申请所提供的几个实施例中,应该理解到,所揭露的技术内容,可通过其它的方式实现。其中,以上所描述的装置实施例仅仅是示意性的,例如所述单元的划分,可以为一种逻辑功能划分,实际实现时可以有另外的划分方式,例如多个单元或组件可以结合或者可以集成到另一个系统,或一些特征可以忽略,或不执行。另一点,所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口,单元或模块的间接耦合或通信连接,可以是电性或其它的形式。In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are only illustrative, for example, the division of the units may be a logical function division, and there may be other division methods in actual implementation, for example, multiple units or components may be combined or Integration into another system, or some features can be ignored, or not implemented. On the other hand, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of units or modules, and may be in electrical or other forms.

所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。The units described as separate components may or may not be physically separated, and components shown as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

另外,在本发明各个实施例中的各功能单元可以集成在一个处理单元中,也可以是各个单元单独物理存在,也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现,也可以采用软件功能单元的形式实现。In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware, or may be implemented in the form of software functional units.

所述集成的单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时,可以存储在一个计算机可读取存储介质中。基于这样的理解,本发明的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的全部或部分可以以软件产品的形式体现出来,该计算机软件产品存储在一个存储介质中,包括若干指令用以使得一台计算机设备(可为个人计算机、服务器或者网络设备等)执行本发明各个实施例所述方法的全部或部分步骤。而前述的存储介质包括:U盘、只读存储器(ROM,Read-Only Memory)、随机存取存储器(RAM,Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。The integrated unit, if implemented in the form of a software functional unit and sold or used as an independent product, may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium , including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes .

以上所述仅是本发明的优选实施方式,应当指出,对于本技术领域的普通技术人员来说,在不脱离本发明原理的前提下,还可以做出若干改进和润饰,这些改进和润饰也应视为本发明的保护范围。The above are only the preferred embodiments of the present invention. It should be pointed out that for those skilled in the art, without departing from the principles of the present invention, several improvements and modifications can be made. It should be regarded as the protection scope of the present invention.

Claims (6)

1. A method for batch database reconstruction, comprising:
obtaining database information on one or more servers in a distributed deployment, wherein the database information at least comprises: the name of the database, the size of the file stored in the database, and the total number of rows of the table stored in the database;
determining a database to be reconstructed according to the size of the file stored in the database and/or the total row number of the table;
reconstructing the database to be reconstructed;
the acquiring database information on one or more servers deployed in a distributed manner includes:
calling a database system table on the server through a database management system;
acquiring the database information from the database system table;
wherein after obtaining database information on one or more servers of a distributed deployment, the method further comprises:
acquiring identification information of one or more servers deployed in a distributed manner;
and storing the identification information and the database information on the server into a database temporary table, wherein the database temporary table is established for summarizing and recording the database information on the server.
2. The method of claim 1, wherein determining the database that needs to be reconstructed based on the size of the file stored in the database and/or the total number of rows in the table comprises:
judging whether the database meets preset conditions or not, wherein the preset conditions at least comprise any one of the following conditions: the size of the file stored in the database is larger than a preset threshold value, and the total row number of the table stored in the database is larger than a preset row number;
and screening out the database meeting the preset conditions from the temporary database table.
3. The method of claim 1, wherein rebuilding the database to be rebuilt comprises:
acquiring a database to be reconstructed in the database temporary table;
deleting the database to be reconstructed;
and rebuilding the database to be rebuilt by utilizing the template database.
4. The method of claim 3, wherein prior to deleting the database requiring reconstruction, the method comprises: and backing up the database to be rebuilt to a local disk or a backup server.
5. The method of claim 4, wherein after the rebuilding the database to be rebuilt using the template database, the method further comprises:
checking the reconstructed database;
and deleting the database backed up in the local disk or the backup server under the condition of successful verification.
6. An apparatus for batch reconstruction of a database, comprising:
a first obtaining module, configured to obtain database information on one or more servers in distributed deployment, where the database information at least includes: the name of the database, the size of a file stored in the database, and the total number of rows of a data table stored in the database;
the determining module is used for determining the database to be reconstructed according to the size of the file stored in the database and/or the total row number of the table;
the creating module is used for reconstructing the database needing to be reconstructed;
wherein the first obtaining module comprises:
the calling module is used for calling the database system table on the server through a database management system;
the second acquisition module is used for acquiring the database information from the database system table;
wherein the apparatus further comprises:
the third acquisition module is used for acquiring the identification information of the server;
and the first storage module is used for storing the identification information and the database information on the server into a database temporary table, wherein the database temporary table is established and used for summarizing and recording the database information on the server.
CN201611044272.7A 2016-11-23 2016-11-23 Method and device for reconstructing database in batches Expired - Fee Related CN108090095B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611044272.7A CN108090095B (en) 2016-11-23 2016-11-23 Method and device for reconstructing database in batches

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611044272.7A CN108090095B (en) 2016-11-23 2016-11-23 Method and device for reconstructing database in batches

Publications (2)

Publication Number Publication Date
CN108090095A CN108090095A (en) 2018-05-29
CN108090095B true CN108090095B (en) 2020-09-15

Family

ID=62170953

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611044272.7A Expired - Fee Related CN108090095B (en) 2016-11-23 2016-11-23 Method and device for reconstructing database in batches

Country Status (1)

Country Link
CN (1) CN108090095B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109522295B (en) * 2018-11-16 2021-07-20 中国银行股份有限公司 Automatic database object constructing and deploying method and device
CN109828865A (en) * 2019-01-24 2019-05-31 北京三快在线科技有限公司 Data reconstruction method, device and electronic equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103136335A (en) * 2013-01-31 2013-06-05 北京千分点信息科技有限公司 Data control method based on data platforms
CN103646111A (en) * 2013-12-25 2014-03-19 普元信息技术股份有限公司 System and method for realizing real-time data association in big data environment
CN103914393A (en) * 2014-04-11 2014-07-09 四川华拓光通信股份有限公司 MCU-based (microprogrammed control unit-based) non-loss FLASH storage update method
CN104166661A (en) * 2013-05-20 2014-11-26 方正宽带网络服务股份有限公司 Data storage system and method

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102142008B (en) * 2010-12-02 2013-04-17 华为技术有限公司 Method and system for implementing distributed memory database, token controller and memory database

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103136335A (en) * 2013-01-31 2013-06-05 北京千分点信息科技有限公司 Data control method based on data platforms
CN104166661A (en) * 2013-05-20 2014-11-26 方正宽带网络服务股份有限公司 Data storage system and method
CN103646111A (en) * 2013-12-25 2014-03-19 普元信息技术股份有限公司 System and method for realizing real-time data association in big data environment
CN103914393A (en) * 2014-04-11 2014-07-09 四川华拓光通信股份有限公司 MCU-based (microprogrammed control unit-based) non-loss FLASH storage update method

Also Published As

Publication number Publication date
CN108090095A (en) 2018-05-29

Similar Documents

Publication Publication Date Title
US10073652B2 (en) Performance optimized storage vaults in a dispersed storage network
US8572108B2 (en) Method and system for specifying, preparing and using parameterized database queries
CN106570086B (en) Data migration system and data migration method
US11593029B1 (en) Identifying a parent event associated with child error states
CN108664480B (en) Multi-data-source user information integration method and device
CN106815326B (en) System and method for detecting consistency of data table without main key
WO2017067117A1 (en) Data query method and device
US20170031948A1 (en) File synchronization method, server, and terminal
CN107423436A (en) A kind of method migrated for online data between distinct type data-base
CN104935469A (en) A log information distributed storage method and system
CN108427728A (en) Management method, equipment and the computer-readable medium of metadata
CN110888837A (en) Object storage small file merging method and device
CN112307297B (en) User identification unification method and system based on priority rule
CN108090095B (en) Method and device for reconstructing database in batches
CN111026711A (en) Block chain based data storage method and device, computer equipment and storage medium
CN109542913B (en) A network asset security management method in complex environment
CN114328749A (en) Business data processing method and device and computer readable storage medium
CN104539449A (en) Handling method and related device for fault information
CN109492051B (en) Pipe network data synchronization method and device
CN107203550B (en) Data processing method and database server
CN101447886B (en) Method for comparing mass data and device thereof
CN113312351A (en) Data processing method and device
EP4292005B1 (en) System and method for representing and verifying a data set using a tree-based data structure
CN112699177A (en) Method, system, computing device and storage medium for synchronizing traffic ticket data
CN113836224B (en) OGG to HDFS synchronous file processing method and device and computer equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
CB02 Change of applicant information
CB02 Change of applicant information

Address after: 100083 No. 401, 4th Floor, Haitai Building, 229 North Fourth Ring Road, Haidian District, Beijing

Applicant after: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd.

Address before: 100086 Cuigong Hotel, 76 Zhichun Road, Shuangyushu District, Haidian District, Beijing

Applicant before: BEIJING GRIDSUM TECHNOLOGY Co.,Ltd.

GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20200915