[go: up one dir, main page]

CN117891661A - Data backup method, device, electronic device and readable storage medium - Google Patents

Data backup method, device, electronic device and readable storage medium Download PDF

Info

Publication number
CN117891661A
CN117891661A CN202410153412.2A CN202410153412A CN117891661A CN 117891661 A CN117891661 A CN 117891661A CN 202410153412 A CN202410153412 A CN 202410153412A CN 117891661 A CN117891661 A CN 117891661A
Authority
CN
China
Prior art keywords
computer room
machine
node
data
main
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202410153412.2A
Other languages
Chinese (zh)
Inventor
肖子桥
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Du Xiaoman Technology Beijing Co Ltd
Original Assignee
Du Xiaoman Technology Beijing Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Du Xiaoman Technology Beijing Co Ltd filed Critical Du Xiaoman Technology Beijing Co Ltd
Priority to CN202410153412.2A priority Critical patent/CN117891661A/en
Publication of CN117891661A publication Critical patent/CN117891661A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/14Error detection or correction of the data by redundancy in operation
    • G06F11/1402Saving, restoring, recovering or retrying
    • G06F11/1446Point-in-time backing up or restoration of persistent data
    • G06F11/1448Management of the data involved in backup or backup restore

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The invention provides a data backup method, a device, an electronic device and a readable storage medium, comprising the following steps: receiving a first response signal used for representing that the logic replication relationship between each machine in the main machine room and each machine in the standby machine room is established, wherein the logic replication relationship is used for representing the mapping relationship between a first main node of each machine in the standby machine room and a second mirror node of each machine in the main machine room; decoding the data change in the second mirror image node of each machine in the main machine room based on the logic replication relationship, and modifying the data in the first main node in the standby machine room according to the decoding result; and if the host computer room fails, controlling a first main node in the standby computer room to respond to a first target request, wherein the first target request is a data request for requesting data stored in the main computer room. The invention solves the problem that real-time backup cannot be performed during cross-machine room data backup.

Description

数据备份方法、装置、电子设备及可读存储介质Data backup method, device, electronic device and readable storage medium

技术领域Technical Field

本发明涉及数据处理技术领域,尤其涉及一种数据备份方法、装置、电子设备及可读存储介质。The present invention relates to the field of data processing technology, and in particular to a data backup method, device, electronic device and readable storage medium.

背景技术Background technique

Greenplum数据库为大规模并行处理架构的分布式数据库管理系统,Greenplum数据库的特点是数据分散的存储在集群不同机器的不同节点上,且数据量普遍较大。但在机房发生故障时,现有的全量备份方案需要一定的备份周期,因此无法做到实时备份,使得备份机房的数据与主机房的数据差距增大。Greenplum database is a distributed database management system with a large-scale parallel processing architecture. The characteristic of Greenplum database is that data is stored in different nodes of different machines in the cluster, and the data volume is generally large. However, when a computer room fails, the existing full backup solution requires a certain backup cycle, so it is impossible to achieve real-time backup, which increases the gap between the data in the backup computer room and the data in the main computer room.

发明内容Summary of the invention

有鉴于此,本发明实施例提供了一种数据备份方法、装置、电子设备及可读存储介质,以解决在跨机房数据备份时,无法实时备份的问题。In view of this, the embodiments of the present invention provide a data backup method, device, electronic device and readable storage medium to solve the problem that real-time backup is not possible when backing up data across computer rooms.

根据本发明的一方面,提供了一种数据备份方法,应用于Greenplum数据库系统中的管理节点,Greenplum数据库系统还包括主机房与备用机房,主机房和备用机房中的各机器均包括第一主节点与第二镜像节点,第二镜像节点为其他机器的第一主节点的镜像节点,方法包括:According to one aspect of the present invention, a data backup method is provided, which is applied to a management node in a Greenplum database system. The Greenplum database system further includes a main computer room and a backup computer room. Each machine in the main computer room and the backup computer room includes a first master node and a second mirror node. The second mirror node is a mirror node of the first master node of other machines. The method includes:

接收第一响应信号,第一响应信号用于表征主机房中各机器与备用机房中的各机器的逻辑复制关系已建立,逻辑复制关系用于表示备用机房内各机器的第一主节点与主机房内各机器的第二镜像节点之间的映射关系;receiving a first response signal, the first response signal being used to indicate that a logical replication relationship between each machine in the main machine room and each machine in the standby machine room has been established, the logical replication relationship being used to indicate a mapping relationship between a first master node of each machine in the standby machine room and a second mirror node of each machine in the main machine room;

基于逻辑复制关系,将主机房内各机器的第二镜像节点中的数据变化进行解码,按照解码结果对备用机房内第一主节点中的数据进行修改;Based on the logical replication relationship, the data changes in the second mirror node of each machine in the main computer room are decoded, and the data in the first main node in the standby computer room is modified according to the decoding results;

若主机房发生故障,控制备用机房内第一主节点响应第一目标请求,其中,第一目标请求为请求主机房中存储数据的数据请求。If a failure occurs in the main computer room, the first master node in the standby computer room is controlled to respond to the first target request, wherein the first target request is a data request for data stored in the main computer room.

可选地,在接收第一响应信号前,还包括:Optionally, before receiving the first response signal, the method further includes:

接收第一指令,第一指令为建立备用机房内各机器与主机房内各机器之间逻辑复制关系的信号;receiving a first instruction, the first instruction being a signal for establishing a logical replication relationship between each machine in the standby computer room and each machine in the main computer room;

响应于第一指令,将主机房内各机器的第二镜像节点中的数据对象及其变化复制至对应的备用机房内各机器的第一主节点中。In response to the first instruction, the data objects and changes therein in the second mirror node of each machine in the main computer room are copied to the first main node of each machine in the corresponding standby computer room.

可选地,在接收第一响应信号后,还包括:Optionally, after receiving the first response signal, the method further includes:

接收第一新增信号,第一新增信号为主机房内各机器的第二镜像节点中数据的新增信号;receiving a first newly added signal, where the first newly added signal is a signal of newly added data in the second mirror node of each machine in the main computer room;

响应于第一新增信号,将主机房内各机器的第二镜像节点中的新增数据复制到至备用机房内第一主节点中。In response to the first new signal, the new data in the second mirror node of each machine in the main computer room is copied to the first main node in the standby computer room.

可选地,在接收第一响应信号前,还包括:Optionally, before receiving the first response signal, the method further includes:

接收第二指令,第二指令为建立第二镜像节点与第一主节点之间复制关系的信号;receiving a second instruction, where the second instruction is a signal for establishing a replication relationship between the second mirror node and the first master node;

响应于第一指令,将主机房内各机器第一主节点中的数据复制至对应的第二镜像节点中;In response to the first instruction, the data in the first main node of each machine in the main computer room is copied to the corresponding second mirror node;

接收第二新增信号,第二新增信号为主机房内各机器的第一主节点中数据的变更信号;receiving a second newly added signal, where the second newly added signal is a signal for changing data in a first main node of each machine in the main computer room;

响应于第二新增信号,将与变更信号对应的数据对象及其变化复制到主机房内各机器的第二镜像主节点中。In response to the second new addition signal, the data object corresponding to the change signal and its changes are copied to the second mirror master node of each machine in the main computer room.

可选地,在响应于第一指令,将主机房内各机器第一主节点中的数据复制至对应的第二镜像节点中后,还包括:Optionally, after the data in the first main node of each machine in the main computer room is copied to the corresponding second mirror node in response to the first instruction, the method further includes:

若主机房内任一机器的第一主节点发生故障,控制对应的第二镜像节点响应第二目标请求,其中,第二目标请求为请求第一主节点存储数据的数据请求。If the first main node of any machine in the main computer room fails, the corresponding second mirror node is controlled to respond to the second target request, wherein the second target request is a data request requesting the first main node to store data.

根据本发明的第二方面,提供了一种数据备份装置,应用于Greenplum数据库系统中的管理节点,Greenplum数据库系统还包括主机房与备用机房,主机房和备用机房中的各机器均包括第一主节点与第二镜像节点,第二镜像节点为其他机器的第一主节点的镜像节点,装置包括:According to a second aspect of the present invention, a data backup device is provided, which is applied to a management node in a Greenplum database system. The Greenplum database system further includes a main computer room and a backup computer room. Each machine in the main computer room and the backup computer room includes a first master node and a second mirror node. The second mirror node is a mirror node of the first master node of other machines. The device includes:

第一接收模块,用于接收第一响应信号,第一响应信号用于表征主机房中各机器与备用机房中的各机器的逻辑复制关系已建立,逻辑复制关系用于表示备用机房内各机器的第一主节点与主机房内各机器的第二镜像节点之间的映射关系;A first receiving module is used to receive a first response signal, the first response signal is used to indicate that a logical replication relationship between each machine in the main computer room and each machine in the standby computer room has been established, and the logical replication relationship is used to indicate a mapping relationship between a first master node of each machine in the standby computer room and a second mirror node of each machine in the main computer room;

第一处理模块,基于逻辑复制关系,将主机房内各机器的第二镜像节点中的数据变化进行解码,按照解码结果对备用机房内第一主节点中的数据进行修改;The first processing module decodes the data changes in the second mirror node of each machine in the main computer room based on the logical replication relationship, and modifies the data in the first main node in the standby computer room according to the decoding result;

第一控制模块,若主机房发生故障,控制备用机房内第一主节点响应第一目标请求,其中,第一目标请求为请求主机房中存储数据的数据请求。The first control module controls the first master node in the standby computer room to respond to the first target request if a failure occurs in the main computer room, wherein the first target request is a data request for storing data in the main computer room.

可选地,数据备份装置,还包括:Optionally, the data backup device further includes:

第二接收模块,用于接收第一指令,第一指令为建立备用机房内各机器与主机房内各机器之间逻辑复制关系的信号;A second receiving module is used to receive a first instruction, where the first instruction is a signal for establishing a logical replication relationship between each machine in the standby computer room and each machine in the main computer room;

第二处理模块,响应于第一指令,将主机房内各机器的第二镜像节点中的数据对象及其变化复制至对应的备用机房内各机器的第一主节点中。The second processing module, in response to the first instruction, copies the data objects and changes thereof in the second mirror nodes of each machine in the main computer room to the first main nodes of each machine in the corresponding standby computer room.

可选地,数据备份装置,还包括:Optionally, the data backup device further includes:

第三接收模块,用于接收第一新增信号,第一新增信号为主机房内各机器的第二镜像节点中数据的新增信号;A third receiving module is used to receive a first newly added signal, where the first newly added signal is a newly added signal of data in the second mirror node of each machine in the main computer room;

第三处理模块,响应于第一新增信号,将主机房内各机器的第二镜像节点中的新增数据复制到至备用机房内第一主节点中。The third processing module, in response to the first new signal, copies the new data in the second mirror node of each machine in the main computer room to the first main node in the standby computer room.

根据本发明的第三方面,提供了一种电子设备,包括:According to a third aspect of the present invention, there is provided an electronic device, comprising:

处理器;以及Processor; and

存储程序的存储器,Memory for storing programs,

其中,程序包括指令,指令在由处理器执行时使处理器执行根据本发明第一方面中任一项的方法。The program includes instructions, and when the instructions are executed by a processor, the processor executes any one of the methods according to the first aspect of the present invention.

根据本发明的第四方面,提供了一种存储有计算机指令的非瞬时计算机可读存储介质,其中,计算机指令用于使计算机执行根据本发明第一方面中任一项的方法。According to a fourth aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute any method according to the first aspect of the present invention.

本申请实施例中提供的一个或多个技术方案,基于备用机房内第一主节点与主机房中第二镜像节点之间建立的逻辑复制关系,在主机房内第二镜像节点的数据发生变化时进行数据变化的解码,按照解码结果在备用机房的第一主节点上执行增删和修改,实现实时、增量的备份效果,无需每次备份所有的历史数据。且基于第二镜像节点作为第一主节点的镜像节点不需要参与运算的特点,将主机房中第二镜像节点中的数据对象及其变化复制到备用机房内的第一主节点中,不会对主机房中的第一主节点的性能造成影响。本方案充分利用Greenplum数据库的特性以及逻辑复制算法,实现了实时的跨机房备份效果,将机房级别的故障造成的数据损失最小化。One or more technical solutions provided in the embodiments of the present application are based on the logical replication relationship established between the first master node in the standby computer room and the second mirror node in the main computer room. When the data of the second mirror node in the main computer room changes, the data change is decoded, and the addition, deletion and modification are performed on the first master node in the standby computer room according to the decoding result, so as to achieve a real-time, incremental backup effect without backing up all historical data every time. And based on the characteristic that the second mirror node, as a mirror node of the first master node, does not need to participate in the operation, the data objects and their changes in the second mirror node in the main computer room are copied to the first master node in the standby computer room, which will not affect the performance of the first master node in the main computer room. This solution makes full use of the characteristics of the Greenplum database and the logical replication algorithm to achieve a real-time cross-computer room backup effect and minimize the data loss caused by computer room-level failures.

附图说明BRIEF DESCRIPTION OF THE DRAWINGS

在下面结合附图对于示例性实施例的描述中,本发明的更多细节、特征和优点被公开,在附图中:Further details, features and advantages of the invention are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

图1示出了根据本发明示例性实施例的数据备份方法的流程图;FIG1 shows a flow chart of a data backup method according to an exemplary embodiment of the present invention;

图2示出了根据本发明示例性实施例的数据备份方法的跨机房结构图;FIG2 shows a cross-machine room structure diagram of a data backup method according to an exemplary embodiment of the present invention;

图3示出了根据本发明示例性实施例的数据备份装置的示意性框图;FIG3 shows a schematic block diagram of a data backup device according to an exemplary embodiment of the present invention;

图4示出了能够用于实现本发明的实施例的示例性电子设备的结构框图。FIG. 4 shows a block diagram of an exemplary electronic device that can be used to implement an embodiment of the present invention.

具体实施方式Detailed ways

下面将参照附图更详细地描述本发明的实施例。虽然附图中显示了本发明的某些实施例,然而应当理解的是,本发明可以通过各种形式来实现,而且不应该被解释为限于这里阐述的实施例,相反提供这些实施例是为了更加透彻和完整地理解本发明。应当理解的是,本发明的附图及实施例仅用于示例性作用,并非用于限制本发明的保护范围。Embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are only for exemplary purposes and are not intended to limit the scope of protection of the present invention.

应当理解,本发明的方法实施方式中记载的各个步骤可以按照不同的顺序执行,和/或并行执行。此外,方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本发明的范围在此方面不受限制。It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and/or in parallel. In addition, the method embodiments may include additional steps and/or omit the steps shown. The scope of the present invention is not limited in this respect.

本文使用的术语“包括”及其变形是开放性包括,即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”;术语“另一实施例”表示“至少一个另外的实施例”;术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。需要注意,本发明中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元进行区分,并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

需要注意,本发明中提及的“一个”、“多个”的修饰是示意性而非限制性的,本领域技术人员应当理解,除非在上下文另有明确指出,否则应该理解为“一个或多个”。It should be noted that the modifications of "one" and "plurality" mentioned in the present invention are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

本发明实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的,而并不是用于对这些消息或信息的范围进行限制。The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

以下参照附图描述本发明的方案,通过具体的实施例及其应用场景对本申请实施例提供的技术方案进行详细地说明。The following describes the solution of the present invention with reference to the accompanying drawings, and explains the technical solution provided by the embodiments of the present application in detail through specific embodiments and their application scenarios.

现有的跨机房备份场景中,常采用全量备份的方式,在每次备份都需要先清空再同步历史全量数据,因此在面对大数据量时,备份的数据同步时间会非常长。还会遗漏备份期间数据的变动,使得备份机房无法保持与主机房数据的一致性。故而现有的跨机房备份的方案无法满足实时备份的目的。In existing cross-datacenter backup scenarios, full backup is often used. Each backup requires clearing and then synchronizing the historical full data. Therefore, when facing a large amount of data, the backup data synchronization time will be very long. It will also miss the changes in data during the backup, making it impossible for the backup data center to maintain data consistency with the main data center. Therefore, the existing cross-datacenter backup solution cannot meet the purpose of real-time backup.

有鉴于此,本发明提供了一种数据备份方法、装置、电子设备及可读存储介质,其中,该数据备份方法应用于任意具备数据备份功能的电子设备中,该电子设备包括但不限于:个人移动终端设备、计算机或服务器设备等等。In view of this, the present invention provides a data backup method, device, electronic device and readable storage medium, wherein the data backup method is applied to any electronic device with data backup function, and the electronic device includes but is not limited to: personal mobile terminal device, computer or server device, etc.

如图1所示,图1是本申请实施例提供的一种数据备份方法的流程示意图,该数据备份方法应用于Greenplum数据库系统中的管理节点,Greenplum数据库系统还包括主机房与备用机房,所述主机房和所述备用机房中的各机器均包括第一主节点与第二镜像节点,所述第二镜像节点为其他机器的第一主节点的镜像节点,数据备份方法可以包括如下步骤S101~步骤S103:As shown in FIG. 1 , FIG. 1 is a flow chart of a data backup method provided in an embodiment of the present application. The data backup method is applied to a management node in a Greenplum database system. The Greenplum database system further includes a main computer room and a backup computer room. Each machine in the main computer room and the backup computer room includes a first master node and a second mirror node. The second mirror node is a mirror node of the first master node of other machines. The data backup method may include the following steps S101 to S103:

S101,接收第一响应信号,所述第一响应信号用于表征所述主机房中各机器与所述备用机房中的各机器的逻辑复制关系已建立,所述逻辑复制关系用于表示所述备用机房内各机器的第一主节点与所述主机房内各机器的第二镜像节点之间的映射关系。S101, receiving a first response signal, wherein the first response signal is used to indicate that a logical replication relationship between each machine in the main computer room and each machine in the backup computer room has been established, and the logical replication relationship is used to indicate a mapping relationship between a first master node of each machine in the backup computer room and a second mirror node of each machine in the main computer room.

作为一种实现方式,Greenplum数据库系统采用分布式数据库集群架构,便于提供大量数据量存储、计算功能。Greenplum数据库系统中包括主机房与备用机房,主机房与备用机房内均包含若干个机器,每个机器中均包括第一主节点和第二镜像节点,第二镜像节点作为其他机器内第一主节点的镜像节点,用于复制对应第一主节点中的数据。且第一主节点与第二镜像节点不在同一机器内,则可以保障在机器故障时,作为当前故障机器内的第一主节点,其对应的第二镜像节点可以替代第一主节点响应针对第一主节点的数据请求。As an implementation method, the Greenplum database system adopts a distributed database cluster architecture to facilitate the provision of large amounts of data storage and computing functions. The Greenplum database system includes a main computer room and a backup computer room. Both the main computer room and the backup computer room contain several machines. Each machine includes a first master node and a second mirror node. The second mirror node serves as a mirror node of the first master node in other machines and is used to replicate the data in the corresponding first master node. If the first master node and the second mirror node are not in the same machine, it can be ensured that when a machine fails, the corresponding second mirror node, which is the first master node in the current failed machine, can replace the first master node to respond to data requests for the first master node.

预先建立Greenplum数据库系统中主机房中各机器与所述备用机房中的各机器的逻辑复制关系,使得主机房内各机器的第二镜像节点中的数据对象及其变化复制到备用机房内各机器的第一主节点中。同时Greenplum数据库系统中的管理节点接收到主机房中各机器与所述备用机房中的各机器的逻辑复制关系已建立的响应信号。A logical replication relationship between each machine in the main machine room and each machine in the standby machine room in the Greenplum database system is established in advance, so that the data objects and changes in the second mirror node of each machine in the main machine room are replicated to the first master node of each machine in the standby machine room. At the same time, the management node in the Greenplum database system receives a response signal that the logical replication relationship between each machine in the main machine room and each machine in the standby machine room has been established.

作为一种具体的实现方式,如图2所示,采用机房A作为主机房,机房B作为机房A的备用机房,用于在机房A发生故障时,Greenplum数据库系统中的管理节点控制机房B响应针对机房A中存储数据的数据请求,其中,主机房和备用机房也称为互联网数据中心IDC。As a specific implementation method, as shown in Figure 2, computer room A is used as the main computer room, and computer room B is used as the backup computer room of computer room A. When computer room A fails, the management node in the Greenplum database system controls computer room B to respond to data requests for data stored in computer room A. The main computer room and the backup computer room are also called Internet data centers IDCs.

机房A与机房B的集群数据分散的存储在机器1、机器2……机器n等多台机器上,每台机器上有多个postgresql数据实例,每个机器中的postgresql数据实例分为第一主节点primary和第二镜像节点mirror,第一主节点primary是实际执行计算的节点,第二镜像节点mirror是其他机器第一主节点的镜像节点。The cluster data of computer room A and computer room B are stored in multiple machines such as machine 1, machine 2...machine n. There are multiple postgresql data instances on each machine. The postgresql data instances in each machine are divided into the first primary node primary and the second mirror node mirror. The first primary node primary is the node that actually executes the calculation, and the second mirror node mirror is the mirror node of the first primary node of other machines.

S102,基于所述逻辑复制关系,将所述主机房内各机器的第二镜像节点中的数据变化进行解码,按照解码结果对所述备用机房内第一主节点中的数据进行修改。S102, based on the logical replication relationship, decode the data changes in the second mirror node of each machine in the main computer room, and modify the data in the first main node in the standby computer room according to the decoding result.

作为一种实现方式,基于备用机房内各机器的第一主节点与主机房内各机器第二镜像节点的逻辑复制关系,在Greenplum数据库系统中的管理节点处生成逻辑复制关系的配置文件,在进行逻辑复制时,读取配置文件中记录的逻辑复制关系进行逻辑复制。As an implementation method, based on the logical replication relationship between the first master node of each machine in the standby computer room and the second mirror node of each machine in the main computer room, a configuration file of the logical replication relationship is generated at the management node in the Greenplum database system. When performing logical replication, the logical replication relationship recorded in the configuration file is read to perform logical replication.

在Greenplum数据库系统中的管理节点获知主机房内第二镜像节点的数据发生变化时,对该变化进行解码,按照解码结果在所述备用机房的第一主节点上执行修改,使得备用机房的第一主节点中的数据与主机房的第二镜像节点中的数据实时保持一致,达到了跨机房实时备份的复制效果。不需要重新备份全部数据,只根据数据变化进行备份的设置,也达到了跨机房增量备份的复制效果。其中,备用机房的第一主节点上执行修改会按照与所述主机房内第二镜像节点中数据变化的时间顺序进行,进而保证逻辑复制的一致性。When the management node in the Greenplum database system learns that the data of the second mirror node in the main computer room has changed, it decodes the change and performs modifications on the first master node in the standby computer room according to the decoding result, so that the data in the first master node in the standby computer room is kept consistent with the data in the second mirror node in the main computer room in real time, achieving the replication effect of real-time backup across computer rooms. There is no need to back up all the data again, and the backup is only performed based on data changes, which also achieves the replication effect of incremental backup across computer rooms. Among them, the modifications performed on the first master node in the standby computer room will be performed in the time sequence of the data changes in the second mirror node in the main computer room, thereby ensuring the consistency of logical replication.

作为一种具体的实现方式,在发布机房A第二镜像节点mirror的节点数据时,使用机房B的单机postgresql数据库进行订阅,实现第二镜像节点mirror和单机postgresql数据库之间的逻辑复制,应对每个第二镜像节点mirror,采用不重复的postgressql数据库进行订阅。在机房A第二镜像节点mirror的节点数据发生变化时,Greenplum数据库系统中的管理节点对第二镜像节点中的数据变化进行解码,并根据配置文件确定对应的备用机房中的第一主节点,按照解码结果对所述备用机房内第一主节点中的数据进行修改。As a specific implementation method, when publishing the node data of the second mirror node mirror in computer room A, the stand-alone postgresql database in computer room B is used for subscription to realize the logical replication between the second mirror node mirror and the stand-alone postgresql database. For each second mirror node mirror, a non-repeated postgressql database is used for subscription. When the node data of the second mirror node mirror in computer room A changes, the management node in the Greenplum database system decodes the data changes in the second mirror node, determines the first master node in the corresponding standby computer room according to the configuration file, and modifies the data in the first master node in the standby computer room according to the decoding result.

其中,备用机房的机器数目不小于主机房中的机器数目,主机房各个机器分别对应一个备用机房中的机器。在进行跨机房备份时,备用机房需要能够存储下主机房所有数据的机器数目。The number of machines in the standby computer room is not less than the number of machines in the main computer room, and each machine in the main computer room corresponds to a machine in the standby computer room. When performing cross-computer room backup, the standby computer room needs to be able to store the number of machines of all the data in the main computer room.

S103,若所述主机房发生故障,控制所述备用机房内第一主节点响应第一目标请求,其中,所述第一目标请求为请求所述主机房中存储数据的数据请求。S103: If a failure occurs in the main computer room, control the first master node in the backup computer room to respond to a first target request, wherein the first target request is a data request for storing data in the main computer room.

作为一种实现方式,Greenplum数据库系统中的管理节点接收到主机房故障的指令,则控制备用机房中的第一主节点替代主机房响应第一目标请求,保障Greenplum数据库系统的正常运行。其中,主机房故障为主机房出现硬件故障、软件故障、病毒故障、人为故障和假故障。As an implementation method, when the management node in the Greenplum database system receives an instruction of a main computer room failure, it controls the first master node in the standby computer room to replace the main computer room to respond to the first target request, thereby ensuring the normal operation of the Greenplum database system. The main computer room failure refers to hardware failure, software failure, virus failure, human failure and false failure in the main computer room.

在本申请的一个实施方式中,在所述接收第一响应信号前,还包括:In one embodiment of the present application, before receiving the first response signal, the method further includes:

S101a,接收第一指令,所述第一指令为建立所述备用机房内各机器与所述主机房内各机器之间逻辑复制关系的信号;S101a, receiving a first instruction, where the first instruction is a signal for establishing a logical replication relationship between each machine in the standby computer room and each machine in the main computer room;

S101b,响应于所述第一指令,将所述主机房内各机器的第二镜像节点中的数据对象及其变化复制至对应的所述备用机房内各机器的第一主节点中。S101b, in response to the first instruction, copy the data objects and changes thereof in the second mirror node of each machine in the main computer room to the corresponding first main node of each machine in the standby computer room.

作为一种实现方式,建立备用机房内各机器的第一主节点与主机房内各机器第二镜像节点的逻辑复制关系,其中,逻辑复制(Logical Replication)是一种根据数据对象的复制标识(Replica Identity)复制数据对象及其变化的方法。Greenplum数据库系统中的管理节点接收到建立备用机房内各机器与主机房内各机器之间逻辑复制关系的信号后,将主机房内各机器的第二镜像节点中的数据对象及其变化复制至对应的所述备用机房内各机器的第一主节点中。As an implementation method, a logical replication relationship is established between the first master node of each machine in the standby computer room and the second mirror node of each machine in the main computer room, wherein logical replication is a method of replicating data objects and their changes according to the replica identity of the data object. After receiving the signal for establishing the logical replication relationship between each machine in the standby computer room and each machine in the main computer room, the management node in the Greenplum database system replicates the data objects and their changes in the second mirror node of each machine in the main computer room to the corresponding first master node of each machine in the standby computer room.

作为一种具体的实现方式,如图2所示,建立备用机房内各机器的第一主节点与主机房内各机器第二镜像节点的逻辑复制关系,其中,第一主节点pg1为在机房B的机器2上搭建单机postgresql数据库,作为该机器的第一主节点,第一主节点pg2为在机房B的机器n上搭建单机postgresql数据库,作为该机器的第一主节点。通过Greenplum数据库系统中的管理节点,按照逐一循环的方式设置机房A中的第二镜像节点mirror1与机房B中的第一主节点pg1之间的逻辑复制关系,机房A中的第二镜像节点mirror2与机房B中的第一主节点pg2之间逻辑复制关系……,以此类推。也可以通过Greenplum数据库系统中的管理节点,按照随机部署的方式设置,根据设置生成逻辑复制关系的配置文件,在进行逻辑复制时,读取配置文件中记录的逻辑复制关系进行逻辑复制。As a specific implementation method, as shown in FIG2 , a logical replication relationship is established between the first master node of each machine in the standby computer room and the second mirror node of each machine in the main computer room, wherein the first master node pg1 is a stand-alone postgresql database built on machine 2 in computer room B, as the first master node of the machine, and the first master node pg2 is a stand-alone postgresql database built on machine n in computer room B, as the first master node of the machine. Through the management node in the Greenplum database system, the logical replication relationship between the second mirror node mirror1 in computer room A and the first master node pg1 in computer room B is set in a one-to-one cycle manner, and the logical replication relationship between the second mirror node mirror2 in computer room A and the first master node pg2 in computer room B is set..., and so on. It is also possible to set it in a random deployment manner through the management node in the Greenplum database system, generate a configuration file for the logical replication relationship according to the setting, and read the logical replication relationship recorded in the configuration file for logical replication when performing logical replication.

在本申请的一个实施方式中,在所述接收第一响应信号后,还包括:In one embodiment of the present application, after receiving the first response signal, the method further includes:

S101c,接收第一新增信号,所述第一新增信号为所述主机房内各机器的第二镜像节点中数据的新增信号;S101c, receiving a first newly added signal, where the first newly added signal is a signal of newly added data in the second mirror node of each machine in the main computer room;

S101d,响应于所述第一新增信号,将所述主机房内各机器的第二镜像节点中的新增数据复制到至所述备用机房内第一主节点中。S101d, in response to the first new signal, copy the new data in the second mirror node of each machine in the main computer room to the first main node in the standby computer room.

作为一种实现方式,基于备用机房内各机器的第一主节点与主机房内各机器第二镜像节点的逻辑复制关系,在主机房内各机器的第二镜像节点中数据进行新增时,产生新增信号时,Greenplum数据库系统中的管理节点将新增信号对应的数据复制到对应的备用机房的第一主节点中,不需要复制全部数据,避免备份效率低下的问题。As an implementation method, based on the logical replication relationship between the first master node of each machine in the standby computer room and the second mirror node of each machine in the main computer room, when data is added to the second mirror node of each machine in the main computer room and a new signal is generated, the management node in the Greenplum database system copies the data corresponding to the new signal to the first master node of the corresponding standby computer room, without the need to copy all the data, thus avoiding the problem of low backup efficiency.

采用增量备份的方式将新增信号对应的数据复制到对应的备用机房的第一主节点中,增量备份(incremental backup)是指在一次全备份或上一次增量备份后,以后每次的备份只需备份与前一次相比增加或者被修改的内容。这种备份方式实现了实时的、增量的数据备份效果,相比于全量备份无需每次备份所有历史数据,因此备份效率有保障,在灾难恢复时不会造成数据损失。相较于全量备份在实时性、效率和恢复数据损失方面都更优。同时也会减轻逻辑复制时的工作量,提高备份效率。The data corresponding to the newly added signal is copied to the first master node of the corresponding standby computer room by using the incremental backup method. Incremental backup means that after a full backup or the last incremental backup, each subsequent backup only needs to back up the content that has been added or modified compared to the previous one. This backup method achieves real-time, incremental data backup. Compared with full backup, it does not need to back up all historical data every time, so the backup efficiency is guaranteed and no data loss will occur during disaster recovery. Compared with full backup, it is better in terms of real-time performance, efficiency and recovery of data loss. At the same time, it will also reduce the workload during logical replication and improve backup efficiency.

在本申请的一个实施方式中,在所述接收第一响应信号前,还包括:In one embodiment of the present application, before receiving the first response signal, the method further includes:

S101e,接收第二指令,所述第二指令为建立所述第二镜像节点与所述第一主节点之间复制关系的信号;S101e, receiving a second instruction, where the second instruction is a signal for establishing a replication relationship between the second mirror node and the first master node;

S101f,响应于所述第一指令,将所述主机房内各机器第一主节点中的数据复制至对应的所述第二镜像节点中;S101f, in response to the first instruction, copying the data in the first main node of each machine in the main computer room to the corresponding second mirror node;

S101g,接收第二新增信号,所述第二新增信号为所述主机房内各机器的第一主节点中数据的变更信号;S101g, receiving a second newly added signal, where the second newly added signal is a signal of data change in the first master node of each machine in the main computer room;

S101h,响应于所述第二新增信号,将与所述变更信号对应的数据对象及其变化复制到所述主机房内各机器的第二镜像主节点中。S101h: In response to the second added signal, the data object corresponding to the change signal and its changes are copied to the second mirror master node of each machine in the main computer room.

作为一种实现方式,通过流复制的方法同步保持第二镜像节点mirror与第一主节点primary的一致性,流复制的方式就是第一主节点不断地发送事务日志xlog,作为镜像节点的第二镜像节点不断地接收机房A的事务日志xlog,并不断地回放保持数据同步的过程。As an implementation method, the consistency of the second mirror node mirror and the first primary node primary is synchronized through the streaming replication method. The streaming replication method is that the first primary node continuously sends the transaction log xlog, and the second mirror node as the mirror node continuously receives the transaction log xlog of computer room A, and continuously replays the process of maintaining data synchronization.

由于第二镜像节点mirror具有和第一主节点primary相同的数据和事务日志xlog,而基于第二镜像节点mirror的发布不会造成数据损失。第二镜像节点mirror在集群运行正常时不会进行运算,仅作为第一主节点primary的镜像使用,因此基于第二镜像节点mirror的发布基本不会对原集群的性能产生影响。并且为了保障第一主节点故障时,能够控制其对应的第二镜像节点响应数据请求的效果,第一主节点与其对应的第二镜像节点不在同一机器内。Since the second mirror node mirror has the same data and transaction log xlog as the first primary node primary, the release based on the second mirror node mirror will not cause data loss. The second mirror node mirror will not perform operations when the cluster is running normally, and is only used as a mirror of the first primary node primary. Therefore, the release based on the second mirror node mirror will basically have no impact on the performance of the original cluster. In addition, in order to ensure that when the first primary node fails, the corresponding second mirror node can be controlled to respond to data requests, the first primary node and its corresponding second mirror node are not in the same machine.

作为一种具体的实现方式,通过Greenplum数据库系统中的管理节点按照逐一循环的方式设置机器1中的第一主节点primary1与机器2中的第二镜像节点mirror2之间的镜像关系,机器2中的第一主节点primary2与机器3中的第二镜像节点mirror3之间的镜像关系……,也可以通过Greenplum数据库系统中的管理节点按照随机部署的方式设置机器1中的第一主节点primary与其他空闲机器中的第二镜像节点mirror之间的镜像关系,达到第一主节点与第二镜像节点不在同一机器内,保障在任一机器故障时,作为当前故障机器内的第一主节点,其对应的第二镜像节点可以替代第一主节点响应针对第一主节点的数据请求。As a specific implementation method, the management node in the Greenplum database system sets the mirroring relationship between the first primary node primary1 in machine 1 and the second mirror node mirror2 in machine 2, the mirroring relationship between the first primary node primary2 in machine 2 and the second mirror node mirror3 in machine 3, and so on in a one-to-one cycle manner. The management node in the Greenplum database system can also set the mirroring relationship between the first primary node primary in machine 1 and the second mirror nodes mirror in other idle machines in a random deployment manner, so that the first primary node and the second mirror node are not in the same machine, ensuring that when any machine fails, the corresponding second mirror node as the first primary node in the current failed machine can replace the first primary node to respond to data requests for the first primary node.

在本申请的一个实施方式中,在所述响应于所述第一指令,将所述主机房内各机器第一主节点中的数据复制至对应的所述第二镜像节点中后,还包括:In one embodiment of the present application, after the data in the first main node of each machine in the main computer room is copied to the corresponding second mirror node in response to the first instruction, the method further includes:

S101i,若所述主机房内任一机器的第一主节点发生故障,控制对应的所述第二镜像节点响应第二目标请求,其中,所述第二目标请求为请求所述第一主节点存储数据的数据请求。S101i, if the first main node of any machine in the main computer room fails, control the corresponding second mirror node to respond to the second target request, wherein the second target request is a data request requesting the first main node to store data.

作为一种具体的实现方式,在机房A的第一主节点primary出现故障时,第二镜像节点mirror作为第一主节点primary的镜像节点,由Greenplum数据库系统中的管理节点控制其第二镜像节点mirror代替第一主节点primary,响应请求第一主节点存储数据的数据请求。As a specific implementation method, when the first primary node primary in computer room A fails, the second mirror node mirror serves as the mirror node of the first primary node primary, and the management node in the Greenplum database system controls its second mirror node mirror to replace the first primary node primary and respond to data requests requesting the first primary node to store data.

本申请实施例中提供的数据备份方法,基于备用机房内第一主节点与主机房中第二镜像节点之间建立的逻辑复制关系,在主机房内第二镜像节点的数据发生变化时进行数据变化的解码,按照解码结果在备用机房的第一主节点上执行增删和修改,实现实时、增量的备份效果,无需每次备份所有的历史数据。且基于第二镜像节点作为第一主节点的镜像节点不需要参与运算的特点,将主机房中第二镜像节点中的数据对象及其变化复制到备用机房内的第一主节点中,不会对主机房中的第一主节点的性能造成影响。本方案充分利用Greenplum数据库的特性以及逻辑复制算法,实现了实时的跨机房备份效果,将机房级别的故障造成的数据损失最小化。The data backup method provided in the embodiment of the present application is based on the logical replication relationship established between the first master node in the standby computer room and the second mirror node in the main computer room. When the data of the second mirror node in the main computer room changes, the data change is decoded, and the addition, deletion and modification are performed on the first master node in the standby computer room according to the decoding result, so as to achieve a real-time, incremental backup effect without backing up all historical data every time. And based on the characteristic that the second mirror node, as a mirror node of the first master node, does not need to participate in the operation, the data objects and their changes in the second mirror node in the main computer room are copied to the first master node in the standby computer room, which will not affect the performance of the first master node in the main computer room. This solution makes full use of the characteristics of the Greenplum database and the logical replication algorithm to achieve a real-time cross-computer room backup effect and minimize the data loss caused by computer room-level failures.

与上述实施例相对应,参见图3,本申请实施例还提供一种数据备份装置300,数据备份装置300应用于Greenplum数据库系统中的管理节点,所述Greenplum数据库系统还包括主机房与备用机房,所述主机房和所述备用机房中的各机器均包括第一主节点与第二镜像节点,所述第二镜像节点为其他机器的第一主节点的镜像节点,数据备份装置300包括:Corresponding to the above embodiment, referring to FIG. 3 , the embodiment of the present application further provides a data backup device 300, which is applied to a management node in a Greenplum database system. The Greenplum database system further includes a main computer room and a backup computer room. Each machine in the main computer room and the backup computer room includes a first master node and a second mirror node. The second mirror node is a mirror node of the first master node of other machines. The data backup device 300 includes:

第一接收模块301,用于接收第一响应信号,所述第一响应信号用于表征所述主机房中各机器与所述备用机房中的各机器的逻辑复制关系已建立,所述逻辑复制关系用于表示所述备用机房内各机器的第一主节点与所述主机房内各机器的第二镜像节点之间的映射关系;A first receiving module 301 is used to receive a first response signal, wherein the first response signal is used to indicate that a logical replication relationship between each machine in the main computer room and each machine in the standby computer room has been established, and the logical replication relationship is used to indicate a mapping relationship between a first primary node of each machine in the standby computer room and a second mirror node of each machine in the main computer room;

第一处理模块302,基于所述逻辑复制关系,将所述主机房内各机器的第二镜像节点中的数据变化进行解码,按照解码结果对所述备用机房内第一主节点中的数据进行修改;The first processing module 302 decodes the data changes in the second mirror node of each machine in the main computer room based on the logical replication relationship, and modifies the data in the first main node in the standby computer room according to the decoding result;

第一控制模块303,若所述主机房发生故障,控制所述备用机房内第一主节点响应第一目标请求,其中,所述第一目标请求为请求所述主机房中存储数据的数据请求。The first control module 303 controls the first master node in the standby computer room to respond to a first target request if the main computer room fails, wherein the first target request is a data request for storing data in the main computer room.

可选地,数据备份装置300,还包括:Optionally, the data backup device 300 further includes:

第二接收模块304,用于接收第一指令,所述第一指令为建立所述备用机房内各机器与所述主机房内各机器之间逻辑复制关系的信号;The second receiving module 304 is used to receive a first instruction, where the first instruction is a signal for establishing a logical replication relationship between each machine in the standby computer room and each machine in the main computer room;

第二处理模块305,响应于所述第一指令,将所述主机房内各机器的第二镜像节点中的数据对象及其变化复制至对应的所述备用机房内各机器的第一主节点中。The second processing module 305, in response to the first instruction, copies the data objects and changes thereof in the second mirror node of each machine in the main computer room to the first main node of each machine in the corresponding standby computer room.

可选地,数据备份装置300,还包括:Optionally, the data backup device 300 further includes:

第三接收模块306,用于接收第一新增信号,所述第一新增信号为所述主机房内各机器的第二镜像节点中数据的新增信号;The third receiving module 306 is used to receive a first newly added signal, where the first newly added signal is a newly added signal of data in the second mirror node of each machine in the main computer room;

第三处理模块307,响应于所述第一新增信号,将所述主机房内各机器的第二镜像节点中的新增数据复制到至所述备用机房内第一主节点中。The third processing module 307, in response to the first new signal, copies the new data in the second mirror node of each machine in the main computer room to the first main node in the standby computer room.

可选地,数据备份装置300,还包括:Optionally, the data backup device 300 further includes:

第四接收模块308,用于接收第二指令,所述第二指令为建立所述第二镜像节点与所述第一主节点之间复制关系的信号;A fourth receiving module 308, configured to receive a second instruction, where the second instruction is a signal for establishing a replication relationship between the second mirror node and the first master node;

第四处理模块309,响应于所述第一指令,将所述主机房内各机器第一主节点中的数据复制至对应的所述第二镜像节点中;The fourth processing module 309, in response to the first instruction, copies the data in the first main node of each machine in the main computer room to the corresponding second mirror node;

第五接收模块310,用于接收第二新增信号,所述第二新增信号为所述主机房内各机器的第一主节点中数据的变更信号;A fifth receiving module 310 is used to receive a second newly added signal, where the second newly added signal is a data change signal in the first master node of each machine in the main computer room;

第五处理模块311,响应于所述第二新增信号,将与所述变更信号对应的数据对象及其变化复制到所述主机房内各机器的第二镜像主节点中。The fifth processing module 311, in response to the second added signal, copies the data object corresponding to the change signal and its changes to the second mirror master node of each machine in the main computer room.

可选地,数据备份装置300,还包括:Optionally, the data backup device 300 further includes:

第二控制模块312,若所述主机房内任一机器的第一主节点发生故障,控制对应的所述第二镜像节点响应第二目标请求,其中,所述第二目标请求为请求所述第一主节点存储数据的数据请求。The second control module 312 controls the corresponding second mirror node to respond to a second target request if a first main node of any machine in the main computer room fails, wherein the second target request is a data request for requesting the first main node to store data.

本申请实施例中提供的数据备份装置,基于备用机房内第一主节点与主机房中第二镜像节点之间建立的逻辑复制关系,在主机房内第二镜像节点的数据发生变化时进行数据变化的解码,按照解码结果在备用机房的第一主节点上执行增删和修改,实现实时、增量的备份效果,无需每次备份所有的历史数据。且基于第二镜像节点作为第一主节点的镜像节点不需要参与运算的特点,将主机房中第二镜像节点中的数据对象及其变化复制到备用机房内的第一主节点中,不会对主机房中的第一主节点的性能造成影响。本方案充分利用Greenplum数据库的特性以及逻辑复制算法,实现了实时的跨机房备份效果,将机房级别的故障造成的数据损失最小化。The data backup device provided in the embodiment of the present application is based on the logical replication relationship established between the first master node in the standby computer room and the second mirror node in the main computer room. When the data of the second mirror node in the main computer room changes, the data change is decoded, and the addition, deletion and modification are performed on the first master node in the standby computer room according to the decoding result, so as to achieve a real-time, incremental backup effect without backing up all historical data every time. And based on the characteristic that the second mirror node, as a mirror node of the first master node, does not need to participate in the operation, the data objects and their changes in the second mirror node in the main computer room are copied to the first master node in the standby computer room, which will not affect the performance of the first master node in the main computer room. This solution makes full use of the characteristics of the Greenplum database and the logical replication algorithm to achieve a real-time cross-computer room backup effect and minimize the data loss caused by computer room-level failures.

本发明示例性实施例还提供一种电子设备,包括:至少一个处理器;以及与至少一个处理器通信连接的存储器。存储器存储有能够被至少一个处理器执行的计算机程序,计算机程序在被至少一个处理器执行时用于使电子设备执行根据本发明实施例的方法。The exemplary embodiment of the present invention also provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to cause the electronic device to perform the method according to the embodiment of the present invention when executed by the at least one processor.

本发明示例性实施例还提供一种存储有计算机程序的非瞬时计算机可读存储介质,其中,计算机程序在被计算机的处理器执行时用于使计算机执行根据本发明实施例的方法。Exemplary embodiments of the present invention also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform a method according to an embodiment of the present invention.

本发明示例性实施例还提供一种计算机程序产品,包括计算机程序,其中,计算机程序在被计算机的处理器执行时用于使计算机执行根据本发明实施例的方法。An exemplary embodiment of the present invention further provides a computer program product, including a computer program, wherein the computer program is used to cause the computer to perform a method according to an embodiment of the present invention when executed by a processor of a computer.

参考图4,现将描述可以作为本发明的服务器或客户端的电子设备400的结构框图,其是可以应用于本发明的各方面的硬件设备的示例。电子设备旨在表示各种形式的数字电子的计算机设备,诸如,膝上型计算机、台式计算机、工作台、个人数字助理、服务器、刀片式服务器、大型计算机、和其它适合的计算机。电子设备还可以表示各种形式的移动装置,诸如,个人数字处理、蜂窝电话、智能电话、可穿戴设备和其它类似的计算装置。本文所示的部件、它们的连接和关系、以及它们的功能仅仅作为示例,并且不意在限制本文中描述的和/或者要求的本发明的实现。With reference to Fig. 4, the structural block diagram of the electronic device 400 that can be used as the server or client of the present invention will now be described, which is an example of the hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only used as examples, and are not intended to limit the implementation of the present invention described and/or required herein.

如图4所示,电子设备400包括计算单元401,其可以根据存储在只读存储器(ROM)402中的计算机程序或者从存储单元408加载到随机访问存储器(RAM)403中的计算机程序,来执行各种适当的动作和处理。在RAM 403中,还可存储电子设备400操作所需的各种程序和数据。计算单元401、ROM 402以及RAM 403通过总线404彼此相连。输入/输出(I/O)接口405也连接至总线404。As shown in Figure 4, electronic device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 402 or a computer program loaded from a storage unit 408 into a random access memory (RAM) 403. In RAM 403, various programs and data required for the operation of electronic device 400 can also be stored. Computing unit 401, ROM 402 and RAM 403 are connected to each other via bus 404. Input/output (I/O) interface 405 is also connected to bus 404.

电子设备400中的多个部件连接至I/O接口405,包括:输入单元406、输出单元407、存储单元408以及通信单元409。输入单元406可以是能向电子设备400输入信息的任何类型的设备,输入单元406可以接收输入的数字或字符信息,以及产生与电子设备的用户设置和/或功能控制有关的键信号输入。输出单元407可以是能呈现信息的任何类型的设备,并且可以包括但不限于显示器、扬声器、视频/音频输出终端、振动器和/或打印机。存储单元408可以包括但不限于磁盘、光盘。通信单元409允许电子设备400通过诸如因特网的计算机网络和/或各种电信网络与其他设备交换信息/数据,并且可以包括但不限于调制解调器、网卡、红外通信设备、无线通信收发机和/或芯片组,例如蓝牙TM设备、WiFi设备、WiMax设备、蜂窝通信设备和/或类似物。A plurality of components in the electronic device 400 are connected to the I/O interface 405, including: an input unit 406, an output unit 407, a storage unit 408, and a communication unit 409. The input unit 406 may be any type of device capable of inputting information to the electronic device 400, and the input unit 406 may receive input digital or character information, and generate key signal inputs related to user settings and/or function control of the electronic device. The output unit 407 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video/audio output terminal, a vibrator, and/or a printer. The storage unit 408 may include, but is not limited to, a disk, an optical disk. The communication unit 409 allows the electronic device 400 to exchange information/data with other devices via a computer network such as the Internet and/or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and/or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and/or the like.

计算单元401可以是各种具有处理和计算能力的通用和/或专用处理组件。计算单元401的一些示例包括但不限于中央处理模块(CPU)、图形处理模块(GPU)、各种专用的人工智能(AI)计算芯片、各种运行机器学习模型算法的计算单元、数字信号处理器(DSP)、以及任何适当的处理器、控制器、微控制器等。计算单元401执行上文所描述的各个方法和处理。例如,在一些实施例中,数据备份方法可被实现为计算机软件程序,其被有形地包含于机器可读介质,例如存储单元408。在一些实施例中,计算机程序的部分或者全部可以经由ROM402和/或通信单元409而被载入和/或安装到电子设备400上。在一些实施例中,计算单元401可以通过其他任何适当的方式(例如,借助于固件)而被配置为执行数据备份方法。The computing unit 401 may be a variety of general and/or special processing components with processing and computing capabilities. Some examples of the computing unit 401 include, but are not limited to, a central processing module (CPU), a graphics processing module (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 401 performs the various methods and processes described above. For example, in some embodiments, the data backup method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 408. In some embodiments, part or all of the computer program may be loaded and/or installed on the electronic device 400 via the ROM 402 and/or the communication unit 409. In some embodiments, the computing unit 401 may be configured to perform the data backup method in any other appropriate manner (e.g., by means of firmware).

用于实施本发明的方法的程序代码可以采用一个或多个编程语言的任何组合来编写。这些程序代码可以提供给通用计算机、专用计算机或其他可编程数据备份装置的处理器或控制器,使得程序代码当由处理器或控制器执行时使流程图和/或框图中所规定的功能/操作被实施。程序代码可以完全在机器上执行、部分地在机器上执行,作为独立软件包部分地在机器上执行且部分地在远程机器上执行或完全在远程机器或服务器上执行。The program code for implementing the method of the present invention can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data backup device, so that the program code, when executed by the processor or controller, enables the functions/operations specified in the flow chart and/or block diagram to be implemented. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

在本发明的上下文中,机器可读介质可以是有形的介质,其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备,或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。In the context of the present invention, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

如本发明使用的,术语“机器可读介质”和“计算机可读介质”指的是用于将机器指令和/或数据提供给可编程处理器的任何计算机程序产品、设备、和/或装置(例如,磁盘、光盘、存储器、可编程逻辑装置(PLD)),包括,接收作为机器可读信号的机器指令的机器可读介质。术语“机器可读信号”指的是用于将机器指令和/或数据提供给可编程处理器的任何信号。As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and/or data to a programmable processor.

为了提供与用户的交互,可以在计算机上实施此处描述的系统和技术,该计算机具有:用于向用户显示信息的显示装置(例如,CRT(阴极射线管)或者LCD(液晶显示器)监视器);以及键盘和指向装置(例如,鼠标或者轨迹球),用户可以通过该键盘和该指向装置来将输入提供给计算机。其它种类的装置还可以用于提供与用户的交互;例如,提供给用户的反馈可以是任何形式的传感反馈(例如,视觉反馈、听觉反馈、或者触觉反馈);并且可以用任何形式(包括声输入、语音输入或者、触觉输入)来接收来自用户的输入。To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

可以将此处描述的系统和技术实施在包括后台部件的计算系统(例如,作为数据服务器)、或者包括中间件部件的计算系统(例如,应用服务器)、或者包括前端部件的计算系统(例如,具有图形用户界面或者网络浏览器的用户计算机,用户可以通过该图形用户界面或者该网络浏览器来与此处描述的系统和技术的实施方式交互)、或者包括这种后台部件、中间件部件、或者前端部件的任何组合的计算系统中。可以通过任何形式或者介质的数字数据通信(例如,通信网络)来将系统的部件相互连接。通信网络的示例包括:局域网(LAN)、广域网(WAN)和互联网。The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

计算机系统可以包括客户端和服务器。客户端和服务器一般远离彼此并且通常通过通信网络进行交互。通过在相应的计算机上运行并且彼此具有客户端-服务器关系的计算机程序来产生客户端和服务器的关系。A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

Claims (10)

1.一种数据备份方法,其特征在于,所述数据备份方法应用于Greenplum数据库系统中的管理节点,所述Greenplum数据库系统还包括主机房与备用机房,所述主机房和所述备用机房中的各机器均包括第一主节点与第二镜像节点,所述第二镜像节点为其他机器的第一主节点的镜像节点,所述方法包括:1. A data backup method, characterized in that the data backup method is applied to a management node in a Greenplum database system, the Greenplum database system further includes a main computer room and a backup computer room, each machine in the main computer room and the backup computer room includes a first master node and a second mirror node, the second mirror node is a mirror node of the first master node of other machines, the method comprising: 接收第一响应信号,所述第一响应信号用于表征所述主机房中各机器与所述备用机房中的各机器的逻辑复制关系已建立,所述逻辑复制关系用于表示所述备用机房内各机器的第一主节点与所述主机房内各机器的第二镜像节点之间的映射关系;receiving a first response signal, wherein the first response signal is used to indicate that a logical replication relationship between each machine in the main machine room and each machine in the standby machine room has been established, wherein the logical replication relationship is used to indicate a mapping relationship between a first master node of each machine in the standby machine room and a second mirror node of each machine in the main machine room; 基于所述逻辑复制关系,将所述主机房内各机器的第二镜像节点中的数据变化进行解码,按照解码结果对所述备用机房内第一主节点中的数据进行修改;Based on the logical replication relationship, the data changes in the second mirror node of each machine in the main computer room are decoded, and the data in the first main node in the standby computer room is modified according to the decoding result; 若所述主机房发生故障,控制所述备用机房内第一主节点响应第一目标请求,其中,所述第一目标请求为请求所述主机房中存储数据的数据请求。If the main computer room fails, the first master node in the standby computer room is controlled to respond to a first target request, wherein the first target request is a data request for storing data in the main computer room. 2.根据权利要求1所述的数据备份方法,其特征在于,在所述接收第一响应信号前,还包括:2. The data backup method according to claim 1, characterized in that before receiving the first response signal, it also includes: 接收第一指令,所述第一指令为建立所述备用机房内各机器与所述主机房内各机器之间逻辑复制关系的信号;receiving a first instruction, wherein the first instruction is a signal for establishing a logical replication relationship between each machine in the standby computer room and each machine in the main computer room; 响应于所述第一指令,将所述主机房内各机器的第二镜像节点中的数据对象及其变化复制至对应的所述备用机房内各机器的第一主节点中。In response to the first instruction, the data objects and changes therein in the second mirror node of each machine in the main computer room are copied to the corresponding first main node of each machine in the standby computer room. 3.根据权利要求1所述的数据备份方法,其特征在于,在所述接收第一响应信号后,还包括:3. The data backup method according to claim 1, characterized in that after receiving the first response signal, it further comprises: 接收第一新增信号,所述第一新增信号为所述主机房内各机器的第二镜像节点中数据的新增信号;receiving a first newly added signal, where the first newly added signal is a newly added signal of data in a second mirror node of each machine in the main computer room; 响应于所述第一新增信号,将所述主机房内各机器的第二镜像节点中的新增数据复制到至所述备用机房内第一主节点中。In response to the first new signal, the new data in the second mirror node of each machine in the main computer room is copied to the first main node in the standby computer room. 4.根据权利要求1所述的数据备份方法,其特征在于,在所述接收第一响应信号前,还包括:4. The data backup method according to claim 1, characterized in that before receiving the first response signal, it also includes: 接收第二指令,所述第二指令为建立所述第二镜像节点与所述第一主节点之间复制关系的信号;receiving a second instruction, where the second instruction is a signal for establishing a replication relationship between the second mirror node and the first master node; 响应于所述第一指令,将所述主机房内各机器第一主节点中的数据复制至对应的所述第二镜像节点中;In response to the first instruction, copy the data in the first main node of each machine in the main computer room to the corresponding second mirror node; 接收第二新增信号,所述第二新增信号为所述主机房内各机器的第一主节点中数据的变更信号;receiving a second newly added signal, where the second newly added signal is a signal for changing data in a first master node of each machine in the main computer room; 响应于所述第二新增信号,将与所述变更信号对应的数据对象及其变化复制到所述主机房内各机器的第二镜像主节点中。In response to the second new signal, the data object corresponding to the change signal and its changes are copied to the second mirror master node of each machine in the main computer room. 5.根据权利要求4所述的数据备份方法,其特征在于,在所述响应于所述第一指令,将所述主机房内各机器第一主节点中的数据复制至对应的所述第二镜像节点中后,还包括:5. The data backup method according to claim 4, characterized in that after the data in the first main node of each machine in the main computer room is copied to the corresponding second mirror node in response to the first instruction, it also includes: 若所述主机房内任一机器的第一主节点发生故障,控制对应的所述第二镜像节点响应第二目标请求,其中,所述第二目标请求为请求所述第一主节点存储数据的数据请求。If the first main node of any machine in the main computer room fails, the corresponding second mirror node is controlled to respond to a second target request, wherein the second target request is a data request requesting the first main node to store data. 6.一种数据备份装置,其特征在于,所述数据备份装置应用于Greenplum数据库系统中的管理节点,所述Greenplum数据库系统还包括主机房与备用机房,所述主机房和所述备用机房中的各机器均包括第一主节点与第二镜像节点,所述第二镜像节点为其他机器的第一主节点的镜像节点,所述装置包括:6. A data backup device, characterized in that the data backup device is applied to a management node in a Greenplum database system, the Greenplum database system further includes a main computer room and a backup computer room, each machine in the main computer room and the backup computer room includes a first master node and a second mirror node, the second mirror node is a mirror node of the first master node of other machines, and the device includes: 第一接收模块,用于接收第一响应信号,所述第一响应信号用于表征所述主机房中各机器与所述备用机房中的各机器的逻辑复制关系已建立,所述逻辑复制关系用于表示所述备用机房内各机器的第一主节点与所述主机房内各机器的第二镜像节点之间的映射关系;A first receiving module, used to receive a first response signal, wherein the first response signal is used to indicate that a logical replication relationship between each machine in the main computer room and each machine in the standby computer room has been established, wherein the logical replication relationship is used to indicate a mapping relationship between a first primary node of each machine in the standby computer room and a second mirror node of each machine in the main computer room; 第一处理模块,基于所述逻辑复制关系,将所述主机房内各机器的第二镜像节点中的数据变化进行解码,按照解码结果对所述备用机房内第一主节点中的数据进行修改;A first processing module, based on the logical replication relationship, decodes the data changes in the second mirror node of each machine in the main computer room, and modifies the data in the first main node in the standby computer room according to the decoding result; 第一控制模块,若所述主机房发生故障,控制所述备用机房内第一主节点响应第一目标请求,其中,所述第一目标请求为请求所述主机房中存储数据的数据请求。The first control module controls the first master node in the standby computer room to respond to a first target request if the main computer room fails, wherein the first target request is a data request for storing data in the main computer room. 7.根据权利要求6所述的数据备份装置,其特征在于,所述数据备份装置,还包括:7. The data backup device according to claim 6, characterized in that the data backup device further comprises: 第二接收模块,用于接收第一指令,所述第一指令为建立所述备用机房内各机器与所述主机房内各机器之间逻辑复制关系的信号;A second receiving module is used to receive a first instruction, where the first instruction is a signal for establishing a logical replication relationship between each machine in the standby computer room and each machine in the main computer room; 第二处理模块,响应于所述第一指令,将所述主机房内各机器的第二镜像节点中的数据对象及其变化复制至对应的所述备用机房内各机器的第一主节点中。The second processing module, in response to the first instruction, copies the data objects and changes thereof in the second mirror node of each machine in the main computer room to the first main node of each machine in the corresponding standby computer room. 8.根据权利要求6所述的数据备份装置,其特征在于,所述数据备份装置,还包括:8. The data backup device according to claim 6, characterized in that the data backup device further comprises: 第三接收模块,用于接收第一新增信号,所述第一新增信号为所述主机房内各机器的第二镜像节点中数据的新增信号;A third receiving module is used to receive a first newly added signal, where the first newly added signal is a newly added signal of data in the second mirror node of each machine in the main computer room; 第三处理模块,响应于所述第一新增信号,将所述主机房内各机器的第二镜像节点中的新增数据复制到至所述备用机房内第一主节点中。The third processing module, in response to the first new signal, copies the new data in the second mirror node of each machine in the main computer room to the first main node in the standby computer room. 9.一种电子设备,包括:9. An electronic device comprising: 处理器;以及Processor; and 存储程序的存储器,Memory for storing programs, 其中,所述程序包括指令,所述指令在由所述处理器执行时使所述处理器执行根据权利要求1-5中任一项所述的方法。The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 5. 10.一种存储有计算机指令的非瞬时计算机可读存储介质,其中,所述计算机指令用于使计算机执行根据权利要求1-5中任一项所述的方法。10. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 5.
CN202410153412.2A 2024-02-02 2024-02-02 Data backup method, device, electronic device and readable storage medium Pending CN117891661A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410153412.2A CN117891661A (en) 2024-02-02 2024-02-02 Data backup method, device, electronic device and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410153412.2A CN117891661A (en) 2024-02-02 2024-02-02 Data backup method, device, electronic device and readable storage medium

Publications (1)

Publication Number Publication Date
CN117891661A true CN117891661A (en) 2024-04-16

Family

ID=90639514

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410153412.2A Pending CN117891661A (en) 2024-02-02 2024-02-02 Data backup method, device, electronic device and readable storage medium

Country Status (1)

Country Link
CN (1) CN117891661A (en)

Similar Documents

Publication Publication Date Title
CN112148798B (en) Data processing method and device applied to distributed system
CN109245908B (en) Method and device for switching master cluster and slave cluster
CN108932338A (en) Data-updating method, device, equipment and medium
US20230052935A1 (en) Asynchronous accounting method and apparatus for blockchain, medium and electronic device
CN114610532A (en) Database processing method and device
CN110413369A (en) System and method for the backup in virtualized environment
CN115292408A (en) Master-slave synchronization method, device, equipment and medium for MySQL database
CN113672354B (en) Virtual machine migration method and related device
CN108733808A (en) Big data software systems switching method, system, terminal device and storage medium
CN114415984B (en) Data processing method and device
CN119557373A (en) Data storage method, device, system and storage medium
CN115189931A (en) Distributed key management method, device, equipment and storage medium
WO2022121387A1 (en) Data storage method and apparatus, server, and medium
CN114661523A (en) Data backup method, device, program product, medium and electronic equipment
CN112463887B (en) A data processing method, device, equipment and storage medium
KR102031589B1 (en) Methods and systems for processing relationship chains, and storage media
WO2024239992A1 (en) Data synchronization method for distributed cluster and related device therefor
CN117891661A (en) Data backup method, device, electronic device and readable storage medium
CN117424799A (en) Disaster recovery method and system
CN118626553A (en) A data synchronization method, device and system
CN115964436A (en) Method and device for database cluster management
US11269923B2 (en) Ability to vary on and vary off databases simultaneously across two systems
CN115185961A (en) Node configuration method, transaction log synchronization method and node of distributed database
CN114969206A (en) Data processing method, device, equipment and storage medium
CN113326268A (en) Data writing and reading method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination