CN101640587B - Data synchronization method and device - Google Patents
Data synchronization method and device Download PDFInfo
- Publication number
- CN101640587B CN101640587B CN 200810131228 CN200810131228A CN101640587B CN 101640587 B CN101640587 B CN 101640587B CN 200810131228 CN200810131228 CN 200810131228 CN 200810131228 A CN200810131228 A CN 200810131228A CN 101640587 B CN101640587 B CN 101640587B
- Authority
- CN
- China
- Prior art keywords
- data
- memory database
- unit
- database
- synchronization
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
- H04L67/1095—Replication or mirroring of data, e.g. scheduling or transport for data synchronisation between network nodes
Landscapes
- Engineering & Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
技术领域 technical field
本发明涉及数据处理技术,尤其涉及数据同步技术。The invention relates to data processing technology, in particular to data synchronization technology.
背景技术 Background technique
一些包括业务层、物理数据库和内存数据库的业务系统在进行业务操作时,往往需要同时向物理数据库和内存数据库中写入数据,,为描述方便,一般将这一过程称为数据同步。例如,在线计费系统(OCS,Online ChargingSystem)在进行业务操作时,需要同时往物理存储(通常是物理数据库)和内存存储(通常是内存数据库)中写入数据,例如写入用户信息、帐户信息等,为描述方便,一般将这一过程称为数据同步。Some business systems including the business layer, physical database and memory database often need to write data to the physical database and memory database at the same time when performing business operations. For the convenience of description, this process is generally called data synchronization. For example, when an Online Charging System (OCS, Online Charging System) performs business operations, it needs to write data to physical storage (usually a physical database) and memory storage (usually a memory database) at the same time, such as writing user information, account Information, etc., for the convenience of description, this process is generally called data synchronization.
现有的一种数据同步方法为,在业务层(处理业务逻辑的层)需要分别将数据同步到物理数据库和内存数据库。即,为实现同一内容的数据分别同步到物理数据库和内存数据库,在业务层需要进行两个操作:一个操作是,将同一个内容的数据同步到物理数据库;另一个操作是,将同一内容的数据同步到内存数据库。An existing data synchronization method is that in the business layer (the layer that processes business logic), data needs to be synchronized to a physical database and an in-memory database respectively. That is, in order to synchronize the data of the same content to the physical database and the in-memory database respectively, two operations need to be performed at the business layer: one operation is to synchronize the data of the same content to the physical database; the other operation is to synchronize the data of the same content Data is synchronized to an in-memory database.
发明人在实现本发明的过程中发现,在上述数据同步方法中,由于在业务层需要分别将数据同步到物理数据库和内存数据库,所以语义的完整性很容易遭到破坏。例如,由于操作的数据量比较大、误操作等原因,分别同步给物理数据库和内存数据库的数据在内容上很可能并不一致。In the process of implementing the present invention, the inventor found that in the above data synchronization method, since the data needs to be synchronized to the physical database and the memory database respectively at the business layer, the integrity of the semantics is easily damaged. For example, due to the relatively large amount of data being operated, misoperation, etc., the data synchronized to the physical database and the in-memory database may not be consistent in content.
发明内容 Contents of the invention
本发明实施例提供数据同步方法及装置,用以在将数据同步给物理数据库和内存数据库时,保持语义的完整性。The embodiments of the present invention provide a data synchronization method and device for maintaining semantic integrity when synchronizing data to a physical database and a memory database.
一种数据同步方法,适用于包括业务层、物理数据库和内存数据库的业务系统,包括:获得业务层需要同步数据的触发;根据所述触发,将需要同步的数据同步到物理数据库,并将同步到物理数据库的数据同步到内存数据库。A data synchronization method, applicable to business systems including a business layer, a physical database and an in-memory database, comprising: obtaining a trigger for synchronizing data in the business layer; according to the trigger, synchronizing the data to be synchronized to the physical database, and synchronizing The data sent to the physical database is synchronized to the memory database.
一种数据同步装置,适用于包括业务层、物理数据库和内存数据库的业务系统,包括:触发获得单元,用于获得业务层需要同步数据的触发;数据同步单元,用于根据所述触发获得单元获得的触发,将需要同步的数据同步到物理数据库,并将同步到物理数据库的数据同步到内存数据库。A data synchronization device, suitable for a business system including a business layer, a physical database, and an in-memory database, comprising: a trigger obtaining unit for obtaining a trigger for synchronizing data at the business layer; a data synchronization unit for obtaining a unit according to the trigger The obtained trigger synchronizes the data that needs to be synchronized to the physical database, and synchronizes the data synchronized to the physical database to the memory database.
一种数据同步装置,包括:业务组件单元、数据操作组件单元、数据类型转换器、数据同步上下文单元、同步拦截器以及远程同步组件单元;所述业务组件单元用于触发数据操作组件单元同步需要同步的数据;所述数据操作组件单元用于根据所述业务组件单元的触发,生成包括需要同步的数据的物理数据库操作指令,将所述物理数据库操作指令发给物理数据库,并提取内存数据库操作所必需的信息,将所述内存数据库操作所必需的信息发给所述数据类型转换器,所述内存数据库操作所必需的信息包括同步给物理数据库的数据;所述数据类型转换器用于根据所述数据操作组件单元提供的内存数据库操作所必需的信息,拼装内存数据库操作指令,并将所述内存数据库操作指令发给所述数据同步上下文单元,其中,如果物理数据库的数据类型与内存数据库的数据类型不一致,则还将所述内存数据库操作所必需的信息中包括的同步给物理数据库的数据转换为内存数据库类型的数据,所述内存数据库操作指令包括所述内存数据库类型的数据;所述数据同步上下文单元用于存储所述数据类型转换器拼装的指令;所述同步拦截器用于主动的从所述数据同步上下文单元获取所述数据同步上下文单元存储的所有指令或关于一个业务的所有指令,并将所述所有指令或关于一个业务的所有指令发给所述远程同步组件单元;所述远程同步组件单元用于将所述同步拦截器提供的所述所有指令或关于一个业务的所有指令远程发给内存数据库。A data synchronization device, comprising: a business component unit, a data operation component unit, a data type converter, a data synchronization context unit, a synchronization interceptor, and a remote synchronization component unit; the business component unit is used to trigger the synchronization needs of the data operation component unit Synchronized data; the data operation component unit is used to generate a physical database operation instruction including the data to be synchronized according to the trigger of the business component unit, send the physical database operation instruction to the physical database, and extract the memory database operation The necessary information, the information necessary for the operation of the memory database is sent to the data type converter, the information necessary for the operation of the memory database includes the data synchronized to the physical database; the data type converter is used to The information necessary for the memory database operation provided by the data operation component unit, assemble the memory database operation instruction, and send the memory database operation instruction to the data synchronization context unit, wherein, if the data type of the physical database is the same as that of the memory database If the data types are inconsistent, then the data synchronized to the physical database included in the information necessary for the operation of the memory database is converted into data of the memory database type, and the memory database operation instruction includes the data of the memory database type; The data synchronization context unit is used to store the instructions assembled by the data type converter; the synchronization interceptor is used to actively obtain all instructions stored by the data synchronization context unit or all instructions about a business from the data synchronization context unit , and send all the instructions or all instructions about a business to the remote synchronization component unit; the remote synchronization component unit is used to send all the instructions provided by the synchronization interceptor or all instructions about a business Remotely sent to the memory database.
在本发明的实施例中,业务层发出一个触发,就可以实现将数据同步到物理数据库和内存数据库,而且由于将同步到物理数据库的数据同步到内存数据库,所以避免了因在业务层分别向物理数据库和内存数据库同步数据而导致同步的数据在内容上不一致的缺陷。由此可见,本发明的实施例可以在将数据同步给物理数据库和内存数据库时,保持语义的完整性。In the embodiment of the present invention, when the business layer sends a trigger, the data can be synchronized to the physical database and the memory database. The defect that the content of the synchronized data is inconsistent due to the synchronization of data between the physical database and the memory database. It can be seen that the embodiment of the present invention can maintain semantic integrity when synchronizing data to a physical database and an in-memory database.
附图说明 Description of drawings
图1为本发明实施例的数据同步方法流程图;FIG. 1 is a flowchart of a data synchronization method according to an embodiment of the present invention;
图2为本发明实施例的一种数据同步装置的结构示意图;2 is a schematic structural diagram of a data synchronization device according to an embodiment of the present invention;
图3为本发明实施例的另一种数据同步装置的结构示意图;FIG. 3 is a schematic structural diagram of another data synchronization device according to an embodiment of the present invention;
图4为本发明实施例的基于图3的数据同步方法的流程图。FIG. 4 is a flowchart of a data synchronization method based on FIG. 3 according to an embodiment of the present invention.
具体实施方式 Detailed ways
首先结合图1,对本发明实施例的数据同步方法进行说明。本发明实施例的数据同步方法适用于包括业务层、物理数据库和内存数据库的业务系统,例如在线计费系统。如图1所示,包括:Firstly, the data synchronization method according to the embodiment of the present invention will be described with reference to FIG. 1 . The data synchronization method in the embodiment of the present invention is applicable to a business system including a business layer, a physical database and an in-memory database, such as an online charging system. As shown in Figure 1, including:
步骤S101:获得业务层需要同步数据的触发。Step S101: Obtain a trigger that the business layer needs to synchronize data.
步骤S102:根据所述触发,将需要同步的数据同步到物理数据库,并将同步到物理数据库的数据同步到内存数据库。Step S102: According to the trigger, synchronize the data to be synchronized to the physical database, and synchronize the data synchronized to the physical database to the memory database.
需要同步的数据可以由多种方式获得。例如,需要同步的数据可以携带于获得的触发中,这样,当获得触发时,也就获得了需要同步的数据。再例如,在获得业务层需要同步数据的触发时,还可以获得业务层提供的所述需要同步的数据,即,触发与需要同步的数据是分别获得的。Data that needs to be synchronized can be obtained in various ways. For example, the data that needs to be synchronized can be carried in the obtained trigger, so that when the trigger is obtained, the data that needs to be synchronized is also obtained. For another example, when obtaining the trigger that the business layer needs to synchronize data, the data that needs to be synchronized provided by the business layer may also be obtained, that is, the trigger and the data that needs to be synchronized are obtained separately.
将需要同步的数据同步到物理数据库时,可以根据需要同步的数据,生成物理数据库操作指令,并将物理数据库操作指令发给物理数据库。When synchronizing the data to be synchronized to the physical database, a physical database operation instruction can be generated according to the data to be synchronized, and the physical database operation instruction can be sent to the physical database.
将需要同步的数据同步到物理数据库时,可以直接或间接通过物理数据库操作应用编程接口(API,Application Programming Interface),将需要同步的数据同步到物理数据库。When synchronizing the data to be synchronized to the physical database, the data to be synchronized can be directly or indirectly operated on the physical database through the Application Programming Interface (API, Application Programming Interface) to synchronize the data to be synchronized to the physical database.
将同步到物理数据库的数据同步到内存数据库时,可以先提取内存数据库操作所必需的信息;如果物理数据库的数据类型与内存数据库的数据类型不一致,则将所述内存数据库操作所必需的信息中的物理数据库类型的数据转换为内存数据库类型的数据;之后,根据所述内存数据库类型的数据及所述内存数据库操作所必需的信息中的其他信息,拼装内存数据库操作指令,其中,一个或多个内存数据库操作指令对应一个业务的一个操作;最后,将一个业务的所有操作分别对应的内存数据库操作指令一同发给内存数据库。When synchronizing the data synchronized to the physical database to the memory database, the information necessary for the operation of the memory database can be extracted first; if the data type of the physical database is inconsistent with the data type of the memory database, then the information necessary for the operation of the memory database will be included The data of the physical database type is converted into the data of the memory database type; after that, according to the data of the memory database type and other information in the information necessary for the operation of the memory database, the memory database operation instruction is assembled, wherein, one or more An in-memory database operation instruction corresponds to an operation of a business; finally, the in-memory database operation instructions corresponding to all operations of a business are sent to the in-memory database together.
拼装内存数据库操作指令后,还可以将内存数据库操作指令存储到数据同步上下文,将一个业务的所有操作分别对应的内存数据库操作指令一同发给内存数据库时,可以由具有拦截功能的同步拦截器主动从数据同步上下文中获取一个业务的所有操作分别对应的内存数据库操作指令,并由具有拦截功能的同步拦截器将一个业务的所有操作分别对应的内存数据库操作指令通过远程同步组件一同发给内存数据库,其中,远程同步组件提供远程访问内存数据库的功能。After the in-memory database operation instructions are assembled, the in-memory database operation instructions can also be stored in the data synchronization context, and when the in-memory database operation instructions corresponding to all operations of a business are sent to the in-memory database together, the synchronous interceptor with interception function can actively Obtain the in-memory database operation instructions corresponding to all operations of a business from the data synchronization context, and send the in-memory database operation instructions corresponding to all operations of a business to the in-memory database through the remote synchronization component by the synchronous interceptor with interception function , wherein the remote synchronization component provides the function of remote accessing the memory database.
提取内存数据库操作所必需的信息后,还可以将内存数据库操作所必需的信息存储到数据同步上下文,然后由具有拦截功能的同步拦截器主动从数据同步上下文中获取一个业务的所有操作分别对应的内存数据库操作所必需的信息,需要说明的是,将所述内存数据库操作所必需的信息中的物理数据库类型的数据转换为内存数据库类型的数据、拼装内存数据库操作指令及将一个业务的所有操作分别对应的内存数据库操作指令一同发给内存数据库都可以由同步拦截器完成。After extracting the information necessary for in-memory database operations, the information necessary for in-memory database operations can also be stored in the data synchronization context, and then the synchronization interceptor with the interception function can actively obtain all operations corresponding to a business from the data synchronization context The information necessary for the operation of the memory database. It should be noted that the data of the physical database type in the information necessary for the operation of the memory database is converted into the data of the memory database type, the operation instructions of the memory database are assembled, and all operations of a business The corresponding in-memory database operation instructions can be sent to the in-memory database together by the synchronous interceptor.
上述数据同步上下文可以是与单次业务所必需的信息绑定的存储变量、数据存储队列或共享区。The aforementioned data synchronization context may be a storage variable, a data storage queue or a shared area bound to information necessary for a single transaction.
上述远程同步组件可以是内存数据库的远程操作API,上述远程同步组件也可以通过分布对象处理技术或者网络通信技术远程访问内存数据库。The above-mentioned remote synchronization component can be a remote operation API of the memory database, and the above-mentioned remote synchronization component can also remotely access the memory database through distributed object processing technology or network communication technology.
此外,如果将同步到物理数据库的数据同步到内存数据库失败,则可以通知物理数据库进行事务回滚操作。In addition, if the data synchronized to the physical database fails to be synchronized to the memory database, the physical database may be notified to perform a transaction rollback operation.
上述数据同步方法可以由多种形式的装置来实现,这些装置适用于包括业务层、物理数据库和内存数据库的业务系统,例如在线计费系统,其中的一种数据同步装置如图2所示,包括:触发获得单元201,用于获得业务层需要同步数据的触发;数据同步单元202,用于根据所述触发获得单元获得的触发,将需要同步的数据同步到物理数据库,并将同步到物理数据库的数据同步到内存数据库。The above-mentioned data synchronization method can be implemented by devices in various forms, and these devices are suitable for business systems including business layers, physical databases and memory databases, such as online billing systems, and one of the data synchronization devices is shown in Figure 2. Including: a
需要同步的数据可以携带于触发获得单元201获得的触发中,这样,当触发获得单元201获得触发时,也就获得了需要同步的数据。另外,触发获得单元201在获得业务层需要同步数据的触发时,还可以获得业务层提供的所述需要同步的数据,即,触发与需要同步的数据是触发获得单元201分别获得的。The data that needs to be synchronized can be carried in the trigger obtained by the
数据同步单元202可以包括:指令生成单元2021,用于根据需要同步的数据,生成物理数据库操作指令;第一指令发出单元2022,用于将指令生成单元2021生成的物理数据库操作指令发给物理数据库。The
数据同步单元202将需要同步的数据同步到物理数据库时,可以直接或间接通过物理数据库操作API,将需要同步的数据同步到物理数据库。When the
数据同步单元202可以包括:信息提取单元2023,用于提取内存数据库操作所必需的信息;数据类型转换单元2024,用于如果物理数据库的数据类型与内存数据库的数据类型不一致,则将信息提取单元2023提取的内存数据库操作所必需的信息中的物理数据库类型的数据转换为内存数据库类型的数据;指令拼装单元2025,用于根据数据类型转换单元2024转换的内存数据库类型的数据及信息提取单元2023提取的内存数据库操作所必需的信息中的其他信息,拼装内存数据库操作指令,其中,一个或多个内存数据库操作指令对应一个业务的一个操作;第二指令发出单元2026,用于将指令拼装单元2025拼装的一个业务的所有操作分别对应的内存数据库操作指令一同发给内存数据库。The
上述数据同步装置还可以包括指令存储单元203,用于将指令拼装单元2025拼装的内存数据库操作指令存储到数据同步上下文;第二指令发出单元2026可以为具有拦截功能的同步拦截器,所述同步拦截器可以主动从数据同步上下文中获取一个业务的所有操作分别对应的内存数据库操作指令。The above-mentioned data synchronization device can also include an
上述数据同步装置还可以包括:信息存储单元204,用于将信息提取单元2023提取的内存数据库操作所必需的信息存储到数据同步上下文;具有拦截功能的同步拦截器205,用于主动从数据同步上下文中获取一个业务的所有操作分别对应的内存数据库操作所必需的信息;并且,数据类型转换单元2024、指令拼装单元2025及第二指令发出单元2026可以集成于同步拦截器205中。The above-mentioned data synchronization device may also include: an
除上述方法实施例和装置实施例外,本发明实施例还提供了如图3所示的数据同步装置,所述数据同步装置包括:业务组件(Bussiness Component)单元301、数据操作组件(Data Manipulation Component)单元302、数据类型转换器303、数据同步上下文单元304、同步拦截器305以及远程同步组件单元306。In addition to the above-mentioned method embodiment and device embodiment, the embodiment of the present invention also provides a data synchronization device as shown in Figure 3, the data synchronization device includes: a business component (Business Component)
业务组件单元301主要处理的是业务逻辑,尤其是需要同步数据的业务逻辑,例如开户、产品订购、产品变更等。具体来说,当业务组件单元301为实现某个业务时,可以将需要实现这个业务的需求通知给数据操作组件单元302,或者说,触发数据操作组件单元302同步需要同步的数据。业务组件单元301可以包括至少一个业务组件。The
数据操作组件单元302主要用于根据业务组件单元301的触发,来负责具体的数据操作。例如,可以直接或者间接通过物理数据库操作API将数据增加到物理数据库100中。数据操作组件单元302可以包括至少一个数据操作组件。The data
数据操作组件单元302可以将具体的数据操作封装,例如,对于增加用户的操作来说,可以封装为一个数据操作组件提供的API,业务组件可以通过这个API实现增加用户的操作,而不需关心数据同步到物理数据库100还是内存数据库200。The data
另外,当数据操作组件单元302将数据同步到物理数据库100时,可以生成包括需要同步的数据的物理数据库操作指令,之后将生成的物理数据库操作指令发给物理数据库100。指令可以包括操作的动作和相关的数据。以开户为例,数据操作组件单元302为物理数据库100生成的指令可以包括与开户相关的动作及相关的数据。一个业务可以对应多个操作,还是以开户为例,其可以对应增加用户、增加客户及增加帐户等三个操作,对此,数据操作组件单元302可以对应每个操作生成一个指令或多个指令,这样,对于开户业务来说,数据操作组件单元302可以生成至少三个指令。In addition, when the data
此外,数据操作组件单元302还可以提取内存数据库操作所必需的信息,并将内存数据库操作所必需的信息发给数据类型转换器303。内存数据库操作所必需的信息可以来源于业务组件单元301发给数据操作组件单元302的通知,也可以来源于数据操作组件单元302同步给物理数据库100的数据。显然,为保证同步给物理数据库100的数据与同步给内数据库200的数据在内容上一致,内存数据库操作所必需的信息包括同步给物理数据库100的数据。In addition, the data
数据类型转换器303主要用于根据数据操作组件单元302提供的内存数据库操作所必需的信息,拼装内存数据库操作指令,并将所述内存数据库操作指令发给所述数据同步上下文单元304。The
其中,如果物理数据库100的数据类型与内存数据库200的数据类型不一致,则可以将所述内存数据库操作所必需的信息中包括的同步给物理数据库100的数据转换为内存数据库类型的数据,所述内存数据库操作指令包括所述内存数据库类型的数据。Wherein, if the data type of the
一般来说,物理数据库的数据类型和内存数据库的数据类型是不一致的,所以,如果将为物理数据库100提供的数据直接或间接提供给内存数据库200时,则需要将物理数据库100类型的数据(可以是内存数据库操作所必需的信息中的具有具体内容的数据)转换为内存数据库200类型的数据,然后将内存数据库200类型的数据拼装为内存数据库操作指令。当然,如果内存数据库操作所必需的信息除包括具有具体内容的数据外,还包括一些必要的操作指示。则数据类型转换器303除根据上述内存数据库200类型的数据外,还需要根据操作指示来拼装内存数据库操作指令。Generally speaking, the data type of the physical database is inconsistent with the data type of the memory database. Therefore, if the data provided by the
可选的,数据类型的转换也可以在远程同步组件单元306中完成,还可以在内存数据库200中完成。需要说明的是,虽然在一般情况下,物理数据库的数据类型和内存数据库的数据类型不一致,但有些情况下,物理数据库的数据类型和内存数据库的数据类型还是有可能会一致的,对此,当数据操作组件单元302将物理数据库100类型的数据发给数据类型转换器303时,数据类型转换器303无需对数据类型转换,而是直接根据物理数据库100类型的数据拼装内存数据库操作指令,并将拼装后的指令发给数据同步上下文单元304。Optionally, the data type conversion can also be completed in the remote
数据同步上下文单元304主要用于存储数据类型转换器303拼装的指令。一个数据同步上下文可以是指一个与单次业务所必需的信息绑定的数据存储,例如是存储变量、数据存储队列或共享区。另外,数据同步上下文是可以共享的,多个数据操作组件都可以共享数据同步上下文。The data
同步拦截器305主要用于主动的从数据同步上下文单元304获取数据同步上下文单元304存储的所有指令或关于一个业务的所有指令。具体的,获取所有指令后,可以将所有指令打包整理,然后发给远程同步组件单元306。同步拦截器305的一个典型特征是对业务组件透明,并且和具体业务语义无关。The
远程同步组件单元306提供了远程访问内存数据库200的能力。具体来说,远程同步组件单元306获得同步拦截器305提供的指令后,可以将指令发给内存数据库200,这样,需要同步给内存数据库200的数据就同步到了内存数据库200。The remote
在具体实现上,远程同步组件单元306包括至少一个远程同步组件,远程同步组件可以是内存数据库的远程操作API,远程同步组件单元306可以通过分布对象处理技术或者网络通信技术来实现远程操作内存数据库200,例如,通过分布对象处理技术或者网络通信技术将指令发给内存数据库200。In a specific implementation, the remote
图4为基于图3的数据同步装置的数据同步方法的流程图。如图4所示,包括:FIG. 4 is a flowchart of a data synchronization method based on the data synchronization device in FIG. 3 . As shown in Figure 4, including:
步骤S401:业务组件单元301调用数据操作组件单元302以完成数据的增加、修改和删除等操作。Step S401: The
步骤S402:数据操作组件单元302生成物理数据库操作指令,完成对物理数据库100的操作,物理数据库操作指令可以由结构化查询语言(SQL,Structured Query Language)实现。Step S402: The data
步骤S403:数据操作组件单元302提取内存数据库操作所必需的信息,然后将必需的信息传递给数据类型转换器303。Step S403: the data
步骤S404:数据类型转换器303根据必需的信息,拼装内存数据库操作指令。Step S404: The
步骤S405:数据类型转换器303将拼装的内存数据库操作指令发给数据同步上下文单元304。Step S405: the
这里需要说明的是,由于数据同步上下文单元304可以存储一个业务的所有操作所涉及的内存数据库操作指令,所以,如果一个完整的业务流程涉及多次操作,例如开户业务涉及增加用户、增加客户及增加帐户等三个操作,则需要重复步骤S402-S405,以实现将所有操作所涉及的内存数据库操作指令都存储在数据同步上下文单元304中。It should be noted here that since the data
对此,在步骤S405之后,还包括步骤S406:判断是否需要返回步骤S402,如果是,则返回步骤S402,否则转步骤S407。In this regard, after step S405, step S406 is also included: judging whether it is necessary to return to step S402, if yes, return to step S402, otherwise go to step S407.
步骤S407:同步拦截器305主动从数据同步上下文单元304获取数据同步上下文单元304存储的所有内存数据库操作指令或者一个业务的所有内存数据库操作指令,并将一个业务的所有内存数据库操作指令组装为一个新的指令,之后再将新的指令发给远程同步组件单元306。Step S407:
步骤S408:远程同步组件单元306将新的指令发送给内存数据库200。Step S408 : the remote
步骤S409:内存数据库根据新的指令,以批量的方式完成数据的操作。Step S409: The in-memory database completes data operations in batches according to the new instructions.
步骤S410:批量更新在内存数据库200的一个事务中进行,如果成功,转步骤S413,否则,转步骤S411。Step S410: batch update is performed in one transaction of the in-
步骤S411:内存数据库200完成事务回滚并通过远程同步组件单元306通知同步拦截器305。Step S411 : the in-
步骤S412:同步拦截器305通知物理数据库100(具体为物理数据操作的事务管理器)事务异常,完成物理数据的回滚。Step S412: the
步骤S413:业务流程结束。Step S413: the business process ends.
需要说明的是,图3所示的装置和图4所示的方法可以适用于在线计费系统。以上所描述的装置实施例仅仅是示意性的,其中所述作为分离部件说明的单元可以是或者也可以不是物理上分开的,作为单元显示的部件可以是或者也可以不是物理单元,即可以位于一个地方,或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部模块来实现本实施例方案的目的。本领域普通技术人员在不付出创造性的劳动的情况下,即可以理解并实施。It should be noted that the apparatus shown in FIG. 3 and the method shown in FIG. 4 can be applied to an online charging system. The device embodiments described above are only illustrative, and the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in One place, or it can be distributed to multiple network elements. Part or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. It can be understood and implemented by those skilled in the art without any creative effort.
通过以上的实施方式的描述,本领域的技术人员可以清楚地了解到各实施方式可借助软件加必需的通用硬件平台的方式来实现,当然也可以通过硬件。基于这样的理解,上述技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来,该计算机软件产品可以存储在计算机可读存储介质中,如ROM/RAM、磁碟、光盘等,包括若干指令用以使得一台计算机设备(可以是个人计算机,服务器,或者网络设备等)执行各个实施例或者实施例的某些部分所述的方法。Through the above description of the implementations, those skilled in the art can clearly understand that each implementation can be implemented by means of software plus a necessary general hardware platform, and of course also by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products can be stored in computer-readable storage media, such as ROM/RAM, magnetic discs, optical discs, etc., including several instructions to make a computer device (which may be a personal computer, server, or network device, etc.) execute the methods described in various embodiments or some parts of the embodiments.
为使本领域技术人员更加清楚的理解本发明实施例,下面以在线计费系统中典型的开户业务为例,对本发明实施例的数据同步方法或者数据同步装置的工作流程进行详细说明。In order to make those skilled in the art understand the embodiment of the present invention more clearly, the following takes a typical account opening service in an online billing system as an example to describe the working process of the data synchronization method or the data synchronization device according to the embodiment of the present invention in detail.
假定在开户过程中需要进行增加用户(addSubscriber)、增加客户(addCustomer)以及增加帐户(addAccount)的操作,即所谓的三户信息增加。三户信息既要增加到物理数据库中,也要增加到内存数据库中,即该开户业务的数据需要同步到物理数据库和内存数据库中。Assume that in the process of opening an account, operations of adding users (addSubscriber), adding customers (addCustomer) and adding accounts (addAccount) are required, that is, the so-called three-account information increase. The information of the three households must be added to both the physical database and the memory database, that is, the data of the account opening business needs to be synchronized to the physical database and the memory database.
对于开户业务的数据同步方法具体如下:The data synchronization method for account opening business is as follows:
业务组件(即开户逻辑)首先调用数据操作组件的增加用户API,开户逻辑可以封装到一个类中,如SubscriberService,其中newSubscriber方法提供开户逻辑。newSubscriber调用数据操作组件的addSubscriber方法。The business component (that is, the account opening logic) first calls the add user API of the data operation component, and the account opening logic can be encapsulated into a class, such as SubscriberService, where the newSubscriber method provides the account opening logic. newSubscriber calls the addSubscriber method of the data operation component.
数据操作组件的addSubscriber方法生成物理数据库操作指令,这里假设直接生成类似下面的SQL语句:The addSubscriber method of the data operation component generates physical database operation instructions. Here, it is assumed that SQL statements similar to the following are directly generated:
insert into subscriber values(…)insert into subscriber values(…)
这样,用户信息被增加到物理数据库中。In this way, user information is added to the physical database.
数据操作组件提取内存数据库操作所必需的信息,这里假定内存数据库的表数据字段和物理数据库的相同,只是数据格式略有差异,例如,用户的生效时间在物理数据库中是时间日期类型,而在内存数据库中是整数类型。数据操作组件将内存数据库操作所必需的信息发给数据类型转换器。The data operation component extracts the information necessary for the operation of the memory database. Here, it is assumed that the table data fields of the memory database are the same as those of the physical database, but the data format is slightly different. In-memory database is an integer type. The data operation component sends the information necessary for the operation of the memory database to the data type converter.
数据类型转换器负责将物理数据库类型的数据转换为内存数据库类型的数据,并根据内存数据库类型的数据,拼装内存数据库操作指令。对于本实施例,数据类型转换器将用户的生效时间转换为相对于格林尼治时间的整数值,经过整理后就生成了内存数据库操作指令,在本实施例中,内存数据库操作指令和物理数据库操作指令基本等同,即:The data type converter is responsible for converting the data of the physical database type into the data of the memory database type, and assembling the operation instructions of the memory database according to the data of the memory database type. For this embodiment, the data type converter converts the effective time of the user into an integer value relative to Greenwich Mean Time, and after sorting out, an in-memory database operation instruction is generated. In this embodiment, the in-memory database operation instruction and the physical database operation instruction Basically equivalent, namely:
insert into subscriber values(…)insert into subscriber values(…)
但是,内存数据库操作指令有一部分数据值与物理数据库操作指令的数据值不同,例如这里的生效时间。However, some data values of the in-memory database operation instructions are different from those of the physical database operation instructions, such as the effective time here.
数据类型转换器将内存数据库操作指令发给数据同步上下文,在本实施例中,数据同步上下文的实现为Java中一个线程绑定的变量,示例如下:The data type converter sends the memory database operation instruction to the data synchronization context. In this embodiment, the realization of the data synchronization context is a thread-bound variable in Java. Examples are as follows:
ThreadLocal requestContextHolder=new InheritableThreadLocal();ThreadLocal requestContextHolder = new InheritableThreadLocal();
其中,线程变量中保存发布给内存的指令。Wherein, the instruction issued to the memory is stored in the thread variable.
对于开户业务需要三次操作数据,即增加用户、增加客户和增加帐户,对此,需要三次执行上述流程,直至数据类型转换器将增加用户、增加客户和增加帐户分别对应的内存数据库操作指令全部同步到数据同步上下文为止。For the account opening business, three operation data are required, that is, adding users, adding customers, and adding accounts. For this, the above process needs to be executed three times until the data type converter synchronizes all the memory database operation instructions corresponding to adding users, adding customers, and adding accounts. Until the data synchronization context.
在本实施例中,使用面向方面编程(AOP,Aspect Oriented Programming)的方式给开户逻辑对应的API增加了一个同步拦截器,同步拦截器只要实现特定的接口并配置到AOP中实现即可,以Spring为例,可以如下配置:In this embodiment, an aspect-oriented programming (AOP, Aspect Oriented Programming) method is used to add a synchronous interceptor to the API corresponding to the account opening logic. The synchronous interceptor only needs to implement a specific interface and configure it in AOP. Taking Spring as an example, you can configure it as follows:
<aop:config><aop:config>
<aop:pointcut id=″syncOperation″<aop:pointcut id="syncOperation"
expression=″execution(*…(..))″/>expression="execution(*...(..))"/>
<aop:advisor advice-ref=″syncAdvice″<aop:advisor advice-ref="syncAdvice"
pointcut-ref=″syncOperation″/>pointcut-ref="syncOperation"/>
</aop:config></aop:config>
<bean id=″syncAdvice″class=″…″/><bean id="syncAdvice" class="..."/>
当数据类型转换器将三个操作分别对应的内存数据库操作指令全部同步到数据同步上下文后,同步拦截器可以主动从数据同步上下文中获取三个操作分别对应的内存数据库操作指令,并将这些指令组装为一个新的指令。After the data type converter synchronizes all the in-memory database operation instructions corresponding to the three operations to the data synchronization context, the synchronization interceptor can actively obtain the in-memory database operation instructions corresponding to the three operations from the data synchronization context, and convert these instructions assembled as a new instruction.
同步拦截器将组装的新的指令通过远程同步组件发送给内存数据库,其中,远程同步组件可以使用套接字方式发送字符串的SQL指令。The synchronization interceptor sends the assembled new instructions to the memory database through the remote synchronization component, wherein the remote synchronization component can use the socket mode to send the SQL instruction of the string.
内存数据库根据SQL指令在一个内存事务中以批量的方式完成增加用户、增加客户和增加帐户的操作。批量更新在内存数据库的一个事务中进行,至此,开户业务的数据同步流程结束。The memory database completes the operations of adding users, adding customers, and adding accounts in batches in a memory transaction according to SQL instructions. The batch update is performed in one transaction of the memory database, so far, the data synchronization process of the account opening business is over.
在本发明的实施例中,将物理数据操作和内存数据库操作统一起来,从而维护了业务语义的完整性,对于业务而言,数据的持久化集中在一起,对外提供一致的语义。In the embodiment of the present invention, the physical data operation and the memory database operation are unified, thereby maintaining the integrity of the business semantics. For the business, data persistence is centralized to provide consistent semantics externally.
在本发明的实施例中,物理数据库的事务采用统一控制,并且在业务最后一次性同步内存,这样,当内存数据处理失败时,可以通知物理数据库进行事务回滚。另外,内存操作也可以控制在一个事务内进行,从而较简单的实现了事务的控制。In the embodiment of the present invention, the transaction of the physical database adopts unified control, and the memory is synchronized once at the end of the business, so that when the memory data processing fails, the physical database can be notified to roll back the transaction. In addition, the memory operation can also be controlled within a transaction, thus realizing the control of the transaction relatively simply.
在本发明的实施例中,可以通过数据同步上下文将业务过程中生成的同步内存指令进行统一搜集、发送,从而可以进行批量处理,减少了不必要的远程交互,提高了数据同步的性能。In the embodiment of the present invention, the synchronization memory instructions generated in the business process can be collected and sent through the data synchronization context, so that batch processing can be performed, unnecessary remote interaction is reduced, and the performance of data synchronization is improved.
在本发明的实施例中,可以通过同步拦截器将数据同步的动作对业务隐藏起来,从而对外提供了数据操作单接口的能力,减少了出错的可能,也便利了系统开发。In the embodiment of the present invention, the data synchronization action can be hidden from the business through the synchronization interceptor, thereby providing the ability of a single interface for data operation, reducing the possibility of errors, and facilitating system development.
以上所述仅是本发明的优选实施方式,应当指出,对于本技术领域的普通技术人员来说,在不脱离本发明原理的前提下,还可以作出若干改进和润饰,这些改进和润饰也应视为本发明的保护范围。The above is only a preferred embodiment of the present invention, it should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, some improvements and modifications can also be made, and these improvements and modifications should also be It is regarded as the protection scope of the present invention.
Claims (18)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 200810131228 CN101640587B (en) | 2008-08-01 | 2008-08-01 | Data synchronization method and device |
PCT/CN2009/072989 WO2010012230A1 (en) | 2008-08-01 | 2009-07-30 | Data synchronization method and device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN 200810131228 CN101640587B (en) | 2008-08-01 | 2008-08-01 | Data synchronization method and device |
Publications (2)
Publication Number | Publication Date |
---|---|
CN101640587A CN101640587A (en) | 2010-02-03 |
CN101640587B true CN101640587B (en) | 2012-04-25 |
Family
ID=41609968
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN 200810131228 Active CN101640587B (en) | 2008-08-01 | 2008-08-01 | Data synchronization method and device |
Country Status (2)
Country | Link |
---|---|
CN (1) | CN101640587B (en) |
WO (1) | WO2010012230A1 (en) |
Families Citing this family (12)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8468172B2 (en) | 2010-05-14 | 2013-06-18 | Sap Ag | Integrated application server and data server processes with matching data formats |
CN103188282B (en) * | 2011-12-27 | 2017-08-04 | 腾讯科技(北京)有限公司 | Method, system and the equipment of file synchronization |
CN102831017B (en) * | 2012-08-31 | 2014-09-10 | 河海大学 | High-efficiency distributed parallel authentication system |
CN102945251A (en) * | 2012-10-12 | 2013-02-27 | 浪潮电子信息产业股份有限公司 | Method for optimizing performance of disk database by memory database technology |
CN102929983B (en) * | 2012-10-18 | 2016-11-16 | 京信通信系统(中国)有限公司 | Embedded database is synchronized to the method and apparatus in flash data storehouse |
CN102982130A (en) * | 2012-11-16 | 2013-03-20 | 深圳市融创天下科技股份有限公司 | Synchronized method for not only structured query language (NOSQL) and relational database management system (RDBMS) database and system thereof |
CN104184822A (en) * | 2014-08-29 | 2014-12-03 | 海视云(北京)科技有限公司 | Data synchronism method and system |
CN105786877B (en) * | 2014-12-23 | 2019-02-26 | 中国移动通信集团山东有限公司 | A data storage method, system and query method |
CN105701190A (en) * | 2016-01-07 | 2016-06-22 | 深圳市金证科技股份有限公司 | Data synchronizing method and device |
CN111930843B (en) * | 2020-08-11 | 2023-07-28 | 中国工商银行股份有限公司 | Database-based data system and processing method |
CN113778570B (en) * | 2021-09-10 | 2023-06-06 | 四川新网银行股份有限公司 | Distributed system breakpoint retry method based on AOP+Threadlocal technology |
CN113946628A (en) * | 2021-10-27 | 2022-01-18 | 中国建设银行股份有限公司 | Data synchronization method and device based on interceptor |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1407465A (en) * | 2001-08-13 | 2003-04-02 | 深圳市丛文软件技术有限公司 | Data exchanging method and device between different databases with different structure |
CN1704936A (en) * | 2004-05-31 | 2005-12-07 | 深圳宇龙计算机通信科技(深圳)有限公司 | Data conversion or synchronization method between data bases |
CN1851658A (en) * | 2005-08-02 | 2006-10-25 | 上海华为技术有限公司 | Data management system and method |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1223948C (en) * | 2002-11-15 | 2005-10-19 | 鸿富锦精密工业(深圳)有限公司 | Data synchronizing system and method |
CN100450199C (en) * | 2004-10-21 | 2009-01-07 | 华为技术有限公司 | System and method for realizing synchronous transmission of wireless broadcast and multicast services |
CN101056249A (en) * | 2006-04-12 | 2007-10-17 | 华为技术有限公司 | A method and system for transmitting the multicast broadcast information |
CN100550719C (en) * | 2006-04-28 | 2009-10-14 | 华为技术有限公司 | The method of equipment room data sync |
-
2008
- 2008-08-01 CN CN 200810131228 patent/CN101640587B/en active Active
-
2009
- 2009-07-30 WO PCT/CN2009/072989 patent/WO2010012230A1/en active Application Filing
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1407465A (en) * | 2001-08-13 | 2003-04-02 | 深圳市丛文软件技术有限公司 | Data exchanging method and device between different databases with different structure |
CN1704936A (en) * | 2004-05-31 | 2005-12-07 | 深圳宇龙计算机通信科技(深圳)有限公司 | Data conversion or synchronization method between data bases |
CN1851658A (en) * | 2005-08-02 | 2006-10-25 | 上海华为技术有限公司 | Data management system and method |
Also Published As
Publication number | Publication date |
---|---|
CN101640587A (en) | 2010-02-03 |
WO2010012230A1 (en) | 2010-02-04 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN101640587B (en) | Data synchronization method and device | |
CN108536761B (en) | Report data query method and server | |
US9477727B2 (en) | Abstracting data for use by a mobile device having occasional connectivity | |
CN107147704B (en) | block chain-oriented universal service middleware system | |
CN106357452B (en) | A kind of the High Availabitity frame system and its implementation of the storage of single-point isomeric data | |
US10142180B2 (en) | Method and system for providing unified configuration information to disparate system software components | |
CN105447014B (en) | Metadata management method based on binlog and for providing the method and device of metadata | |
CN102542007B (en) | Method and system for synchronization of relational databases | |
CN109522029A (en) | A kind of method and device for disposing cloud platform technology component | |
CN102314358B (en) | A method for deploying conventional applications in SOA mode on cloud platform | |
CN102917020A (en) | A data packet-based method for synchronizing data between a mobile terminal and a business system | |
CN112149079A (en) | Planning review management platform and user access authorization method based on microservice architecture | |
CN101847100A (en) | Method for expanding software application and device | |
CN104778175A (en) | Method and system for realizing data synchronization of heterogeneous database | |
CN103164206A (en) | Software customization system based on software as a service (SAAS) | |
CN105893115A (en) | Method and system for differential deployment of virtual machine | |
CN109376197A (en) | A kind of method of data synchronization, server and computer storage medium | |
CN110138876A (en) | Task deployment method, apparatus, equipment and platform | |
CN113672240A (en) | Container-based multi-machine-room batch automatic deployment application method and system | |
CN110177144B (en) | Method for copying application environment by one key based on private cloud | |
CN107025279B (en) | SCD file online parallel configuration system and method based on database interface layer operation flow | |
US9207966B2 (en) | Method and system for providing a high-availability application | |
CN114240409A (en) | Cross-chain asset interaction method based on improved Hash time lock | |
CN102982171A (en) | Database synchronization method | |
CN109614271A (en) | Control method, device, device and storage medium for data consistency of multiple clusters |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
C06 | Publication | ||
PB01 | Publication | ||
C10 | Entry into substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
C14 | Grant of patent or utility model | ||
GR01 | Patent grant | ||
TR01 | Transfer of patent right | ||
TR01 | Transfer of patent right |
Effective date of registration: 20201127 Address after: Unit 2414-2416, main building, no.371, Wushan Road, Tianhe District, Guangzhou City, Guangdong Province Patentee after: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd. Address before: 518129 Bantian HUAWEI headquarters office building, Longgang District, Guangdong, Shenzhen Patentee before: HUAWEI TECHNOLOGIES Co.,Ltd. |
|
TR01 | Transfer of patent right | ||
TR01 | Transfer of patent right |
Effective date of registration: 20201216 Address after: Building 2, No. 3, Fuqian Road, Hailing District, Taizhou City, Jiangsu Province Patentee after: Taizhou Haitong Asset Management Co., Ltd Address before: Unit 2414-2416, main building, no.371, Wushan Road, Tianhe District, Guangzhou City, Guangdong Province Patentee before: GUANGDONG GAOHANG INTELLECTUAL PROPERTY OPERATION Co.,Ltd. |