[go: up one dir, main page]

CN107908472A - Data synchronization unit, method and computer-readable recording medium - Google Patents

Data synchronization unit, method and computer-readable recording medium Download PDF

Info

Publication number
CN107908472A
CN107908472A CN201710916482.9A CN201710916482A CN107908472A CN 107908472 A CN107908472 A CN 107908472A CN 201710916482 A CN201710916482 A CN 201710916482A CN 107908472 A CN107908472 A CN 107908472A
Authority
CN
China
Prior art keywords
task
synchrodata
data synchronization
synchronous
queue
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710916482.9A
Other languages
Chinese (zh)
Inventor
李占川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ping An Technology Shenzhen Co Ltd
Original Assignee
Ping An Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ping An Technology Shenzhen Co Ltd filed Critical Ping An Technology Shenzhen Co Ltd
Priority to CN201710916482.9A priority Critical patent/CN107908472A/en
Publication of CN107908472A publication Critical patent/CN107908472A/en
Priority to PCT/CN2018/089187 priority patent/WO2019062182A1/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/52Program synchronisation; Mutual exclusion, e.g. by means of semaphores
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/23Updating
    • G06F16/2358Change logging, detection, and notification
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/275Synchronous replication
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5005Allocation of resources, e.g. of the central processing unit [CPU] to service a request
    • G06F9/5027Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals
    • G06F9/5038Allocation of resources, e.g. of the central processing unit [CPU] to service a request the resource being a machine, e.g. CPUs, Servers, Terminals considering the execution order of a plurality of tasks, e.g. taking priority or time dependency constraints into consideration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/54Interprogram communication
    • G06F9/544Buffers; Shared memory; Pipes
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/10Services
    • G06Q50/20Education
    • G06Q50/205Education administration or guidance

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Databases & Information Systems (AREA)
  • Tourism & Hospitality (AREA)
  • Strategic Management (AREA)
  • Educational Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Educational Administration (AREA)
  • Marketing (AREA)
  • Human Resources & Organizations (AREA)
  • General Health & Medical Sciences (AREA)
  • Primary Health Care (AREA)
  • General Business, Economics & Management (AREA)
  • Computing Systems (AREA)
  • Economics (AREA)
  • Health & Medical Sciences (AREA)
  • Multi Processors (AREA)

Abstract

The invention discloses a kind of data synchronization unit, including memory and processor, the data synchronization program that can be run on a processor is stored with memory, which realizes following steps when being executed by processor:When receiving renewal request, obtain with updating the corresponding synchrodata of request, and establish synchronous task corresponding with synchrodata;Synchronous task is added in task queue, and is stored after synchrodata is associated with synchronous task to buffer zone;When the amount of storage for detecting the synchrodata cached in buffer zone reaches the first predetermined threshold value, or the task amount in task queue is when reaching the second predetermined threshold value, start the synchronous task in the thread process task queue in thread pool, the synchrodata stored in buffer zone is sent to external management system.The present invention also proposes a kind of method of data synchronization and a kind of computer-readable recording medium.The present invention reduces code redundancy, improves data synchronization efficiency.

Description

Data synchronization unit, method and computer-readable recording medium
Technical field
The present invention relates to field of computer technology, more particularly to a kind of data synchronization unit, method and computer-readable deposit Storage media.
Background technology
Educational training management organization is managed, it is necessary to which the data in Training Management Information System are synchronous for the ease of business datum Into Service Synthesis Management System, the mode of existing data synchronization is:Service Synthesis Management System opens specific interface to Training Management System, when there is data update, (related data such as lesson data, demographic data occurs increase, deletes, modification Training Management Information System Deng operation), the request of triggering renewal in real time, Service Synthesis Management System is synchronized to by fresh information, and existing request method be Synchronization of the synchronizing code into row information is added in each requesting method, therefore, when Training Management Information System is huger, example Such as, training organization is distributed throughout the country multiple cities, and it is very frequent that this results in the daily renewal request of Training Management Information System, Need to trigger substantial amounts of renewal request, cause to produce substantial amounts of code redundancy, and then cause the synchronous efficiency to data low.
The content of the invention
The present invention provides a kind of data synchronization unit, method and computer-readable recording medium, and its main purpose is to subtract Few code redundancy, improves data synchronization efficiency.
To achieve the above object, the present invention provides a kind of data synchronization unit, and sychronisation includes memory and processor, The data synchronization program that can be run on the processor is stored with the memory, the data synchronization program is by the place Reason device realizes following steps when performing:
Receive renewal request when, obtain with it is described renewal the corresponding synchrodata of request, and foundation with it is described synchronous The corresponding synchronous task of data;
Synchronous task is added in task queue, and store after the synchrodata is associated with the synchronous task to Buffer zone;
When the amount of storage for detecting the synchrodata cached in the buffer zone reaches the first predetermined threshold value, or it is described When task amount in task queue reaches the second predetermined threshold value, start same in task queue described in the thread process in thread pool Step task, the synchrodata stored in the buffer zone is sent to external management system.
Alternatively, it is described when the amount of storage for detecting the synchrodata cached in the buffer zone reaches the first default threshold Value, or when task amount in the task queue reaches the second predetermined threshold value, start and appoint described in the thread process in thread pool The synchronous task being engaged in queue, the synchrodata stored in the buffer zone is sent can to the step of external management system To replace with following steps:
When detecting that synchronization of time intenals reaches prefixed time interval, start task described in the thread process in thread pool Synchronous task in queue, the synchrodata stored in the buffer zone is sent to external management system.
Alternatively, the processor is additionally operable to perform the data synchronization program, to realize following steps:
During synchronous task in the thread process task queue in starting thread pool, according to pending synchronous task The maximum thread amount of total quantity and the pre-set thread pool, dynamically adjusts the quantity of the thread of startup.
Alternatively, the step that synchronous task is added in task queue includes:
The synchronous task is added to the task according to the settling time of the synchronous task by order of the elder generation after In queue.
Alternatively, the processor is additionally operable to perform the data synchronization program, to be added to synchronous task described Before step in task queue, following steps are also realized:
Getting with after the renewal corresponding synchrodata of request, mechanism mark is obtained from the renewal request Know, to determine the priority of the corresponding agency node of the request to be updated according to the authority identification;
The step that synchronous task is added in task queue includes:
The synchronous task is added to the task according to the priority order from high to low of corresponding agency node In queue.
In addition, to achieve the above object, the present invention also provides a kind of method of data synchronization, this method includes:
Receive renewal request when, obtain with it is described renewal the corresponding synchrodata of request, and foundation with it is described synchronous The corresponding synchronous task of data;
Synchronous task is added in task queue, and store after the synchrodata is associated with the synchronous task to Buffer zone;
When the amount of storage for detecting the synchrodata cached in the buffer zone reaches the first predetermined threshold value, or it is described When task amount in task queue reaches the second predetermined threshold value, start same in task queue described in the thread process in thread pool Step task, the synchrodata stored in the buffer zone is sent to external management system.
Alternatively, it is described when the amount of storage for detecting the synchrodata cached in the buffer zone reaches the first default threshold Value, or when task amount in the task queue reaches the second predetermined threshold value, start and appoint described in the thread process in thread pool The synchronous task being engaged in queue, the synchrodata stored in the buffer zone is sent can to the step of external management system To replace with:
When detecting that synchronization of time intenals reaches prefixed time interval, start task described in the thread process in thread pool Synchronous task in queue, the synchrodata stored in the buffer zone is sent to external management system.
Alternatively, the method for data synchronization further includes:
During synchronous task in the thread process task queue in starting thread pool, according to pending synchronous task The maximum thread amount of total quantity and the pre-set thread pool, dynamically adjusts the quantity of the thread of startup.
Alternatively, it is described by synchronous task be added to task queue in step before, the method further includes as follows Step:
Getting with after the renewal corresponding synchrodata of request, mechanism mark is obtained from the renewal request Know, to determine the priority of the corresponding agency node of the request to be updated according to the authority identification;
The step that synchronous task is added in task queue includes:
The synchronous task is added to the task according to the priority order from high to low of corresponding agency node In queue.
In addition, to achieve the above object, it is described computer-readable the present invention also provides a kind of computer-readable recording medium Data synchronization program is stored with storage medium, the data synchronization program can be performed by one or more processor, with reality Now the step of method of data synchronization as described above.
Data synchronization unit, method and computer-readable recording medium proposed by the present invention, investigation to have renewal request When, data to be updated are obtained from renewal request, and synchrodata is generated according to the data of acquisition, establish and the same step number According to corresponding synchronous task, which is added in task queue, while above-mentioned synchrodata is associated with synchronous task After be cached in buffer zone, when the amount of storage of the synchrodata for the caching into buffer zone of being in reaches predetermined threshold value, Start the synchronous task in the thread start to process task queue in thread pool, the synchrodata cached in buffer zone is sent out It is sent in the external management system with docking, the present invention is by above-mentioned task list mode data cached at the same time of establishing to data Carry out the synchronization process of batch, it is not necessary to renewal request is all triggered when there is renewal to ask each time, avoids that code occur superfluous Remaining situation, in addition, by the task of the thread process task queue in thread pool, drastically increases the effect of data synchronization Rate.
Brief description of the drawings
Fig. 1 is the schematic diagram of data synchronization unit preferred embodiment of the present invention;
Fig. 2 is the high-level schematic functional block diagram of data synchronization program in one embodiment of data synchronization unit of the present invention;
Fig. 3 is the flow chart of method of data synchronization preferred embodiment of the present invention.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Embodiment
It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, it is not intended to limit the present invention.
The present invention provides a kind of data synchronization unit.It is data synchronization unit preferred embodiment of the present invention with reference to shown in Fig. 1 Schematic diagram.
In the present embodiment, data synchronization unit can be PC (Personal Computer, PC), can also It is smart mobile phone, tablet computer, E-book reader, MP3 (Moving Picture Experts Group Audio Layer III, dynamic image expert's compression standard audio aspect 3) player, MP4 (Moving Picture Experts Group Audio Layer IV, dynamic image expert's compression standard audio aspect 4) player, pocket computer etc. have display function Packaged type terminal device.
The data synchronization unit includes memory 11, processor 12, communication bus 13, and network interface 14.
Wherein, memory 11 includes at least a type of readable storage medium storing program for executing, the readable storage medium storing program for executing include flash memory, Hard disk, multimedia card, card-type memory (for example, SD or DX memories etc.), magnetic storage, disk, CD etc..Memory 11 Can be the internal storage unit of data synchronization unit in certain embodiments, such as the hard disk of the data synchronization unit.Storage Device 11 can also be in further embodiments the External memory equipment of data synchronization unit, such as be equipped with data synchronization unit Plug-in type hard disk, intelligent memory card (Smart Media Card, SMC), secure digital (Secure Digital, SD) card, Flash card (Flash Card) etc..Further, memory 11 can also both include the internal storage unit of data synchronization unit Also External memory equipment is included.Memory 11 can be not only used for storage and be installed on the application software of data synchronization unit and all kinds of Data, such as code of data synchronization program etc., can be also used for temporarily storing the data that has exported or will export.
Processor 12 can be in certain embodiments a central processing unit (Central Processing Unit, CPU), controller, microcontroller, microprocessor or other data processing chips, for the program stored in run memory 11 Code or processing data, such as perform data synchronization program etc..
Communication bus 13 is used for realization the connection communication between these components.
Network interface 14 can optionally include standard wireline interface and wireless interface (such as WI-FI interfaces), be commonly used in Communication connection is established between the device and other electronic equipments.
Fig. 1 illustrate only the data synchronization unit with component 11-14 and data synchronization program, it should be understood that It is, it is not required that implement all components shown, the more or less component of the implementation that can be substituted.
Alternatively, which can also include user interface, and user interface can include display (Display), input Unit such as keyboard (Keyboard), optional user interface can also include standard wireline interface and wireless interface.It is optional Ground, in certain embodiments, display can be light-emitting diode display, liquid crystal display, touch-control liquid crystal display and OLED (Organic Light-Emitting Diode, Organic Light Emitting Diode) touches device etc..Wherein, what display can also be suitably Referred to as display screen or display unit, for being shown in the information that is handled in data synchronization unit and for showing visual use Family interface.
Alternatively, which can also include touch sensor.What the touch sensor was provided is touched for user The region for touching operation is known as touch area.In addition, touch sensor described here can be resistive touch sensor, capacitance Formula touch sensor etc..Moreover, the touch sensor not only includes the touch sensor of contact, proximity may also comprise Touch sensor etc..In addition, the touch sensor can be single sensor, or such as multiple biographies of array arrangement Sensor.The area of the display of the device can be identical with the area of the touch sensor, can not also be same.Alternatively, will Display is set with touch sensor stacking, to form touch display screen.The device is based on touch display screen detecting user The touch control operation of triggering.
In the device embodiment shown in Fig. 1, data synchronization program is stored with memory 11;Processor 12 performs storage Following steps are realized during the data synchronization program stored in device 11:
Receive renewal request when, obtain with it is described renewal the corresponding synchrodata of request, and foundation with it is described synchronous The corresponding synchronous task of data.
In order to make it easy to understand, herein below by taking the data update in training class's management system as an example to the embodiment of the present invention Scheme illustrate, training class's management system may operate in the sychronisation of the present embodiment proposition, and sychronisation can be with One or more training class's management client establishes telecommunication, if client generates new data in use, Then need these data updates to external management system synchronously.Such as there are the phases such as lesson data, demographic data in client Close data increase occurs, deletes, change when operation, renewal request can be sent from client to sychronisation, these are operated New data after data update is sent in the external management system docked with sychronisation, such as Service Synthesis Management System.This reality The sychronisation for applying example proposition is docked by specific interface with external management system.
Client detects above-mentioned renewal operation in real time, for example, for dependency numbers such as lesson data, demographic datas According to increase operation, delete operation, modification operation etc..When the renewal for detecting the above-mentioned type operates, renewal operation is obtained Corresponding data content.By taking the modification to lesson data as an example, when detecting modification operation, obtain modification and operate corresponding repair The position where data content and the data content after changing, generates synchrodata, and client is triggered based on the synchrodata Renewal asks to send to sychronisation, and sychronisation obtains corresponding synchronous with renewal request when receiving renewal request Data, and establish synchronous task corresponding with synchrodata.
For being sent subsequently through the updating device of the present embodiment to the external management system docked with updating device, for External management system repaiies the data of corresponding position according to amended data content according to the synchrodata received Change, to realize synchronized update of the data in different system.
Synchronous task is added in task queue, and store after the synchrodata is associated with the synchronous task to Buffer zone.
In the present embodiment, a task queue is pre-established, the synchronous task established in updating device is added to this It is engaged in queue, as pending task.On the order of addition of synchronous task, in certain embodiments, can have it is new When synchronous task produces, by the synchronous task according to the settling time of the synchronous task by order of the elder generation after added to described In task queue.
Alternatively, in other embodiments, arranged according to the priority orders of each agency node of training class's management system Task in task queue.
Specifically, processor 12 is additionally operable to perform the data synchronization program, to appoint in described be added to synchronous task It is engaged in before the step in queue, also realizes following steps:Get with it is described renewal the corresponding synchrodata of request after, from Authority identification is obtained in the renewal request, to determine the corresponding agency node of the request to be updated according to the authority identification Priority.
The step that synchronous task is added in task queue is included:By the synchronous task according to corresponding agency node Priority order from high to low be added in the task queue.
Said mechanism node is to be distributed in each local mechanism of training class, these mechanisms are managed using same training class Client, these clients carry the mechanism mark of mechanism when sending renewal request to sychronisation in renewal is asked Know.In addition, sychronisation is provided with different priority orders for each agency node in advance, for the high mechanism section of priority The data of point renewal, can preferentially synchronize, the data updated for the low agency node of priority, can be relatively same late Step.The authority identification that sychronisation carries in being asked according to renewal determines that corresponding structure node is asked in the renewal, and according to pre- The priority orders of the agency node first set, determine the priority of the agency node, are arrived by corresponding synchrodata storage In task list, order according to the priority is stored.
In addition, division has a certain size storage region in advance in sychronisation, it is corresponding same for storing synchronous task Step data, the size of the storage region can be configured according to actual applicable cases.
When the amount of storage for detecting the synchrodata cached in the buffer zone reaches the first predetermined threshold value, or it is described When task amount in task queue reaches the second predetermined threshold value, start same in task queue described in the thread process in thread pool Step task, the synchrodata stored in the buffer zone is sent to external management system.
Task quantity in the sychronisation amount of storage to the synchrodata in buffer zone or task queue in real time It is detected, when the amount of storage of the synchrodata cached in buffer zone reaches the first predetermined threshold value, or in task queue When task amount reaches the second predetermined threshold value, start the thread in thread pool, synchronization is performed according to the order of task in task queue Task.It is understood that can according to task amount need pre-set maximum thread, every time start thread pool perform During task, multiple threads no more than maximum thread can be opened according to the task amount in task queue.
Alternatively, as a kind of embodiment, when the amount of storage for detecting the synchrodata cached in buffer zone reaches pre- If during threshold value, start the synchronous task in the thread process task queue in thread pool, the synchrodata of caching is synchronized to Step in second system could alternatively be following steps:When detecting that synchronization of time intenals reaches prefixed time interval, open Synchronous task in task queue described in thread process in dynamic thread pool, the synchrodata that will be stored in the buffer zone Send to external management system.
The data synchronization unit that the present embodiment proposes, in investigation to when having renewal to ask, obtains from renewal request and treats The data of renewal, and synchrodata is generated according to the data of acquisition, synchronous task corresponding with the synchrodata is established, by the synchronization Task is added in task queue, while is cached to after above-mentioned synchrodata is associated with synchronous task in buffer zone, is in When the amount of storage of the synchrodata of caching into buffer zone reaches predetermined threshold value, start the thread beginning in thread pool The synchronous task in task queue is managed, the synchrodata cached in buffer zone is sent to the external management system with docking In, the present invention is not required to by the above-mentioned synchronization process established task list mode data cached at the same time and data are carried out with batch Renewal request is all triggered when there is renewal to ask each time, the situation for code redundancy occur is avoided, in addition, passing through thread pool In thread process task queue task, drastically increase the efficiency of data synchronization.
Alternatively, in other examples, data synchronization program can also be divided into one or more module, and one A or multiple modules are stored in memory 11, and are held by one or more processors (the present embodiment is by processor 12) For row to complete the present invention, the module alleged by the present invention is the series of computation machine programmed instruction section for referring to complete specific function, For describing implementation procedure of the data synchronization program in data synchronization unit.
It is the function mould of the data synchronization program in one embodiment of data synchronization unit of the present invention for example, referring to shown in Fig. 2 Block schematic diagram, in the embodiment, data synchronization program can be divided into data processing module 10, task creation module 20, number According to memory module 30 and data transmission blocks 40, exemplarily:
Data processing module 10 is used for:When receiving renewal request, same step number corresponding with the renewal request is obtained According to;
Task creation module 20 is used for:Establish synchronous task corresponding with the synchrodata;
Data memory module 30 is used for:By synchronous task be added to task queue in, and by the synchrodata with it is described Stored after synchronous task association to buffer zone;
Data transmission blocks 40 are used for:When the amount of storage for detecting the synchrodata cached in the buffer zone reaches When task amount in one predetermined threshold value, or the task queue reaches the second predetermined threshold value, start at the thread in thread pool The synchronous task in the task queue is managed, the synchrodata stored in the buffer zone is sent to external management system System.
Above-mentioned data processing module 10, task creation module 20, data memory module 30 and data transmission blocks 40 are held The realized functions or operations step of row is substantially the same with above-described embodiment, and details are not described herein.
In addition, the present invention also provides a kind of method of data synchronization.With reference to shown in Fig. 3, be method of data synchronization of the present invention compared with The flow chart of good embodiment.
In the present embodiment, method of data synchronization includes:
Step S10, receive renewal request when, obtain with it is described renewal the corresponding synchrodata of request, and foundation and The corresponding synchronous task of the synchrodata.
The method that the present embodiment proposes can be performed by a device, and sychronisation can be by software and/or hardware realization. The device can be a sychronisation.In order to make it easy to understand, herein below is with to the data update in training class's management system Exemplified by the scheme of the embodiment of the present invention is illustrated, training class's management system may operate at the sychronisation of the present embodiment proposition In, sychronisation can establish telecommunication with one or more training class's management client, if client is using process In generate new data, then need these data updates to external management system synchronously.Such as there is course in client The related datas such as data, demographic data occur increase, delete, change when operation, can be sent more from client to sychronisation New request, these operations is sent in the external management system docked with sychronisation the new data after data update, such as Service Synthesis Management System.The sychronisation that the present embodiment proposes is docked by specific interface with external management system.
Client detects above-mentioned renewal operation in real time, for example, for dependency numbers such as lesson data, demographic datas According to increase operation, delete operation, modification operation etc..When the renewal for detecting the above-mentioned type operates, renewal operation is obtained Corresponding data content.By taking the modification to lesson data as an example, when detecting modification operation, obtain modification and operate corresponding repair The position where data content and the data content after changing, generates synchrodata, and client is triggered based on the synchrodata Renewal asks to send to sychronisation, and sychronisation obtains corresponding synchronous with renewal request when receiving renewal request Data, and establish synchronous task corresponding with synchrodata.
For being sent subsequently through the updating device of the present embodiment to the external management system docked with updating device, for External management system repaiies the data of corresponding position according to amended data content according to the synchrodata received Change, to realize synchronized update of the data in different system.
Step S20, synchronous task is added in task queue, and the synchrodata is associated with the synchronous task After store to buffer zone.
In the present embodiment, a task queue is pre-established, the synchronous task established in updating device is added to this It is engaged in queue, as pending task.On the order of addition of synchronous task, in certain embodiments, can have it is new When synchronous task produces, by the synchronous task according to the settling time of the synchronous task by order of the elder generation after added to described In task queue.
Alternatively, in other embodiments, arranged according to the priority orders of each agency node of training class's management system Task in task queue.
Specifically, it is described by synchronous task be added to task queue in step before, this method further includes following step Suddenly:Getting with after the renewal corresponding synchrodata of request, authority identification being obtained from the renewal request, with root The priority of the corresponding agency node of the request to be updated is determined according to the authority identification.
The step that synchronous task is added in task queue is included:By the synchronous task according to corresponding agency node Priority order from high to low be added in the task queue.
Said mechanism node is to be distributed in each local mechanism of training class, these mechanisms are managed using same training class Client, these clients carry the mechanism mark of mechanism when sending renewal request to sychronisation in renewal is asked Know.In addition, sychronisation is provided with different priority orders for each agency node in advance, for the high mechanism section of priority The data of point renewal, can preferentially synchronize, the data updated for the low agency node of priority, can be relatively same late Step.The authority identification that sychronisation carries in being asked according to renewal determines that corresponding structure node is asked in the renewal, and according to pre- The priority orders of the agency node first set, determine the priority of the agency node, are arrived by corresponding synchrodata storage In task list, order according to the priority is stored.
In addition, division has a certain size storage region in advance in sychronisation, it is corresponding same for storing synchronous task Step data, the size of the storage region can be configured according to actual applicable cases.
Step S30, when the amount of storage for detecting the synchrodata cached in the buffer zone reaches the first predetermined threshold value, Or the task amount in the task queue starts task team described in the thread process in thread pool when reaching the second predetermined threshold value Synchronous task in row, the synchrodata stored in the buffer zone is sent to external management system.
Task quantity in the sychronisation amount of storage to the synchrodata in buffer zone or task queue in real time It is detected, when the amount of storage of the synchrodata cached in buffer zone reaches the first predetermined threshold value, or in task queue When task amount reaches the second predetermined threshold value, start the thread in thread pool, synchronization is performed according to the order of task in task queue Task.It is understood that can according to task amount need pre-set maximum thread, every time start thread pool perform During task, multiple threads no more than maximum thread can be opened according to the task amount in task queue.
Alternatively, as a kind of embodiment, when the amount of storage for detecting the synchrodata cached in buffer zone reaches pre- If during threshold value, start the synchronous task in the thread process task queue in thread pool, the synchrodata of caching is synchronized to Step in second system could alternatively be following steps:When detecting that synchronization of time intenals reaches prefixed time interval, open Synchronous task in task queue described in thread process in dynamic thread pool, the synchrodata that will be stored in the buffer zone Send to external management system.
The method of data synchronization that the present embodiment proposes, in investigation to when having renewal to ask, obtains from renewal request and treats The data of renewal, and synchrodata is generated according to the data of acquisition, synchronous task corresponding with the synchrodata is established, by the synchronization Task is added in task queue, while is cached to after above-mentioned synchrodata is associated with synchronous task in buffer zone, is in When the amount of storage of the synchrodata of caching into buffer zone reaches predetermined threshold value, start the thread beginning in thread pool The synchronous task in task queue is managed, the synchrodata cached in buffer zone is sent to the external management system with docking In, the present invention is not required to by the above-mentioned synchronization process established task list mode data cached at the same time and data are carried out with batch Renewal request is all triggered when there is renewal to ask each time, the situation for code redundancy occur is avoided, in addition, passing through thread pool In thread process task queue task, drastically increase the efficiency of data synchronization.
In addition, the embodiment of the present invention also proposes a kind of computer-readable recording medium, the computer-readable recording medium On be stored with data synchronization program, the data synchronization program can be executed by one or more processors, to realize following operation:
Receive renewal request when, obtain with it is described renewal the corresponding synchrodata of request, and foundation with it is described synchronous The corresponding synchronous task of data;
Synchronous task is added in task queue, and store after the synchrodata is associated with the synchronous task to Buffer zone;
When the amount of storage for detecting the synchrodata cached in the buffer zone reaches the first predetermined threshold value, or it is described When task amount in task queue reaches the second predetermined threshold value, start same in task queue described in the thread process in thread pool Step task, the synchrodata stored in the buffer zone is sent to external management system.
Further, it is described to be preset when the amount of storage for detecting the synchrodata cached in the buffer zone reaches first When task amount in threshold value, or the task queue reaches the second predetermined threshold value, start described in the thread process in thread pool Synchronous task in task queue, the step of the synchrodata stored in the buffer zone is sent to external management system It could alternatively be following steps:
When detecting that synchronization of time intenals reaches prefixed time interval, start task described in the thread process in thread pool Synchronous task in queue, the synchrodata stored in the buffer zone is sent to external management system.
Further, following operation is also realized when the data synchronization program is executed by processor:
During synchronous task in the thread process task queue in starting thread pool, according to pending synchronous task The maximum thread amount of total quantity and the pre-set thread pool, dynamically adjusts the quantity of the thread of startup.
Further, following operation is also realized when the data synchronization program is executed by processor:
The synchronous task is added to the task according to the settling time of the synchronous task by order of the elder generation after In queue.
Computer-readable recording medium embodiment of the present invention and above-mentioned data synchronization unit and each embodiment of method It is essentially identical, do not make tired state herein.
It should be noted that the embodiments of the present invention are for illustration only, the quality of embodiment is not represented.And Term " comprising " herein, "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that bag To include process, device, article or the method for a series of elements not only include those key elements, but also including being not explicitly listed Other element, or further include as this process, device, article or the intrinsic key element of method.Do not limiting more In the case of, the key element that is limited by sentence "including a ...", it is not excluded that in the process including the key element, device, article Or also there are other identical element in method.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side Method can add the mode of required general hardware platform to realize by software, naturally it is also possible to by hardware, but in many cases The former is more preferably embodiment.Based on such understanding, technical scheme substantially in other words does the prior art Going out the part of contribution can be embodied in the form of software product, which is stored in one as described above In storage medium (such as ROM/RAM, magnetic disc, CD), including some instructions use so that a station terminal equipment (can be mobile phone, Computer, server, or network equipment etc.) perform method described in each embodiment of the present invention.
It these are only the preferred embodiment of the present invention, be not intended to limit the scope of the invention, it is every to utilize this hair The equivalent structure or equivalent flow shift that bright specification and accompanying drawing content are made, is directly or indirectly used in other relevant skills Art field, is included within the scope of the present invention.

Claims (10)

1. a kind of data synchronization unit, it is characterised in that described device includes memory and processor, is stored on the memory There is the data synchronization program that can be run on the processor, realized such as when the data synchronization program is performed by the processor Lower step:
When receiving renewal request, obtain and update the corresponding synchrodata of request with described, and establish and the synchrodata Corresponding synchronous task;
Synchronous task is added in task queue, and is stored after the synchrodata is associated with the synchronous task to caching Region;
When the amount of storage for detecting the synchrodata cached in the buffer zone reaches the first predetermined threshold value, or the task When task amount in queue reaches the second predetermined threshold value, the synchronization started in task queue described in the thread process in thread pool is appointed Business, the synchrodata stored in the buffer zone is sent to external management system.
2. data synchronization unit according to claim 1, it is characterised in that described slow in the buffer zone when detecting The task amount that the amount of storage for the synchrodata deposited reaches in the first predetermined threshold value, or the task queue reaches the second default threshold During value, start the synchronous task in task queue described in the thread process in thread pool, by what is stored in the buffer zone Synchrodata sends to the step of external management system and could alternatively be following steps:
When detecting that synchronization of time intenals reaches prefixed time interval, start task queue described in the thread process in thread pool In synchronous task, the synchrodata stored in the buffer zone is sent to external management system.
3. data synchronization unit according to claim 1, it is characterised in that the processor is additionally operable to perform the data Synchronization program, to realize following steps:
During synchronous task in the thread process task queue in starting thread pool, according to the sum of pending synchronous task The maximum thread amount of amount and the pre-set thread pool, dynamically adjusts the quantity of the thread of startup.
4. data synchronization unit according to any one of claim 1 to 3, it is characterised in that described to add synchronous task The step added in task queue includes:
The synchronous task is added to the task queue according to the settling time of the synchronous task by order of the elder generation after In.
5. data synchronization unit according to any one of claim 1 to 3, it is characterised in that the processor is additionally operable to Perform the data synchronization program, with it is described by synchronous task be added to task queue in step before, also realize it is as follows Step:
Getting with after the renewal corresponding synchrodata of request, authority identification is obtained from the renewal request, with The priority of the corresponding agency node of the request to be updated is determined according to the authority identification;
The step that synchronous task is added in task queue includes:
The synchronous task is added to the task queue according to the priority order from high to low of corresponding agency node In.
A kind of 6. method of data synchronization, it is characterised in that the described method includes:
When receiving renewal request, obtain and update the corresponding synchrodata of request with described, and establish and the synchrodata Corresponding synchronous task;
Synchronous task is added in task queue, and is stored after the synchrodata is associated with the synchronous task to caching Region;
When the amount of storage for detecting the synchrodata cached in the buffer zone reaches the first predetermined threshold value, or the task When task amount in queue reaches the second predetermined threshold value, the synchronization started in task queue described in the thread process in thread pool is appointed Business, the synchrodata stored in the buffer zone is sent to external management system.
7. method of data synchronization according to claim 6, it is characterised in that described slow in the buffer zone when detecting The task amount that the amount of storage for the synchrodata deposited reaches in the first predetermined threshold value, or the task queue reaches the second default threshold During value, start the synchronous task in task queue described in the thread process in thread pool, by what is stored in the buffer zone Synchrodata sends to the step of external management system and could alternatively be:
When detecting that synchronization of time intenals reaches prefixed time interval, start task queue described in the thread process in thread pool In synchronous task, the synchrodata stored in the buffer zone is sent to external management system.
8. method of data synchronization according to claim 6, it is characterised in that the method for data synchronization further includes:
During synchronous task in the thread process task queue in starting thread pool, according to the sum of pending synchronous task The maximum thread amount of amount and the pre-set thread pool, dynamically adjusts the quantity of the thread of startup.
9. the method for data synchronization according to any one of claim 6 to 8, it is characterised in that described by synchronous task Added to before the step in task queue, the method further includes following steps:
Getting with after the renewal corresponding synchrodata of request, authority identification is obtained from the renewal request, with The priority of the corresponding agency node of the request to be updated is determined according to the authority identification;
The step that synchronous task is added in task queue includes:
The synchronous task is added to the task queue according to the priority order from high to low of corresponding agency node In.
10. a kind of computer-readable recording medium, it is characterised in that it is same to be stored with data on the computer-readable recording medium Program is walked, the data synchronization program can be performed by one or more processor, to realize as any in claim 6 to 9 Described in method of data synchronization the step of.
CN201710916482.9A 2017-09-30 2017-09-30 Data synchronization unit, method and computer-readable recording medium Pending CN107908472A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201710916482.9A CN107908472A (en) 2017-09-30 2017-09-30 Data synchronization unit, method and computer-readable recording medium
PCT/CN2018/089187 WO2019062182A1 (en) 2017-09-30 2018-05-31 Data synchronization device and method, and computer readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710916482.9A CN107908472A (en) 2017-09-30 2017-09-30 Data synchronization unit, method and computer-readable recording medium

Publications (1)

Publication Number Publication Date
CN107908472A true CN107908472A (en) 2018-04-13

Family

ID=61841078

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710916482.9A Pending CN107908472A (en) 2017-09-30 2017-09-30 Data synchronization unit, method and computer-readable recording medium

Country Status (2)

Country Link
CN (1) CN107908472A (en)
WO (1) WO2019062182A1 (en)

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108572919A (en) * 2018-05-30 2018-09-25 平安普惠企业管理有限公司 Automated testing method, device, computer equipment and storage medium
CN109116818A (en) * 2018-08-08 2019-01-01 新智能源系统控制有限责任公司 Real time data dump method and device when a kind of SCADA system upgrades
CN109299122A (en) * 2018-09-26 2019-02-01 努比亚技术有限公司 A kind of method of data synchronization, equipment and computer can storage mediums
CN109542428A (en) * 2018-10-16 2019-03-29 平安普惠企业管理有限公司 Method for processing business, device, computer equipment and storage medium
CN109542216A (en) * 2018-10-11 2019-03-29 平安科技(深圳)有限公司 Man-machine interaction method, system, computer equipment and storage medium
WO2019062182A1 (en) * 2017-09-30 2019-04-04 平安科技(深圳)有限公司 Data synchronization device and method, and computer readable storage medium
CN109634853A (en) * 2018-11-30 2019-04-16 平安科技(深圳)有限公司 Data test method, system, electronic device and computer readable storage medium
CN110196884A (en) * 2019-05-31 2019-09-03 北京大米科技有限公司 Method for writing data, storage medium and electronic equipment based on distributed data base
CN110602165A (en) * 2019-08-08 2019-12-20 数字广东网络建设有限公司 Government affair data synchronization method, device, system, computer equipment and storage medium
CN110765288A (en) * 2019-09-04 2020-02-07 北京旷视科技有限公司 Image information synchronization method, device and system and storage medium
CN111340202A (en) * 2018-12-18 2020-06-26 上海寒武纪信息科技有限公司 Operation method, device and related product
CN111488366A (en) * 2020-04-09 2020-08-04 百度在线网络技术(北京)有限公司 Relational database updating method, device, equipment and storage medium
CN111666339A (en) * 2020-05-26 2020-09-15 和芯星通科技(北京)有限公司 Multithreading data synchronization method
CN111865834A (en) * 2019-04-26 2020-10-30 华为技术有限公司 Message processing method and device
CN112818054A (en) * 2020-10-15 2021-05-18 广州南天电脑系统有限公司 Data synchronization method and device, computer equipment and storage medium
WO2023279677A1 (en) * 2021-07-06 2023-01-12 上海商汤智能科技有限公司 Curriculum customization method, device, apparatus, storage medium and program

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110569123B (en) * 2019-07-31 2022-08-02 苏宁云计算有限公司 Thread allocation method and device, computer equipment and storage medium
CN111444203B (en) * 2020-03-24 2021-09-28 腾讯科技(深圳)有限公司 Synchronous processing method, device, equipment and medium
CN112187889A (en) * 2020-09-15 2021-01-05 中信银行股份有限公司 Data synchronization method, device and storage medium
CN113467661A (en) * 2021-07-19 2021-10-01 维沃移动通信有限公司 Task synchronization method, device, equipment and readable storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140279871A1 (en) * 2013-03-13 2014-09-18 Marcelo Ochoa System and method for providing near real time data synchronization
CN105208060A (en) * 2014-06-19 2015-12-30 阿里巴巴集团控股有限公司 Service data synchronization method, service data synchronization device and service data synchronization system
CN105677849A (en) * 2016-01-06 2016-06-15 北京京东尚科信息技术有限公司 Data updating method and device
CN106156165A (en) * 2015-04-16 2016-11-23 阿里巴巴集团控股有限公司 Method of data synchronization between heterogeneous data source and device
CN106412088A (en) * 2016-10-25 2017-02-15 深圳市万普拉斯科技有限公司 Data synchronization method and terminal

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6446176B1 (en) * 2000-03-09 2002-09-03 Storage Technology Corporation Method and system for transferring data between primary storage and secondary storage using a bridge volume and an internal snapshot copy of the data being transferred
CN104408048B (en) * 2014-10-27 2017-06-13 清华大学 A kind of method and apparatus of buffer-type cloud storage data syn-chronization
CN107908472A (en) * 2017-09-30 2018-04-13 平安科技(深圳)有限公司 Data synchronization unit, method and computer-readable recording medium

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140279871A1 (en) * 2013-03-13 2014-09-18 Marcelo Ochoa System and method for providing near real time data synchronization
CN105208060A (en) * 2014-06-19 2015-12-30 阿里巴巴集团控股有限公司 Service data synchronization method, service data synchronization device and service data synchronization system
CN106156165A (en) * 2015-04-16 2016-11-23 阿里巴巴集团控股有限公司 Method of data synchronization between heterogeneous data source and device
CN105677849A (en) * 2016-01-06 2016-06-15 北京京东尚科信息技术有限公司 Data updating method and device
CN106412088A (en) * 2016-10-25 2017-02-15 深圳市万普拉斯科技有限公司 Data synchronization method and terminal

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2019062182A1 (en) * 2017-09-30 2019-04-04 平安科技(深圳)有限公司 Data synchronization device and method, and computer readable storage medium
CN108572919A (en) * 2018-05-30 2018-09-25 平安普惠企业管理有限公司 Automated testing method, device, computer equipment and storage medium
CN109116818A (en) * 2018-08-08 2019-01-01 新智能源系统控制有限责任公司 Real time data dump method and device when a kind of SCADA system upgrades
CN109299122A (en) * 2018-09-26 2019-02-01 努比亚技术有限公司 A kind of method of data synchronization, equipment and computer can storage mediums
CN109542216A (en) * 2018-10-11 2019-03-29 平安科技(深圳)有限公司 Man-machine interaction method, system, computer equipment and storage medium
US11842574B2 (en) 2018-10-11 2023-12-12 Ping An Technology (Shenzhen) Co., Ltd. Man-machine interaction method and system, computer device, and storage medium
CN109542428A (en) * 2018-10-16 2019-03-29 平安普惠企业管理有限公司 Method for processing business, device, computer equipment and storage medium
CN109542428B (en) * 2018-10-16 2024-06-11 北京神州数码云科信息技术有限公司 Business processing method, device, computer equipment and storage medium
CN109634853A (en) * 2018-11-30 2019-04-16 平安科技(深圳)有限公司 Data test method, system, electronic device and computer readable storage medium
CN111340202A (en) * 2018-12-18 2020-06-26 上海寒武纪信息科技有限公司 Operation method, device and related product
CN111340202B (en) * 2018-12-18 2023-06-09 上海寒武纪信息科技有限公司 Operation method, device and related product
CN111865834A (en) * 2019-04-26 2020-10-30 华为技术有限公司 Message processing method and device
CN111865834B (en) * 2019-04-26 2021-12-03 华为技术有限公司 Message processing method and device
CN110196884A (en) * 2019-05-31 2019-09-03 北京大米科技有限公司 Method for writing data, storage medium and electronic equipment based on distributed data base
CN110602165A (en) * 2019-08-08 2019-12-20 数字广东网络建设有限公司 Government affair data synchronization method, device, system, computer equipment and storage medium
CN110765288A (en) * 2019-09-04 2020-02-07 北京旷视科技有限公司 Image information synchronization method, device and system and storage medium
CN110765288B (en) * 2019-09-04 2022-09-27 北京旷视科技有限公司 Image information synchronization method, device and system and storage medium
CN111488366A (en) * 2020-04-09 2020-08-04 百度在线网络技术(北京)有限公司 Relational database updating method, device, equipment and storage medium
CN111666339A (en) * 2020-05-26 2020-09-15 和芯星通科技(北京)有限公司 Multithreading data synchronization method
CN111666339B (en) * 2020-05-26 2023-10-24 和芯星通科技(北京)有限公司 Multithreading data synchronization method
CN112818054A (en) * 2020-10-15 2021-05-18 广州南天电脑系统有限公司 Data synchronization method and device, computer equipment and storage medium
WO2023279677A1 (en) * 2021-07-06 2023-01-12 上海商汤智能科技有限公司 Curriculum customization method, device, apparatus, storage medium and program

Also Published As

Publication number Publication date
WO2019062182A1 (en) 2019-04-04

Similar Documents

Publication Publication Date Title
CN107908472A (en) Data synchronization unit, method and computer-readable recording medium
CN104412249B (en) File disposal in file system based on cloud
CN108376079B (en) Automatic application updates
CN104704468B (en) The cross-system installation of WEB application program
CN109446570B (en) Business process display method, device, computer equipment and storage medium
US20130283267A1 (en) Virtual machine construction
CN108427698A (en) Updating device, method and the computer readable storage medium of prediction model
CN109816481A (en) Billing processing method, apparatus and computer-readable storage medium
CN107038044A (en) The discrete loading method of game resource and system based on Unity3D
CN101840418A (en) User word library synchronous update method, update server and input method system
CN106201613A (en) Preheating software is installed
US9411640B2 (en) Method for efficiently managing application and electronic device implementing the method
CN107515933A (en) Change method, server, system and the storage medium of source data database table structure
CN109614238A (en) A target object identification method, device, system and readable storage medium
CN107656729A (en) Updating device, method and the computer-readable recording medium of List View
CN108958881A (en) Data processing method, device and computer readable storage medium
CN108055320B (en) Processing method, terminal device and the computer readable storage medium of resource downloading
CN107092652B (en) Navigation method and device for target page
CN107357857A (en) A kind of method and service node device for updating cache information
US10242102B2 (en) Network crawling prioritization
CN109254808A (en) Interface call method, device and computer readable storage medium
CN105871695A (en) Emoji sending method and device
CN107408065A (en) Supervision application loads
CN107247550A (en) One kind application switching method and device, computer installation, readable storage medium storing program for executing
CN104424224B (en) A kind of file index storage method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20180413