CN110795505A - Cross-domain synchronous distribution system and method for user data - Google Patents
Cross-domain synchronous distribution system and method for user data Download PDFInfo
- Publication number
- CN110795505A CN110795505A CN201911012304.9A CN201911012304A CN110795505A CN 110795505 A CN110795505 A CN 110795505A CN 201911012304 A CN201911012304 A CN 201911012304A CN 110795505 A CN110795505 A CN 110795505A
- Authority
- CN
- China
- Prior art keywords
- user data
- data
- module
- queue
- receiving
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
- G06F21/60—Protecting data
- G06F21/64—Protecting data integrity, e.g. using checksums, certificates or signatures
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/54—Interprogram communication
- G06F9/546—Message passing systems or structures, e.g. queues
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Security & Cryptography (AREA)
- Databases & Information Systems (AREA)
- Bioethics (AREA)
- General Health & Medical Sciences (AREA)
- Computer Hardware Design (AREA)
- Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Data Exchanges In Wide-Area Networks (AREA)
Abstract
The invention discloses a cross-domain synchronous distribution system of user data, which comprises: the reading module is used for receiving a plurality of user data from different sources and generating corresponding labels according to the content of each user data, wherein each user data carries the corresponding label; the routing module is used for sending the user data to a plurality of data queues according to different sources of the user data and the labels carried by the user data respectively; a receiving module, configured to receive user data included in the data queue and generate a reception success signal; a verification module verifying whether the user data contained in the data queue is received by the reception module in response to the reception success signal; and the log module is used for storing the user data contained in the data queue received by the receiving module. The invention ensures the consistency and stability of the user data and improves the data quality and the data coordination efficiency.
Description
Technical Field
The invention relates to the technical field of Internet. And more particularly, to a system and method for cross-domain synchronized distribution of user data.
Background
The user data is basic data of the application systems, each application system is managed by a user, the source of the user data is mostly imported by adopting an initialized Excel table and is manually created by a system administrator, the method often causes inaccuracy of the user data, when the number of the application systems is large, the operation and maintenance cost is high, and the data is not updated timely.
With the rise of the main data technology, enterprises usually establish user main data of the unit, and the distribution synchronization of users is realized through a main data system. With the development of application system construction, all user data need to be integrated in an application system constructed by academy-level or even group-level units, and the user data of a specified unit can be received at the same time, which all put higher requirements on synchronous distribution of the user data.
Therefore, in order to satisfy the requirement of synchronously distributing user data from different sources to application systems from different sources and satisfy the requirement of mutually cooperating applications, a system and a method for synchronously distributing user data across domains need to be provided.
Disclosure of Invention
To achieve the above object, an embodiment of the present invention provides a cross-domain synchronous distribution system for user data, including:
the reading module is used for receiving a plurality of user data from different sources and generating corresponding labels according to the content of each user data, wherein each user data carries the corresponding label;
the routing module is used for sending the user data to a plurality of data queues according to different sources of the user data and the labels carried by the user data respectively;
a receiving module, configured to receive user data included in the data queue and generate a reception success signal;
a verification module verifying whether the user data contained in the data queue is received by the reception module in response to the reception success signal;
and the log module is used for storing the user data contained in the data queue received by the receiving module.
Optionally, the routing module is further configured to:
sending the user data containing the same label to the same data queue; and/or
And sending the user data of different sources to different data queues according to the source of each user data.
Optionally, the receiving module is further configured to:
and matching the label of the user data contained in the data queue with a preset subscription label, and receiving the user data contained in the data queue after the matching is successful.
Optionally, the verification module is further configured to:
and deleting the user data contained in the data queue when the verification result shows that the user data contained in the data queue is received by the receiving module.
Optionally, the method further comprises:
the timing module is used for periodically generating a reading signal;
wherein the read module receives a plurality of user data from different sources in response to the read signal.
Another embodiment of the present invention provides a cross-domain synchronous distribution method for user data, including the following steps:
receiving a plurality of user data from different sources, and generating a corresponding label according to the content of each user data, wherein each user data carries the corresponding label;
sending the user data to a plurality of data queues according to different sources of the user data and the labels carried by the user data respectively;
receiving user data contained in the data queue and generating a receiving success signal;
verifying whether user data contained in the data queue is received in response to the reception success signal;
and storing the user data contained in the received data queue.
Optionally, the sending the plurality of user data to the plurality of data queues according to different sources of the plurality of user data and the tags respectively carried by the plurality of user data includes:
sending the user data containing the same label to the same data queue; and/or
And sending the user data of different sources to different data queues according to the source of each user data.
Optionally, the receiving the user data included in the data queue includes:
and matching the label of the user data contained in the data queue with a preset subscription label, and receiving the user data contained in the data queue after the matching is successful.
Optionally, said verifying whether the user data contained in the data queue is received in response to the reception success signal comprises:
and deleting the user data contained in the data queue when the verification result shows that the user data contained in the data queue is received by the receiving module.
Optionally, the receiving a plurality of user data from different sources further comprises:
periodically generating a read signal;
a plurality of user data from different sources is received in response to the read signal.
The invention has the following beneficial effects:
the technical scheme of the invention has the advantages of clear principle and simple design, compared with the traditional method, the method can simultaneously acquire the user data from a plurality of different sources, send the user data to a plurality of data queues according to the labels and the sources carried by the user data, further receive the user data through the receiving module, and verify the transmission of the user data by adopting the verifying module in the transmission process of the user data, thereby preventing the data from being lost, ensuring the consistency and the stability of the user data, improving the data quality and the data coordination efficiency and reducing the operation and maintenance cost.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.
Fig. 1 shows a block diagram of a cross-domain synchronous distribution system for user data according to an embodiment of the present invention;
fig. 2 shows a flowchart of a method for synchronously distributing user data across domains according to another embodiment of the present invention.
Detailed Description
In order to make the technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.
One embodiment of the present invention provides a cross-domain synchronous distribution system for user data, as shown in fig. 1, including a reading module, a routing module, a receiving module, a verification module, and a log module.
Specifically, the reading module in this embodiment may be configured to receive a plurality of user data from different sources, and generate a corresponding tag according to a content of each of the user data, where each of the user data carries the corresponding tag, where the different sources ((in the example in fig. 1, three different sources are illustrated in common, respectively, a user system a, a user system B, and a user system C, it should be understood by those skilled in the art that the number of the source is not limited to three, and only is illustrated in fig. 1) may be understood as different user system terminals or different network security domains, and the reading module may be configured to receive the user data from the different sources at the same time and establish the corresponding tag according to the specific content of the user data, where a rule for establishing the tag may be set by a worker, and may be established according to a type or a content of the user data, for example: the website and the terminal may be established as different tags, and when the website and/or the terminal appear in the user data, the tags of the user data may include the website and/or the terminal, it should be noted that the tag content of the user data may be multiple, and after a corresponding tag is established for each user data, the user data is carried with the corresponding tag.
The routing module in this embodiment is configured to send the plurality of user data to the plurality of data queues according to different sources of the plurality of user data and the tags carried by the plurality of user data.
Specifically, in some optional implementation manners of this embodiment, the routing module is further configured to:
sending the user data containing the same label to the same data queue; and/or
And sending the user data of different sources to different data queues according to the source of each user data.
That is, the routing module may send multiple user data to multiple data queues according to a variety of different rules, such as: the user data including the same tag is sent to the same data queue and/or the user data from different sources is sent to different data queues according to the source of each user data, where the number of the data queues can be set by the staff, which is not specifically limited in this embodiment.
After the process of sending the user data to the data queue by the routing module is completed, the receiving module in this embodiment starts to receive the data queue and generates a receiving success signal.
After the receiving module receives the data queue, the receiving module correspondingly receives the user data contained in the data queue, and in some optional implementation manners of this embodiment, the receiving module is further configured to: and matching the label of the user data contained in the data queue with a preset subscription label, and receiving the user data contained in the data queue after the matching is successful.
That is to say, the receiving module may receive the data queue correspondingly according to a preset receiving rule, where the receiving module includes a preset subscription tag, and when the receiving module receives the data array, a user tag included in the data array may be matched with the preset subscription tag, and after the matching is successful, the receiving module may receive the user data included in the data queue.
As can be seen from the above, the receiving module generates a receiving success signal after receiving the user data included in the data array, and the verifying module in this embodiment verifies whether the user data included in the data queue is received by the receiving module in response to the receiving success signal.
In some optional implementation manners of this embodiment, when the verification result is that the user data included in the data queue has been received by the receiving module, the user data included in the data queue is deleted.
Specifically, in this embodiment, the verification module mainly plays a role of verification, when the verification result is that the user data in the data array has been received by the receiving module, the verification module deletes the user data included in the data array, and when the verification module does not receive the reception success signal generated by the receiving module or the verification result is that the user data in the data array is not received by the receiving module, the verification module continuously retains the user data in the data queue, thereby preventing the user data from being lost.
In summary, the technical scheme of the embodiment has the advantages of being clear in principle and simple in design, compared with a conventional method, the method can simultaneously acquire user data from a plurality of different sources, send the user data to a plurality of data queues according to tags and sources carried by the user data, further receive the user data through the receiving module, and verify the transmission of the user data by using the verifying module in the transmission process of the user data, so that data loss is prevented, the consistency and stability of the user data are ensured, the data quality and the data cooperation efficiency are improved, and the operation and maintenance cost is reduced.
In some optional implementations of this embodiment, the system further includes: and the log module is used for storing the user data contained in the data queue received by the receiving module.
Specifically, the log module is mainly used for storing the received user data to prevent the user data from being lost, and can record the data transmission condition among the modules, so that the whole data transmission process can be conveniently tracked.
In some optional implementations of this embodiment, the system further includes: and writing into a module.
Specifically, in this embodiment, the writing module is mainly configured to write the user data received by the receiving module into corresponding service systems (in the example in fig. 1, three different service systems, namely, a service system a, a service system B, and a service system C, are illustrated in total, and it should be understood by those skilled in the art that the number of the service systems is not limited to three, and is only illustrated in fig. 1), so that it can be ensured that the user data obtained in different service systems may not be consistent.
In some optional implementations of this embodiment, the system further includes:
and the timing module is used for periodically generating a reading signal.
Specifically, in this embodiment, the reading module receives a plurality of user data from different sources in response to the reading signal, so that the purpose that the reading module periodically obtains the user data can be achieved, where the period set by the timing module can be set by an operator according to an actual business requirement, which is not specifically limited in this embodiment.
Another embodiment of the present invention provides a cross-domain synchronous distribution method of user data, as shown in fig. 2, including the following steps:
receiving a plurality of user data from different sources, and generating a corresponding label according to the content of each user data, wherein each user data carries the corresponding label;
sending the user data to a plurality of data queues according to different sources of the user data and the labels carried by the user data respectively;
receiving user data contained in the data queue and generating a receiving success signal;
verifying whether user data contained in the data queue is received in response to the reception success signal;
and storing the user data contained in the received data queue.
In some optional implementation manners of this embodiment, the sending, according to different sources of the plurality of user data and the tags respectively carried by the user data, the plurality of user data to the plurality of data queues includes:
sending the user data containing the same label to the same data queue; and/or
And sending the user data of different sources to different data queues according to the source of each user data.
In some optional implementation manners of this embodiment, the receiving user data included in the data queue includes:
and matching the label of the user data contained in the data queue with a preset subscription label, and receiving the user data contained in the data queue after the matching is successful.
In some optional implementations of this embodiment, the verifying, in response to the reception success signal, whether the user data contained in the data queue is received includes:
and deleting the user data contained in the data queue when the verification result shows that the user data contained in the data queue is received by the receiving module.
In some optional implementations of this embodiment, the receiving the plurality of user data from different sources further includes:
periodically generating a read signal;
a plurality of user data from different sources is received in response to the read signal.
It should be understood that the above-mentioned embodiments of the present invention are only examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention, and it will be obvious to those skilled in the art that other variations or modifications may be made on the basis of the above description, and all embodiments may not be exhaustive, and all obvious variations or modifications may be included within the scope of the present invention.
Claims (10)
1. A system for synchronous cross-domain distribution of user data, comprising:
the reading module is used for receiving a plurality of user data from different sources and generating corresponding labels according to the content of each user data, wherein each user data carries the corresponding label;
the routing module is used for sending the user data to a plurality of data queues according to different sources of the user data and the labels carried by the user data respectively;
a receiving module, configured to receive user data included in the data queue and generate a reception success signal;
a verification module verifying whether the user data contained in the data queue is received by the reception module in response to the reception success signal;
and the log module is used for storing the user data contained in the data queue received by the receiving module.
2. The system of claim 1, wherein the routing module is further configured to:
sending the user data containing the same label to the same data queue; and/or
And sending the user data of different sources to different data queues according to the source of each user data.
3. The system of claim 1, wherein the receiving module is further configured to:
and matching the label of the user data contained in the data queue with a preset subscription label, and receiving the user data contained in the data queue after the matching is successful.
4. The system of claim 1, wherein the verification module is further configured to:
and deleting the user data contained in the data queue when the verification result shows that the user data contained in the data queue is received by the receiving module.
5. The system of claim 1, further comprising:
the timing module is used for periodically generating a reading signal;
wherein the read module receives a plurality of user data from different sources in response to the read signal.
6. A cross-domain synchronous distribution method for user data is characterized by comprising the following steps:
receiving a plurality of user data from different sources, and generating a corresponding label according to the content of each user data, wherein each user data carries the corresponding label;
sending the user data to a plurality of data queues according to different sources of the user data and the labels carried by the user data respectively;
receiving user data contained in the data queue and generating a receiving success signal;
verifying whether user data contained in the data queue is received in response to the reception success signal;
and storing the user data contained in the received data queue.
7. The method of claim 6, wherein sending the plurality of user data to a plurality of data queues according to different sources of the plurality of user data and the tags carried by the plurality of user data respectively comprises:
sending the user data containing the same label to the same data queue; and/or
And sending the user data of different sources to different data queues according to the source of each user data.
8. The method of claim 6, wherein receiving the user data contained in the data queue comprises:
and matching the label of the user data contained in the data queue with a preset subscription label, and receiving the user data contained in the data queue after the matching is successful.
9. The method of claim 6, wherein verifying whether user data contained in the data queue is received in response to the reception success signal comprises:
and deleting the user data contained in the data queue when the verification result shows that the user data contained in the data queue is received by the receiving module.
10. The method of claim 6, wherein receiving a plurality of user data from different sources further comprises:
periodically generating a read signal;
a plurality of user data from different sources is received in response to the read signal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911012304.9A CN110795505A (en) | 2019-10-23 | 2019-10-23 | Cross-domain synchronous distribution system and method for user data |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911012304.9A CN110795505A (en) | 2019-10-23 | 2019-10-23 | Cross-domain synchronous distribution system and method for user data |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110795505A true CN110795505A (en) | 2020-02-14 |
Family
ID=69441123
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911012304.9A Pending CN110795505A (en) | 2019-10-23 | 2019-10-23 | Cross-domain synchronous distribution system and method for user data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110795505A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120036200A1 (en) * | 2010-08-09 | 2012-02-09 | Bank Of America Corporation | Social media engagement system |
| CN104679888A (en) * | 2015-03-17 | 2015-06-03 | 浪潮集团有限公司 | Device and method for data synchronization |
| CN106503149A (en) * | 2016-10-21 | 2017-03-15 | 广东亿迅科技有限公司 | A kind of method of data synchronization and its system |
| CN106911790A (en) * | 2010-12-02 | 2017-06-30 | 微软技术许可有限责任公司 | Mixed synchronization and asynchronous flow |
| CN107729366A (en) * | 2017-09-08 | 2018-02-23 | 广东省建设信息中心 | A kind of pervasive multi-source heterogeneous large-scale data synchronization system |
| CN108900434A (en) * | 2018-07-18 | 2018-11-27 | 北京木瓜移动科技股份有限公司 | Data collection distribution method and device |
| CN109788053A (en) * | 2019-01-04 | 2019-05-21 | 深圳壹账通智能科技有限公司 | Method of data synchronization and system |
-
2019
- 2019-10-23 CN CN201911012304.9A patent/CN110795505A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20120036200A1 (en) * | 2010-08-09 | 2012-02-09 | Bank Of America Corporation | Social media engagement system |
| CN106911790A (en) * | 2010-12-02 | 2017-06-30 | 微软技术许可有限责任公司 | Mixed synchronization and asynchronous flow |
| CN104679888A (en) * | 2015-03-17 | 2015-06-03 | 浪潮集团有限公司 | Device and method for data synchronization |
| CN106503149A (en) * | 2016-10-21 | 2017-03-15 | 广东亿迅科技有限公司 | A kind of method of data synchronization and its system |
| CN107729366A (en) * | 2017-09-08 | 2018-02-23 | 广东省建设信息中心 | A kind of pervasive multi-source heterogeneous large-scale data synchronization system |
| CN108900434A (en) * | 2018-07-18 | 2018-11-27 | 北京木瓜移动科技股份有限公司 | Data collection distribution method and device |
| CN109788053A (en) * | 2019-01-04 | 2019-05-21 | 深圳壹账通智能科技有限公司 | Method of data synchronization and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11561984B2 (en) | Systems and methods for generating customized filtered-and-partitioned market-data feeds | |
| CN111030784A (en) | Information synchronization method and device | |
| CN110688828A (en) | File processing method and device, file processing system and computer equipment | |
| CN109361525B (en) | Method, device, control terminal and medium for restarting distributed deployment of multiple services | |
| CN110719221B (en) | Instant messaging method, device, equipment and storage medium | |
| CN106874334A (en) | A kind of data processing method and device, information handling systems utilizing | |
| CN102609479A (en) | Memory database node copying method | |
| CN104657841A (en) | Express item delivery method, delivery processing method, express cabinet terminal and service system | |
| CN110912814A (en) | Interface data distribution method and device | |
| CN112689020A (en) | Message transmission method, message middleware, electronic equipment and storage medium | |
| CN110138738B (en) | Method, device and system for intelligentizing office conference, storage medium and server | |
| CN103716230A (en) | Message sending method, device and server | |
| CN107222392B (en) | Communication method, apparatus, system, and computer storage medium | |
| CN107094085A (en) | A kind of signaling methods and device | |
| JP5961471B2 (en) | Output comparison method in multiple information systems | |
| CN112836201A (en) | Method, apparatus, device and computer-readable medium for multi-platform information exchange | |
| CN115619559A (en) | Consensus transaction visualization method, device, equipment and storage medium | |
| CN113114557B (en) | Message sending method and device, electronic equipment and storage medium | |
| CN111600942B (en) | Report processing method, device and equipment | |
| CN114490128A (en) | Data storage method and device under remote multi-active architecture | |
| CN102984676B (en) | Method and terminal for processing message session | |
| CN110022323A (en) | A kind of method and system of the cross-terminal real-time, interactive based on WebSocket and Redux | |
| CN110795505A (en) | Cross-domain synchronous distribution system and method for user data | |
| CN101505228B (en) | Method, system and apparatus for sending broadcast message in instant communication tool | |
| CN112887925B (en) | SMS push method, edge server node and service server node |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200214 |