CN119046298B - Multi-source data synchronization update system based on spatiotemporal digital base - Google Patents
Multi-source data synchronization update system based on spatiotemporal digital base Download PDFInfo
- Publication number
- CN119046298B CN119046298B CN202411533245.0A CN202411533245A CN119046298B CN 119046298 B CN119046298 B CN 119046298B CN 202411533245 A CN202411533245 A CN 202411533245A CN 119046298 B CN119046298 B CN 119046298B
- Authority
- CN
- China
- Prior art keywords
- data
- time
- space
- source
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000004458 analytical method Methods 0.000 claims abstract description 29
- 238000007781 pre-processing Methods 0.000 claims abstract description 17
- 238000013523 data management Methods 0.000 claims abstract description 13
- 238000007726 management method Methods 0.000 claims abstract description 12
- 238000012545 processing Methods 0.000 claims abstract description 12
- 238000005516 engineering process Methods 0.000 claims abstract description 6
- 230000001360 synchronised effect Effects 0.000 claims description 25
- 238000004364 calculation method Methods 0.000 claims description 11
- 230000001419 dependent effect Effects 0.000 claims description 7
- 230000007246 mechanism Effects 0.000 claims description 7
- 238000012544 monitoring process Methods 0.000 claims description 7
- 238000011156 evaluation Methods 0.000 claims description 6
- 238000013500 data storage Methods 0.000 claims description 4
- 238000010586 diagram Methods 0.000 claims description 4
- 238000010276 construction Methods 0.000 claims description 3
- 238000013079 data visualisation Methods 0.000 claims description 3
- 230000002452 interceptive effect Effects 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 3
- 230000000007 visual effect Effects 0.000 claims description 3
- 238000004146 energy storage Methods 0.000 claims 1
- 238000000034 method Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 230000002123 temporal effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/23—Updating
- G06F16/2379—Updates performed during online database operations; commit processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/215—Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/22—Indexing; Data structures therefor; Storage structures
- G06F16/2228—Indexing structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/26—Visual data mining; Browsing structured data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/284—Relational databases
- G06F16/285—Clustering or classification
- G06F16/287—Visualization; Browsing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/29—Geographical information databases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/904—Browsing; Visualisation therefor
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/906—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/907—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/907—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/909—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Software Systems (AREA)
- Library & Information Science (AREA)
- Remote Sensing (AREA)
- Computing Systems (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
本发明涉及数据同步更新技术领域，具体涉及基于时空数字底座的多源数据同步更新系统，包括数据采集模块、时空数据管理模块、数据预处理模块、数据同步更新模块以及智能分析模块，其中；所述数据采集模块实时采集不同类型的多源数据；所述时空数据管理模块对采集的多源数据进行时空标记和管理；所述数据预处理模块对采集到的多源数据进行预处理；所述数据同步更新模块采用智能算法动态计算各类数据源的更新频率和时序关系；所述智能分析模块对同步更新后的多源数据进行智能分析，提供实时反馈和预警。本发明，提升系统的运行效率和数据处理能力，具有极强的实用性与扩展性。
The present invention relates to the field of data synchronization and updating technology, and specifically to a multi-source data synchronization and updating system based on a spatiotemporal digital base, including a data acquisition module, a spatiotemporal data management module, a data preprocessing module, a data synchronization and updating module, and an intelligent analysis module, wherein: the data acquisition module acquires different types of multi-source data in real time; the spatiotemporal data management module performs spatiotemporal tagging and management on the acquired multi-source data; the data preprocessing module performs preprocessing on the acquired multi-source data; the data synchronization and updating module uses an intelligent algorithm to dynamically calculate the update frequency and timing relationship of various data sources; the intelligent analysis module performs intelligent analysis on the synchronously updated multi-source data, and provides real-time feedback and early warning. The present invention improves the operating efficiency and data processing capability of the system, and has extremely strong practicality and expansibility.
Description
Technical Field
The invention relates to the technical field of data synchronous updating, in particular to a multi-source data synchronous updating system based on a space-time digital base.
Background
Along with the continuous advancement of big data technology and informatization process, more and more application scenes relate to the collection, processing and management of multi-source heterogeneous data, and the data sources possibly comprise a sensor network, internet of things equipment, satellite data, video monitoring and the like, the data types comprise structured data, semi-structured data and unstructured data, so that in order to better utilize the massive and diversified data, space-time information becomes a key dimension for realizing data integration and analysis, and particularly in the complex application fields of smart cities, intelligent transportation, industrial Internet and the like, unified alignment and efficient analysis of the multi-source data can be realized through space-time management, however, how to realize synchronous update and intelligent analysis of the data in the space-time dimension is still an important challenge faced by the current technology.
In the prior art, a decentralized or independent processing mode is generally adopted for managing and updating multi-source data, so that the real-time performance and consistency of the data are poor, in many systems, the space-time marking of the data is not accurate enough, the effective alignment cannot be realized in time and space dimensions, further, the subsequent analysis and decision process is influenced, in addition, because the updating frequency and time sequence relation of each data source are different, the system often lacks an intelligent synchronization mechanism, the updating lag or the data redundancy is caused, and meanwhile, when the large-scale heterogeneous data are processed in the prior art, the prior art generally depends on a fixed updating strategy, lacks dynamic adjustment capability, is difficult to cope with complex application scenes with frequent data changes, and the problems limit the application value of the multi-source data in intelligent analysis.
Aiming at the defects in the prior art, the invention provides a multi-source data synchronous updating system based on a space-time digital base, which can analyze and early warn updated multi-source data in real time, provides stronger support for decision making in complex scenes and improves the efficiency and expansibility of the system.
Disclosure of Invention
The invention provides a multi-source data synchronous updating system based on a space-time digital base.
The multi-source data synchronous updating system based on the space-time digital base comprises a data acquisition module, a space-time data management module, a data preprocessing module, a data synchronous updating module and an intelligent analysis module, wherein the data acquisition module is used for acquiring data of a user;
the data acquisition module acquires different types of multi-source data in real time from a plurality of data sources, wherein the data sources comprise a sensor network, satellite data, internet of things equipment and video monitoring, and the multi-source data comprise structured data, semi-structured data and unstructured data;
The space-time data management module performs space-time marking and management on the collected multi-source data, establishes a unified space-time index system and performs space-time management based on a space-time digital base;
The data preprocessing module is used for preprocessing the acquired multi-source data, and comprises formatting, denoising and deduplication;
The data synchronous updating module realizes synchronous updating of multi-source data based on the space-time digital base, and dynamically calculates updating frequency and time sequence relation of various data sources by adopting an intelligent algorithm;
the intelligent analysis module performs intelligent analysis on the synchronously updated multi-source data, provides real-time feedback and early warning, generates decision suggestions or executes corresponding adjustment operation.
Optionally, the spatiotemporal data management module includes:
constructing a space-time index, namely constructing a multidimensional space-time index through a Geographic Information System (GIS) technology, and supporting the positioning of different data types in time and space;
the space-time data association is used for associating data with different time and different space dimensions, supporting the tracing of historical data and the dynamic generation of space-time paths, and realizing the semantic association and logical relation construction of the space-time data;
And the space-time data storage is used for storing indexed and associated space-time data, supporting access operation and having distributed storage capacity.
Optionally, the distributed storage includes:
the distributed data acquisition nodes are distributed at different geographic positions and used for locally acquiring data and uploading the data to the central data processing center, and each node has the functions of data preprocessing and temporary storage;
Distributed computing nodes, namely, processing large-scale data in a distributed mode on different computing nodes, reasonably distributing computing resources through a task scheduling mechanism, and supporting parallel processing of space-time data;
And the distributed storage nodes are used for providing distributed storage service for the space-time data and supporting an elastic expansion and redundancy backup mechanism.
Optionally, the data synchronization update module includes:
the priority evaluation of space-time data, namely, the data of each data source can be subjected to priority evaluation according to the real-time property, the space-time dependency and the importance of the data source;
Updating frequency calculation, namely dynamically adjusting the updating frequency based on the priority of the data and the current system resource condition;
Timing relationship calculation the timing relationship of data synchronization is managed by a timing diagram, the update timing of each data source being dependent on the updates of the other related data sources.
Optionally, the calculation formula of the spatio-temporal data priority is:
;
Wherein, Is the data sourceIs set according to the priority of (1),Is the data sourceIs used to update the historical update frequency of (a),Is the data sourceIs used to determine the spatial-temporal dependent weights of (1),Is the data sourceIs added to the importance of the weight of (a),Is a weight coefficient.
Optionally, the update frequency calculation is expressed as:
;
Wherein, Is the data sourceIs updated (number of updates per unit time),Is the data sourceIs used for the time interval of (a),For the current utilization of the system resources,AndIs a weight coefficient.
Optionally, the timing relationship is represented by an adjacency matrix, expressed as:
;
The timing update is expressed as: ;
Wherein, Is the data sourceIs used for the next time of updating the data,For all dependent data sourcesIs used for the update time of the (c),Is the data sourceIs a time delay of (a) to (b).
Optionally, the intelligent analysis module includes:
The data early warning is based on the synchronized multi-source data, and the functions of space-time dynamic monitoring and intelligent early warning are supported;
data clustering and classifying, namely classifying data from different data sources according to preset rules;
And (3) data visualization, namely performing visual display on the classification result in the forms of graphs, charts and the like, and supporting the generation of interactive analysis views.
The invention has the beneficial effects that:
According to the invention, data acquisition, management, preprocessing, synchronous updating and intelligent analysis from a plurality of heterogeneous data sources are realized through the multi-module collaborative work, the system realizes the accurate marking and management of the space-time dimension of multi-source data through the space-time data management module, the consistency of the data of different data sources in time and space is ensured, the alignment precision of the data is greatly improved, secondly, the data preprocessing module carries out formatting, denoising and deduplication processing on the acquired data, the quality and the integrity of the data are ensured, the data synchronous updating module can intelligently adjust the updating frequency and the time sequence relation of the data according to the space-time priority and the system resource condition, the real-time performance and the high efficiency of the data synchronization are ensured, and in addition, the intelligent analysis module carries out analysis on the synchronized data to generate a visual analysis result, provides real-time early warning and decision advice, and the application value and the response capability of the system are greatly improved.
The system not only improves the timeliness and the space consistency of data management, but also provides deep insight into multi-source data through intelligent analysis, supports synchronous updating and dynamic analysis of large-scale heterogeneous data, and particularly can remarkably improve the operation efficiency and the data processing capacity of the system in complex scenes with large data quantity, various data types and frequent change.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the drawings which are used in the description of the embodiments or the prior art will be briefly described, it being obvious that the drawings in the description below are only of the invention and that other drawings can be obtained from them without inventive effort for a person skilled in the art.
Fig. 1 is a schematic diagram of a system functional module according to an embodiment of the present invention.
Detailed Description
The invention will now be described in detail with reference to the drawings and to specific embodiments. While the invention has been described herein in detail in order to make the embodiments more detailed, the following embodiments are preferred and can be embodied in other forms as well known to those skilled in the art, and the accompanying drawings are only for the purpose of describing the embodiments more specifically and are not intended to limit the invention to the specific forms disclosed herein.
It should be noted that references in the specification to "one embodiment," "an example embodiment," "some embodiments," etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the relevant art to effect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described.
Generally, the terminology may be understood, at least in part, from the use of context. For example, the term "one or more" as used herein may be used to describe any feature, structure, or characteristic in a singular sense, or may be used to describe a combination of features, structures, or characteristics in a plural sense, depending at least in part on the context. In addition, the term "based on" may be understood as not necessarily intended to convey an exclusive set of factors, but may instead, depending at least in part on the context, allow for other factors that are not necessarily explicitly described.
As shown in fig. 1, a multi-source data synchronous updating system based on a space-time digital base, the system comprises a data acquisition module, a space-time data management module, a data preprocessing module, a data synchronous updating module and an intelligent analysis module, wherein the data acquisition module is used for acquiring data of a user;
the data acquisition module acquires different types of multi-source data in real time from a plurality of data sources, wherein the data sources comprise a sensor network, satellite data, internet of things equipment and video monitoring, and the multi-source data comprises structured data, semi-structured data and unstructured data;
the space-time data management module performs space-time marking and management on the collected multi-source data, establishes a unified space-time index system, enables heterogeneous data of different sources to be accurately aligned in space-time dimension, performs space-time management based on a space-time digital base, and ensures consistency of the data in two dimensions of time and space;
The data preprocessing module performs preprocessing on the acquired multi-source data, including formatting, denoising and de-duplication, so as to ensure the data quality;
The data synchronous updating module realizes synchronous updating of multi-source data based on the space-time digital base, and dynamically calculates updating frequency and time sequence relation of various data sources by adopting an intelligent algorithm;
the intelligent analysis module performs intelligent analysis on the synchronously updated multi-source data, provides real-time feedback and early warning, generates decision suggestions or executes corresponding adjustment operation.
The space-time data management module comprises:
Constructing a multi-dimensional space-time index through a Geographic Information System (GIS) technology, supporting the positioning of different data types in time and space, and being capable of quickly searching a data set in any space-time segment;
the space-time data association is used for associating data with different time and different space dimensions, supporting the tracing of historical data and the dynamic generation of space-time paths, and realizing the semantic association and logical relation construction of the space-time data;
And the space-time data storage is used for storing indexed and associated space-time data, supports access operation, has distributed storage capacity, and ensures the expansibility of large-scale data processing and the storage safety.
The distributed storage includes:
the distributed data acquisition nodes are distributed at different geographic positions and used for locally acquiring data and uploading the data to the central data processing center, and each node has the functions of data preprocessing and temporary storage;
Distributed computing nodes, namely, processing large-scale data in a distributed mode on different computing nodes, reasonably distributing computing resources through a task scheduling mechanism, and supporting parallel processing of space-time data;
And the distributed storage nodes provide distributed storage service for the space-time data, support elastic expansion and redundancy backup mechanisms and ensure the safety and high availability of data storage.
The data synchronization updating module comprises:
the priority evaluation of space-time data, namely, the data of each data source can be subjected to priority evaluation according to the real-time property, the space-time dependency and the importance of the data source;
Updating frequency calculation, namely dynamically adjusting the updating frequency based on the priority of the data and the current system resource condition;
Timing relationship calculation the timing relationship of data synchronization is managed by a timing diagram, the update timing of each data source being dependent on the updates of the other related data sources.
The calculation formula of the space-time data priority is as follows:
;
Wherein, Is the data sourceIs set according to the priority of (1),Is the data sourceIs indicative of how frequently it changes,Is the data sourceRepresenting the temporal and spatial dependence of the data with other data sources,Is the data sourceThe importance weight of the data is set according to the actual application scene, the importance degree of the data is represented,Is a weight coefficient.
The update frequency calculation is expressed as:
;
Wherein, Is the data sourceIs updated (number of updates per unit time),Is the data sourceRepresenting the data sourceIs a time interval from the present time interval,For the current system resource utilization rate, the range is thatAnd, representing the load condition of the system,AndThe weighting coefficient represents the influence degree of the priority and the utilization rate of the system resources on the update frequency.
The timing relationship is represented by an adjacency matrix, expressed as:
;
The timing update is expressed as: ;
Wherein, Is the data sourceIs used for the next time of updating the data,For all dependent data sourcesIs used for the update time of the (c),Is the data sourceRepresents the timing skew allowed during the data synchronization.
The intelligent analysis module comprises:
Based on the synchronized multi-source data, the data early warning system supports space-time dynamic monitoring and intelligent early warning functions and helps a user to identify possible abnormal or emergency situations in advance;
Data clustering and classification, namely classifying data from different data sources according to preset rules, so that the structured management and quick query of large-scale data are facilitated;
and data visualization, namely visually displaying classification results in the forms of graphs, charts and the like, supporting the generation of interactive analysis views, and helping users to intuitively know the space-time distribution and change of data.
The invention is intended to cover any alternatives, modifications, equivalents, and variations that fall within the spirit and scope of the invention. In the following description of preferred embodiments of the invention, specific details are set forth in order to provide a thorough understanding of the invention, and the invention will be fully understood to those skilled in the art without such details. In other instances, well-known methods, procedures, flows, components, circuits, and the like have not been described in detail so as not to unnecessarily obscure aspects of the present invention.
The foregoing is merely a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention, which are intended to be comprehended within the scope of the present invention.
Claims (4)
1. The multi-source data synchronous updating system based on the space-time digital base is characterized by comprising a data acquisition module, a space-time data management module, a data preprocessing module, a data synchronous updating module and an intelligent analysis module, wherein the data acquisition module is used for acquiring data of a user;
the data acquisition module acquires different types of multi-source data in real time from a plurality of data sources, wherein the data sources comprise a sensor network, satellite data, internet of things equipment and video monitoring, and the multi-source data comprise structured data, semi-structured data and unstructured data;
The space-time data management module performs space-time marking and management on the collected multi-source data, a unified space-time index system is established, space-time management is performed based on a space-time digital base, and the space-time data management module comprises:
Constructing a space-time index, namely constructing a multidimensional space-time index through a geographic information system technology, and supporting the positioning of different data types in time and space;
the space-time data association is used for associating data with different time and different space dimensions, supporting the tracing of historical data and the dynamic generation of space-time paths, and realizing the semantic association and logical relation construction of the space-time data;
the space-time data storage is used for storing indexed and associated space-time data, supporting access operation and having distributed energy storage capacity;
the distributed storage includes:
the distributed data acquisition nodes are distributed at different geographic positions and used for locally acquiring data and uploading the data to the central data processing center, and each node has the functions of data preprocessing and temporary storage;
Distributed computing nodes, namely, processing large-scale data in a distributed mode on different computing nodes, reasonably distributing computing resources through a task scheduling mechanism, and supporting parallel processing of space-time data;
The distributed storage node provides distributed storage service for the space-time data and supports an elastic expansion and redundancy backup mechanism;
The data preprocessing module is used for preprocessing the acquired multi-source data, and comprises formatting, denoising and deduplication;
The data synchronous updating module realizes the synchronous updating of multi-source data based on a space-time digital base, adopts an intelligent algorithm to dynamically calculate the updating frequency and time sequence relation of various data sources, and comprises the following components:
the priority evaluation of space-time data, namely, the data of each data source can be subjected to priority evaluation according to the real-time property, the space-time dependency and the importance of the data source;
Updating frequency calculation, namely dynamically adjusting the updating frequency based on the priority of the data and the current system resource condition;
The time sequence relation of the data synchronization is managed through a time sequence diagram, and the updating time sequence of each data source depends on the updating of other related data sources;
The calculation formula of the space-time data priority is as follows:
;
Wherein, Is the data sourceIs set according to the priority of (1),Is the data sourceIs used to update the historical update frequency of (a),Is the data sourceIs used to determine the spatial-temporal dependent weights of (1),Is the data sourceIs added to the importance of the weight of (a),Is a weight coefficient;
the intelligent analysis module performs intelligent analysis on the synchronously updated multi-source data, provides real-time feedback and early warning, generates decision suggestions or executes corresponding adjustment operation.
2. The spatio-temporal digital base based multi-source data synchronization update system of claim 1, characterized in that the update frequency calculation is expressed as:
;
Wherein, Is the data sourceIs used for the update frequency of (a),Is the data sourceIs used for the time interval of (a),For the current utilization of the system resources,AndIs a weight coefficient.
3. The spatio-temporal digital base based multi-source data synchronization update system of claim 2, wherein said timing relationship is represented by an adjacency matrix expressed as:
;
The timing update is expressed as: ;
Wherein, Is the data sourceIs used for the next time of updating the data,For all dependent data sourcesIs used for the update time of the (c),Is the data sourceIs a time delay of (a) to (b).
4. The spatio-temporal digital base-based multi-source data synchronization update system of claim 3, wherein said intelligent analysis module includes:
The data early warning is based on the synchronized multi-source data, and the functions of space-time dynamic monitoring and intelligent early warning are supported;
data clustering and classifying, namely classifying data from different data sources according to preset rules;
and (3) data visualization, namely performing visual display on the classification result in a graph and chart form, and supporting the generation of an interactive analysis view.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411533245.0A CN119046298B (en) | 2024-10-31 | 2024-10-31 | Multi-source data synchronization update system based on spatiotemporal digital base |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411533245.0A CN119046298B (en) | 2024-10-31 | 2024-10-31 | Multi-source data synchronization update system based on spatiotemporal digital base |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN119046298A CN119046298A (en) | 2024-11-29 |
| CN119046298B true CN119046298B (en) | 2025-03-21 |
Family
ID=93580188
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411533245.0A Active CN119046298B (en) | 2024-10-31 | 2024-10-31 | Multi-source data synchronization update system based on spatiotemporal digital base |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119046298B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116244399A (en) * | 2023-03-13 | 2023-06-09 | 中城智慧(北京)城市规划设计研究院有限公司 | Urban space-time information engine method of digital space model DSM |
| CN117493412A (en) * | 2023-10-26 | 2024-02-02 | 北京红山信息科技研究院有限公司 | Digital base operation management system |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115168396A (en) * | 2022-07-15 | 2022-10-11 | 全图通位置网络有限公司 | A data management method and system for a comprehensive intelligent platform based on a space-time system |
| CN116303856B (en) * | 2023-03-07 | 2024-01-09 | 北京龙软科技股份有限公司 | Industrial geographic information system |
| CN117851490A (en) * | 2024-01-09 | 2024-04-09 | 浙江极象科技有限公司 | Data analysis processing system based on big data |
| CN118643188A (en) * | 2024-05-24 | 2024-09-13 | 贵州电网有限责任公司 | A multi-source distributed power grid spatiotemporal information management method and system |
| CN118590503B (en) * | 2024-08-05 | 2024-10-18 | 浙江云针信息科技有限公司 | Multi-source data synchronization processing system and method |
-
2024
- 2024-10-31 CN CN202411533245.0A patent/CN119046298B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116244399A (en) * | 2023-03-13 | 2023-06-09 | 中城智慧(北京)城市规划设计研究院有限公司 | Urban space-time information engine method of digital space model DSM |
| CN117493412A (en) * | 2023-10-26 | 2024-02-02 | 北京红山信息科技研究院有限公司 | Digital base operation management system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN119046298A (en) | 2024-11-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Ding et al. | Enabling smart transportation systems: A parallel spatio-temporal database approach | |
| CN108446293A (en) | A method of based on urban multi-source isomeric data structure city portrait | |
| Difallah et al. | Scalable anomaly detection for smart city infrastructure networks | |
| CN109361577A (en) | A kind of data monitoring method and system based on distributed sensor | |
| CN106126328B (en) | A kind of traffic metadata management method and system based on event category | |
| Ma et al. | Ksq: Top-k similarity query on uncertain trajectories | |
| Acquaviva et al. | Energy signature analysis: Knowledge at your fingertips | |
| CN106777093A (en) | Skyline inquiry systems based on space time series data stream application | |
| Zeng et al. | Data visualization for air quality analysis on bigdata platform | |
| Ding et al. | SeaCloudDM: a database cluster framework for managing and querying massive heterogeneous sensor sampling data | |
| Ding et al. | Massive heterogeneous sensor data management in the Internet of Things | |
| CN109145225B (en) | Data processing method and device | |
| Chao et al. | Efficient trajectory contact query processing | |
| Xu | The analytics and applications on supporting big data framework in wireless surveillance networks | |
| Yin et al. | A novel temporal and spatial panorama stream processing engine on IoT applications | |
| Kim et al. | m Trend: discovery of topic movements on geo-microblogging messages | |
| Amagata et al. | Sliding window top-k dominating query processing over distributed data streams | |
| Zhao et al. | A parameter space framework for online outlier detection over high-volume data streams | |
| CN119941237A (en) | A charging station operation and maintenance method based on multi-dimensional data system | |
| CN104598623A (en) | Comprehensive sensing network stratified data management system | |
| CN119046298B (en) | Multi-source data synchronization update system based on spatiotemporal digital base | |
| Lee et al. | A big data management system for energy consumption prediction models | |
| Yuan et al. | dTexSL: A dynamic disaster textual storyline generating framework | |
| Kwoczek et al. | An architecture to process massive vehicular traffic data | |
| Ghahramani et al. | Analysis of mobile phone data under a cloud computing framework |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |