Disclosure of Invention
The invention provides a convergence digital twin modeling system based on multi-modal data fusion, which comprises a multi-modal data acquisition module, a development analysis module and a history modeling module, wherein the multi-modal data acquisition module is used for acquiring current multi-modal data for convergence and digitization, the current multi-modal data for convergence and digitization at least comprises space vector data, images, laser scanning data, meteorological data, vegetation data and text data, the digital twin modeling module is used for building a convergence entity three-dimensional model according to the current multi-modal data for convergence and digitization, constructing a convergence current digital twin model, the development analysis module is used for determining a plurality of historical development nodes for convergence according to the current multi-modal data for convergence and digitization, and the digital twin modeling module is also used for building a three-dimensional model of the historical development nodes for convergence according to the predicted multiple historical development nodes for convergence and the entity three-dimensional model for convergence, and constructing the historical development node convergence digital twin model.
The digital twin creation module creates a current digital twin model of a colony according to current multi-mode data for performing colony digitization, and includes the steps of fusing the image and laser scanning data of the colony by using a three-dimensional reconstruction algorithm to create the three-dimensional model of the colony, creating information labels of the three-dimensional model of the colony according to text data, creating a plurality of key environments and ecological simulation scenes of the colony according to the meteorological data and vegetation data of the colony, and creating the digital twin model of the colony according to the space vector data, the three-dimensional model of the colony, the information labels of the three-dimensional model of the colony and the key environments and ecological simulation scenes of the colony.
Further, the digital twin establishing module establishes a plurality of key environments and ecological simulation scenes according to the collected weather data and vegetation data, wherein the key environments and the ecological simulation scenes comprise acquisition of a plurality of sample collected weather data, vegetation data and a plurality of key environments and ecological simulation scenes, determination of sample collected weather environment characteristics and vegetation characteristics according to the sample collected weather data and vegetation data, determination of collected weather environment characteristics and vegetation characteristics according to the collected weather data and vegetation data, determination of similar sample collected according to the collected weather environment characteristics and vegetation characteristics and the sample collected weather environment characteristics and vegetation characteristics, and establishment of a plurality of key environments and ecological simulation scenes according to the similar sample collected according to the plurality of key environments and the ecological simulation scenes.
Further, the development analysis module determines a plurality of historical development nodes of the aggregation according to the current situation multi-mode data for digitizing the aggregation, and comprises the steps of backtracking the building time of each building of the aggregation according to the building characteristics and text data of each building of the aggregation, and determining the evolution information of the building of the aggregation according to the building time of each building of the aggregation; and determining the aggregated land type spatial evolution information according to the aggregated space vector data, wherein the aggregated plurality of historical development nodes at least comprise aggregated building evolution information, road network evolution information and land type spatial evolution information.
Further, the digital twin establishing module establishes a three-dimensional model of a landing historical development node according to the predicted landing multiple historical development nodes and the landing entity three-dimensional model, and establishes a landing digital twin model of the historical development node, wherein the digital twin model comprises a plurality of landing historical development nodes determined based on landing building evolution information, road network evolution information and land type space evolution information, and establishes a landing digital twin model of the historical development node based on the landing building evolution information, the road network evolution information, the land type space evolution information and the landing entity three-dimensional model for each historical development node.
The digital twin building module is used for building a three-dimensional model of a focused historical development node based on focused building evolution information, road network evolution information, land type space evolution information and a focused solid three-dimensional model, and comprises the steps of extracting building types, heights and distribution ranges of the historical development node, generating the three-dimensional model of the historical building by combining building geometric data in the solid three-dimensional model, adjusting materials and colors, generating a road network three-dimensional model of the historical development node according to historical road vector data, converting the historical land type space data into texture or elevation change of a three-dimensional terrain surface to generate an agricultural model, fusing the three-dimensional model of the historical building, the road network three-dimensional model and the agricultural model, and building the three-dimensional model of the focused historical development node.
Further, the multi-mode data acquisition module acquires focused laser scanning data, which comprises the steps of determining focused key buildings according to focused images and space vector data, determining optimal laser scanning parameters and optimal scanning paths of the key buildings according to the images of the key buildings for each focused key building, and scanning the key buildings according to the optimal laser scanning parameters and the optimal scanning paths of the key buildings to acquire the laser scanning data of the key buildings.
Further, the multi-mode data acquisition module determines a focused key building according to the focused image and the space vector data, and comprises the steps of determining building characteristics of each focused building according to the focused image, wherein the building characteristics at least comprise color characteristics and outline characteristics, and determining the focused key building based on the building characteristics of each focused building and the focused space vector data.
Further, the multi-mode data acquisition module determines a focused key building based on building features of each focused building and focused space vector data, and comprises determining feature difference values of the building according to the building features of each focused building, determining target buildings according to the feature difference values of each focused building, determining similar sample buildings according to the building features of the target buildings and the building features of the sample buildings, determining historical cultural values of the target buildings according to the historical cultural values of the similar sample buildings, calculating position center values of the target buildings according to the focused space vector data, and determining the focused key building according to the historical cultural values and the position center values of each focused target building.
The invention provides a convergence digital twin modeling method based on multi-mode data fusion, which is applied to the convergence digital twin modeling system based on multi-mode data fusion, and comprises the steps of obtaining current multi-mode data for convergence digitization, wherein the current multi-mode data for convergence digitization at least comprises images, laser scanning data, environment data and text data; the method comprises the steps of establishing a focused entity three-dimensional model according to current multi-mode data for carrying out focused digitization, constructing a focused current digital twin model, determining a plurality of focused historical development nodes according to the current multi-mode data for carrying out focused digitization, establishing a focused historical development node three-dimensional model according to the predicted focused historical development nodes and the focused entity three-dimensional model, and constructing a historical development node focused digital twin model.
Compared with the prior art, the method and the system for modeling the landing digital twin based on the multi-mode data fusion have the following beneficial effects:
1. the current three-dimensional model is related with the historical data through the extraction of the historical development information (such as building evolution, road network change and land type space) to form a dynamic file of a time sequence. The user can visually observe the evolution process of aggregation from the past to the present through the digital file, and understand the formation mechanism of the cultural landscape.
2. By combining the color features and the outline features of the images and the position information of the space vector data, key buildings with historical cultural value or space representativeness can be accurately identified. The indiscriminate scanning of all buildings is avoided, the data redundancy is reduced, and the building with the largest contribution to the cultural landscape digital archives is focused. Based on the similarity of building characteristics of the target building and the sample building, the historical cultural value of the target building is evaluated by combining the historical cultural value of the sample building. Providing scientific basis for screening key buildings and ensuring that the cultural landscape digital file contains the most representative cultural heritage.
3. And matching similar historical scanning buildings for the key buildings through building features of the historical scanning buildings and optimal laser scanning parameters, and referencing the optimal scanning parameters. The blind setting of scanning parameters is avoided, the scanning efficiency and the data quality are improved, and the three-dimensional model precision of the key building is ensured. The optimal laser scanning parameters comprise resolution, sampling rate and scanning interval, and can be finely adjusted according to the building characteristics of the key building. The scanning requirements of different building types (such as ancient buildings and modern buildings) are met, and the detailed representation of the three-dimensional model is ensured. And generating a three-dimensional model based on the image of the key building, determining a key area, and providing visual basis for scanning path planning. The scanning path is ensured to cover the key area, and important details are avoided being omitted. Generating a plurality of scanning paths, and generating an optimal scanning path by combining path evaluation indexes (such as path length, scanning time and coverage) and key areas through a genetic algorithm. The scanning quality is ensured, the scanning time and the labor cost are reduced, and the scanning efficiency is improved.
4. And establishing a solid three-dimensional model through the collected images and laser scanning data, and generating information labels by combining text data, so that the accuracy of the model in terms of spatial morphology and cultural connotation is ensured. And establishing a key environment and an ecological simulation scene (such as seasonal variation and vegetation coverage variation) according to the meteorological data and the vegetation data, and blending the key environment and the ecological simulation scene into the three-dimensional digital twin base. The sense of reality and immersion of the sand table are enhanced, and the interactive relationship between the deagglomeration and the natural environment is intuitively managed by a user. The building time is predicted through multidimensional data such as building characteristics, text data, space vector data and the like, the building evolution, road network evolution and land type space evolution information are determined, the historical development rule of aggregation is systematically revealed, and scientific basis is provided for cultural heritage protection and historical research. For each historical development node, constructing a three-dimensional model of the historical development node based on evolution information and the three-dimensional model of the entity, and combining the environment and the ecological simulation scene to generate a landing digital twin model. The sand table scene of different historical periods is supported to be dynamically switched by the user, the historical evolution process of the aggregation is intuitively perceived, and decision support is provided for cultural heritage protection, urban planning and the like.
Detailed Description
In order to more clearly illustrate the technical solutions of the embodiments of the present specification, the drawings that are required to be used in the description of the embodiments will be briefly described below. It is apparent that the drawings in the following description are only some examples or embodiments of the present specification, and it is possible for those of ordinary skill in the art to apply the present specification to other similar situations according to the drawings without inventive effort. Unless otherwise apparent from the context of the language or otherwise specified, like reference numerals in the figures refer to like structures or operations.
FIG. 1 is a flow diagram of a method of digital twin modeling of a colony based on multimodal data fusion, as shown in FIG. 1, according to some embodiments of the present description, which may include the following steps.
Step 110, current multi-mode data for aggregation and digitization is acquired.
The current multi-mode data for performing aggregation and digitization at least comprises space vector data, images, laser scanning data, meteorological data, vegetation data and text data.
Specifically, the space vector data reflects the aggregated space structure and functional partition by geographic information represented by geometric elements such as points, lines, planes and the like, and may specifically include:
Building data, namely building plane boundary.
Road network, road center line, width, grade (arterial road, branch road), material (asphalt, stone slab).
Water system and greenbelt, river, lake, park, boundary and attribute of protective forest.
Functional areas, namely living areas, business areas and agricultural areas.
The collected images are used for reflecting visual information of collected appearance, building style and space layout, and can specifically comprise:
aerial image, namely, covering the whole aggregate aerial view for analyzing the space pattern.
Ground photography, building elevation and street details, which are used for identifying building styles and materials.
Historical photos, namely file photos reflecting the appearances of different historical development nodes.
The laser scan data may include three-dimensional point cloud data acquired by a laser radar (LiDAR) reflecting the spatial morphology and details of the fall. In particular, the laser scan data may include high-precision building surface data for extracting rooftop types, facade structures.
The weather data is used to reflect the microclimate conditions of the area where the aggregate is located and may include day/month recordings of temperature, precipitation, humidity, wind speed.
The vegetation data is used for reflecting the coverage of the vegetation around the settlement and the land type space, and can specifically comprise farmland, woodland, grassland, distribution and area of water areas, types and proportions of plants, vegetation indexes and the like.
The text data is used for reflecting text information of aggregation history, culture and social structure, and can specifically comprise:
historical literature including territories, genealogy, inscription, contractual literature, ancestral hall temple books.
Oral history, resident interview records, folk legend.
Modern archives, news reports, statistical annual certificates, shi Zhi records of village history stadium, and non-genetic information.
In some embodiments, acquiring clustered laser scan data includes:
Determining a focused key building according to the focused image and the space vector data;
for each key building which is gathered, determining the optimal laser scanning parameters and the optimal scanning paths of the key buildings according to the images of the key buildings, and scanning the key buildings according to the optimal laser scanning parameters and the optimal scanning paths of the key buildings to acquire the laser scanning data of the key buildings.
Specifically, the key building of the colony may be a building having historical, cultural, structural or functional importance in the colony, such as an ancient building, a signage public building, a traditional civilian house, and the like.
In some embodiments, determining a clustered key building from clustered image and space vector data includes:
Determining building features of each building according to the collected images, wherein the building features at least comprise color features and outline features;
For each building which is converged, determining a characteristic difference value of the building according to the building characteristics of each building which is converged;
Determining a target building according to the characteristic difference value of each building;
for each converged target building, determining a similar sample building based on building features of the target building and building features of the sample building, determining a historical cultural value of the target building according to the historical cultural value of the similar sample building, and calculating a position center value of the target building according to converged space vector data;
and determining the focused key building according to the historical cultural value and the position central value of each focused target building.
In particular, architectural features are used to describe attributes of architectural appearance and structure, including color, outline, and the like.
Color characteristics may include the dominant hue (e.g., red, gray) of the exterior wall of the building, the roof, and the color distribution. For each building which is gathered, an image segmentation algorithm can be used for separating an image of a building area, counting the frequency distribution of different colors in the image of the building area, obtaining a feature vector reflecting the color distribution of the building by calculating the histogram of each channel in an HSV color space, counting the frequency of different tone values (expressed by angles and ranging from 0 DEG to 360 DEG) in the histogram of an H (tone) channel, dividing the tone range into a plurality of intervals (for example, 10 DEG or 20 DEG is an interval), and counting the number of pixels in each interval to form a tone histogram vector. For example, if divided into 18 bins (one bin per 20 °), the tone histogram vector contains 18 data points. And counting the occurrence frequency of different saturation values (expressed as a percentage and ranging from 0% to 100%) in the S (saturation) channel histogram, dividing the saturation range into a plurality of intervals (for example, 10% is an interval), and counting the number of pixels in each interval to form a saturation histogram vector. For example, if divided into 10 bins (one bin per 10%), the saturation histogram vector contains 10 data points. The frequency of occurrence of different brightness values (usually expressed in percentage and ranging from 0% to 100%) in the V (brightness) channel histogram is counted, the brightness range is divided into a plurality of intervals (for example, 10% is one interval), and the number of pixels in each interval is counted to form a brightness histogram vector. For example, if divided into 10 bins (one bin for every 10%), the luminance histogram vector contains 10 data points. In order to eliminate the influence of the histogram vector dimensions of different channels, the histogram vector of each channel is normalized so that the sum of the histogram vectors is 1. In this way, each data point in the feature vector represents the relative frequency of the color interval in the image, and the normalized hue, saturation and brightness histogram vectors are sequentially combined to form the color feature vector of the building.
Profile features may include planar shapes of buildings (e.g., rectangular, L-shaped), facade levels (e.g., single-layer, multi-layer), roof forms (e.g., flat tops, sloped tops), etc. For each building which is gathered, an image segmentation algorithm can be used for separating an image of a building area, the image of the building area is based on the image of the building area through a self-adaptive threshold and a double-threshold detection algorithm, the edge of the building is generated, the energy minimization is carried out through a gradient vector flow field, the initially extracted building outline is optimized, the outline is simplified into a polygon, and the plane shape (such as 4 vertexes, a rectangle, 5 vertexes, an L shape and the like) is judged through the number of vertexes. The shape classification result is encoded as a discrete value (e.g., rectangle=1, l-shape=2) as part of the contour feature vector. The number of layers (e.g., each layer is about 3 meters in height) was detected by vertical projection statistics of pixel density variations. The layer number detection result is encoded into discrete values (e.g., single layer=1, two layers=2, three layers=3, etc.), as part of the contour feature vector. Roof-form detection results are encoded as discrete values (e.g., plateau = 1, peak = 2, dome = 3, etc.) as part of the contour feature vector, using semantic segmentation models (e.g., deepLabV +) to segment the roof area, or by geometric analysis (e.g., differences in contour convex hulls from building contours) to locate the roof, calculating the height standard deviation of the roof area (e.g., plateau standard deviation <0.5 meters, peak >1 meter).
For each building which is converged, calculating the cosine similarity of the color feature vector of the building and the color feature vector of any one of other buildings which are converged, calculating the cosine similarity of the contour feature vector of the building and the contour feature vector of any one of the other buildings which are converged, carrying out weighted summation on the cosine similarity of the color feature vector of the building and the color feature vector of the other buildings which are converged and the cosine similarity of the contour feature vector, obtaining the first building similarity of the building and the other buildings which are converged, and solving the variance of the first building similarity of the building and any one of the other buildings which are converged, thereby obtaining the feature difference value of the building. The building with the characteristic difference value larger than the characteristic difference value threshold can be used as a target building, wherein the characteristic difference value threshold can be obtained according to human experience or big data statistical analysis. It will be appreciated that a target building is one that is identified as having significant feature differences in a colony based on analysis of similarity of its color and outline features to other buildings, and that the target building may have a higher value for characterizing the colony.
Sample buildings are building instances of known historic cultural value (e.g., traditional folk houses, historic remains). The historical cultural value is a comprehensive evaluation result of the building in the aspects of history, culture, art, science and the like, and can be obtained by scoring by experts according to the dimensions of protection level, building age, style uniqueness and the like. For each target building which is gathered, referring to the mode, calculating to obtain the second building similarity of the target building and the sample building, taking the sample building with the second building similarity being larger than the first building similarity threshold as a similar sample building, and averaging the historical cultural value of the similar sample building to obtain the historical cultural value of the target building, wherein the first building similarity threshold can be determined through experiments or experience of manual work (such as an expert and the like).
The location center value of the target building may be used to analyze whether the target building is central to the development of the colony (e.g., extending outward centered on the building), or to evaluate its importance in the spatial layout. For example only, the coordinates of each building that is clustered may be averaged to obtain the coordinates of the center of the cluster, and for each target building that is clustered, the distance between the target building and the center of the cluster is calculated according to the coordinates of the target building and the coordinates of the center of the cluster, the position center value of the target building is calculated according to the distance between the target building and the center of the cluster, and the shorter the distance between the target building and the center of the cluster is, the larger the position center value of the target building is.
For each converged target building, weighting and summing the historical cultural value and the position central value of the target building to obtain the comprehensive value of the target building, and taking the target building with the comprehensive value larger than the comprehensive value threshold as a converged key building, wherein the comprehensive value threshold can be obtained according to human experience or big data statistical analysis.
In some embodiments, determining optimal laser scanning parameters for a critical building from an image of the critical building comprises:
acquiring building characteristics and optimal laser scanning parameters of a history scanning building;
Determining similar historical scanning buildings according to the building characteristics of the key buildings and the building characteristics of the historical scanning buildings;
And determining optimal laser scanning parameters of the key building according to the similar optimal laser scanning parameters of the historical scanning building, wherein the optimal laser scanning parameters at least comprise resolution, sampling rate and scanning interval.
Specifically, the history scan building may be a building for which the optimal laser scan parameters have been determined, and the history scan building may not belong to the current convergence. The optimal parameters for the historic scanning of the building may be determined empirically, either experimentally or manually (e.g., expert, etc.).
The optimal laser scanning parameters refer to a set of parameter combinations set for historical scanning buildings in the laser scanning (LiDAR) process, so that the scanning result reaches the optimal balance among precision, efficiency and cost.
Resolution, the minimum distance between adjacent points in the point cloud (e.g., 0.01 meters).
Sampling rate, point cloud density per unit area (e.g., 1000 points/m 2).
The scanning interval refers to the physical interval between two continuous scanning positions of the laser scanner in the moving process. This parameter directly affects the integrity, overlap rate and overall efficiency of the scan data. The scan pitch determines the size of the overlap region between adjacent scan positions. If the distance is too large, data blank may appear between the scanning areas, and the target object cannot be covered completely. If the spacing is too small, scan time and data redundancy are increased, but data integrity is improved. The proper overlapping rate (30% -50% is usually recommended) is helpful for splicing and registering the point cloud data, and registration errors are reduced. Too little overlap rate may lead to registration failure, while too much overlap rate increases computational complexity.
For each key building which is converged, the color feature vector cosine similarity of the color feature vector of the key building and the color feature vector of any one historical scanning building can be calculated, the contour feature vector cosine similarity of the contour feature vector of the key building and the contour feature vector of the historical scanning building is calculated, and the color feature vector cosine similarity of the color feature vector of the key building and the color feature vector cosine similarity of the historical scanning building and the contour feature vector cosine similarity are weighted and summed to obtain the third building similarity of the key building and the historical scanning building. A historic scan building with a third building similarity greater than a second building similarity threshold may be considered a similar historic scan building, where the second building similarity threshold may be empirically determined by experimentation or manually (e.g., expert, etc.).
As an example, the optimal laser scanning parameters of the similar historic scanning building with the highest third similarity can be used as the optimal laser scanning parameters of the key building. Also for example, the optimal laser scanning parameters for similar historic scanning buildings can be averaged as the optimal laser scanning parameters for key buildings.
In some embodiments, determining an optimal scan path for a critical building from an image of the critical building comprises:
Generating a three-dimensional model of the key building according to the image of the key building;
Determining a key area of a three-dimensional model of the key building according to the image of the key building;
generating a plurality of scanning paths of the key buildings according to the three-dimensional model of the key buildings;
determining a plurality of path evaluation indexes and key areas of a three-dimensional model of a key building, and establishing an adaptability function;
and generating an optimal scanning path of the key building based on the fitness function and the scanning paths of the plurality of key buildings through a genetic algorithm.
Specifically, a three-dimensional reconstruction algorithm may be used to generate a three-dimensional model of the critical building from the images of the critical building.
The key areas of the three-dimensional model of the key building may include geometrically complex areas in the key building, functionally important areas, and accessible restricted areas, wherein the geometrically complex areas may include areas requiring high-precision scanning such as engraving of building facades, special structures of roofs, etc., the functionally important areas may include areas critical to building structures or functions such as entrances, windows, structural support points, etc., and the accessible restricted areas may include areas that are blocked or difficult to directly scan (e.g., back, high areas of the building). The geometric complexity region (e.g., sculptures, pinnacles) may be extracted by geometric features such as curvature, normal variations, etc. The three-dimensional model is semantically segmented using a deep learning model (e.g., pointNet) and functionally important areas (e.g., windows, doors) are labeled. The limit of accessibility region is determined by Visibility Graph (Visibility Graph) or ray casting (RAY CASTING).
The scan path refers to a sequence of trajectories in which the laser scanning device moves in three-dimensional space.
FIG. 2 is a schematic flow diagram of generating a plurality of scan paths for a critical building according to some embodiments of the present description, as shown in FIG. 2, from a three-dimensional model of the critical building, which may include the following flows:
s11, uniformly setting a plurality of viewpoints on a three-dimensional model of a key building according to the scanning interval included by the optimal laser scanning parameter, wherein the distance between any two adjacent viewpoints is the scanning interval included by the optimal laser scanning parameter;
S12, setting a path generation constraint set, wherein the path generation constraint set can comprise a total length maximum value of a scanning path, a scanning total time maximum value, a next viewpoint selection constraint and the like, and the next viewpoint selection constraint refers to an adjacent viewpoint of which the next viewpoint is required to be the current viewpoint;
s13, taking the view points in the key area of the three-dimensional model of the key building as a plurality of starting points;
s14, randomly selecting one viewpoint in each key region as a current viewpoint;
s15, randomly selecting one view from adjacent views of the current view as a next view;
s16, taking the next viewpoint as the current viewpoint of the currently generated scanning path, judging whether the currently generated scanning path meets the preset condition, wherein the preset condition is that the total length of the scanning path is equal to the maximum length of the scanning path, the total scanning time is the maximum scanning time or the corresponding non-scanning area of the currently generated scanning path is smaller than the non-scanning area threshold, if not, executing S15, if so, completing the generation of one scanning path, and executing S17;
s17, judging whether the number of scanning paths taking the viewpoint in the key area as a starting point is larger than a number threshold, if so, completing the generation of the scanning paths of the key area, executing S18, and if not, executing S14.
And S18, judging whether the scanning paths of each key area are larger than a quantity threshold value, if so, completing generation of all the scanning paths, and if not, selecting the key area of the next non-generated scanning path, and executing S14.
The plurality of path evaluation indexes may include:
the path length index is that the shorter the path length is, the higher the scanning efficiency is, and the lower the energy consumption is.
Scan time index, the total time required to complete the scan path (including movement time and viewpoint dwell time).
Meaning that the shorter the scan time, the shorter the project period and the lower the cost.
Data coverage index, the proportion of key buildings covered by the scanning path. The higher the coverage, the better the data integrity, and the suitability for subsequent analysis or modeling.
Critical area coverage index, the proportion of the scan path covering the critical area. Ensuring data integrity in high precision or functionally critical areas.
Path smoothness index, i.e. the smoothness of the scanned path (such as the number of sharp turns, the frequency of height changes). The smooth path reduces mechanical wear and scanning errors.
The fitness function is a mathematical model used in genetic algorithms to evaluate path quality and may be a function of a weighted combination of scores of multiple path evaluation metrics.
The fitness value may be calculated for each individual (scan path) in the population according to a fitness function. The higher the fitness value, the better the path quality. Individuals are selected to enter the next generation based on fitness values using roulette selection, tournament selection, etc. High quality paths are preserved and low quality paths are eliminated. Performing crossover operations (e.g., single point crossover, sequential crossover) on selected individuals to generate new individuals. A new path combination is generated by gene recombination, and a solution space is explored. And performing mutation operation (such as randomly replacing the view points and adjusting the view point sequence) on the crossed individuals to introduce randomness. Avoiding sinking into local optimum and increasing population diversity. The genetic algorithm is terminated when the maximum number of iterations is reached, the fitness value converges or a satisfactory solution is found. And returning the individual with the highest fitness value as the optimal scanning path.
And 120, establishing a solid three-dimensional model of aggregation according to the current multi-mode data for aggregation digitization, and constructing a current digital twin model of aggregation.
The method specifically comprises the following steps:
Establishing a solid three-dimensional model of the aggregation as shown in figure 3 according to the aggregated image and the laser scanning data;
Generating information labels of the aggregated entity three-dimensional model according to the text data;
establishing a plurality of converged key environments and ecological simulation scenes according to the converged meteorological data and vegetation data;
and generating a digital twin model of the current aggregation state according to the aggregated space vector data, the solid three-dimensional model, the information label of the solid three-dimensional model and a plurality of key environments and ecological simulation scenes.
Specifically, a three-dimensional reconstruction algorithm (such as Structure from Motion, sfM) or specialized software (such as ContextCapture, agisoft Metashape) is used to fuse the landing image and the laser scanning data to generate a landing solid three-dimensional model, where the landing solid three-dimensional model may at least include a three-dimensional model of a landing building, and may further include a three-dimensional model of an infrastructure (e.g., an electric facility (such as a telegraph pole, a transformer), a communication facility (such as a base station), a water supply and drainage facility (such as a manhole cover, a pipeline), etc.), an artificial ground (e.g., a sculpture, a monument, a fountain, a street lamp, etc.).
Text information is embedded into the aggregated solid three-dimensional model in the form of labels, popups, or layers using labeling tools (e.g., arcGIS, QGIS). The annotation content may include building name, purpose, historical context, protection level, etc.
In some embodiments, building a plurality of key environments and ecological simulation scenarios of the aggregate from the aggregate meteorological data and vegetation data, comprising:
acquiring meteorological data, vegetation data and a plurality of key environments and ecological simulation scenes of a plurality of sample clusters;
Determining weather environmental characteristics and vegetation characteristics of sample aggregation according to the weather data and vegetation data of the sample aggregation;
Determining the characteristic of the converged meteorological environment and the characteristic of vegetation according to the converged meteorological data and the vegetation data;
Determining similar sample settlement according to the collected meteorological environment characteristics and vegetation characteristics and the collected meteorological environment characteristics and vegetation characteristics of the sample;
And establishing a plurality of converged key environments and ecological simulation scenes according to the plurality of similar converged key environments and ecological simulation scenes of the sample.
In particular, the sample aggregate may be an aggregate for which a plurality of critical environments and ecologically simulated scenarios have been determined. The sample aggregation may be a virtual aggregation or a real aggregation. The plurality of key environments and the ecological simulation scene where the samples are gathered can be a plurality of key environments and ecological simulation scenes where natural landscapes are different. The multiple key environments and ecological simulation scenes can simulate the influence of storms, drought, typhoons, high temperature, sunny weather, rainy weather and other weather on the landing environment and the ecological system. The key environment and the ecological simulation scene emphasize that vegetation and weather are important components of natural environment, and a more real and vivid three-dimensional scene can be constructed by simulating the distribution and the change of the vegetation and the weather. The simulation is not only helpful for improving the visual effect of the scene, but also can provide powerful support for biological research, environment planning and the like.
Statistical analysis is performed on the collected weather data and vegetation data, collected weather environment characteristics (such as annual average temperature and precipitation season distribution) and vegetation characteristics (such as vegetation coverage rate, distribution proportion of different types of plants and the like) are extracted, and likewise, the collected weather data and vegetation data of the sample are determined, so that the collected weather environment characteristics and vegetation characteristics of the sample are determined. And calculating the characteristic similarity of the clustered weather environment characteristics and vegetation characteristics and the weather environment characteristics and vegetation characteristics clustered by using a similarity measurement method (such as Euclidean distance and cosine similarity), and taking the clustered samples with the characteristic similarity larger than a characteristic similarity threshold as similar sample clustered, wherein the characteristic similarity threshold can be determined through experiments or experience of manpower (such as expert and the like).
The multiple key environments and the ecological simulation scenes with the largest feature similarity and similar samples can be used as the multiple key environments and the ecological simulation scenes.
Assuming that the current fall is a plain fall in subtropical areas, the main vegetation is farmlands and thin forests. The sample is collected as a mountain land in a similar climate zone, and a plurality of key environments and ecological simulation scenes including storm flood simulation, vegetation growth simulation and soil erosion simulation are determined. Similar sample colonies were selected. And migrating a plurality of key environments similar to the sample aggregation and a storm flood simulation scene included by the ecological simulation scene to the current aggregation, and reducing the precipitation intensity by 20% because the current aggregation precipitation is less. And adjusting the growth rate parameters in the vegetation growth model to adapt to the current converged farmland and sparse woodland. Hydrologic simulation was performed using a SWAT (hydrologic simulation using a SWAT model) model, with model parameters adjusted according to current landing topography and soil characteristics. Generating a current converged storm flood simulation scene.
And integrating the aggregated space vector data (such as roads, water systems and building outlines) with the solid three-dimensional model, the information label and the environment and ecological simulation scene. Data fusion is performed using a Geographic Information System (GIS) or a three-dimensional visualization platform (e.g., unity, unreal Engine). And constructing a landing digital twin model in the three-dimensional visualization platform, and displaying the landing space layout, building morphology, environmental characteristics and ecological simulation. User interaction operations such as zooming, rotating, inquiring information labeling, switching simulation scenes and the like are supported.
Step 130, determining a plurality of historical development nodes for aggregation according to the current multi-modal data for aggregation digitization.
The method specifically comprises the following steps:
According to building characteristics and text data of each building, backtracking the building time of each building, and determining evolution information of the building according to the building time of each building;
determining evolution information of the converged road network according to the converged space vector data;
and determining the aggregated land type space evolution information according to the aggregated space vector data, wherein the aggregated plurality of historical development nodes at least comprise aggregated building evolution information, road network evolution information and land type space evolution information.
Specifically, building characteristic data are converted into digital variables (such as styles are coded into digital labels), text data related to the building are extracted through a natural language processing technology, and a deep learning model is used for building time prediction. When the deep learning model is trained, building characteristics and text data are taken as input, and known build time is taken as output. And verifying the accuracy of the deep learning model by cross verification or a leave-out method, and adjusting parameters of the deep learning model to improve the prediction accuracy. For each building, determining the most probable building time range according to the prediction result of the deep learning model. Determining the aggregated building evolution information according to the building time and the space position of each aggregated building, wherein the aggregated building evolution information can comprise main characteristics and distribution conditions of different historical development node buildings.
The collected paper map or electronic map with different historic periods can be collected, digital processing (such as scanning and vectorization) is carried out on the paper map, information such as roads and the like is extracted, the extracted road data are arranged according to time sequence, a time sequence data set is constructed, a GIS tool (such as ArcGIS Pro and QGIS) or a WebGIS platform (such as Leaflet, mapbox) is used for manufacturing a dynamic map, and collected road network evolution information, namely the change of a road network along with time, is displayed. Example 1950s of the road is red, 1980s of the new road is blue, and 2000s of the reroute is green.
Land utilization classification data (such as cultivated land, woodland and construction land) of historical periods (such as 1980s, 2000s and 2020 s) are obtained, sources comprise a historical map, remote sensing image interpretation and the like, the total area and various types of areas (such as paddy fields and dry lands) of agricultural lands of different historical development nodes are calculated, and the distribution proportion of the agricultural lands in different slopes and elevation intervals is counted, so that the space evolution information of the type of the landing land is determined.
And 140, building a three-dimensional model of the aggregated historical development nodes according to the plurality of predicted aggregated historical development nodes and the aggregated solid three-dimensional model, and building a digital twin model of the aggregated historical development nodes.
The method specifically comprises the following steps:
determining a plurality of historical development nodes of the aggregation based on the building evolution information, the road network evolution information and the land type space evolution information of the aggregation;
For each historical development node, a three-dimensional model of the historical development node is built based on the aggregated building evolution information, the road network evolution information, the land type space evolution information and the aggregated solid three-dimensional model, and a digital twin model of the historical development node is built based on a plurality of aggregated key environments and ecological simulation scenes.
Specifically, a representative point in time may be selected as a historic development node based on building evolution information (e.g., building style, functional change), road network evolution information (e.g., road widening, new construction), and land type space evolution information (e.g., tilling expansion, returning to tilling). For example, node 1:1950 (building with greenhouses as main, road with soil road, and high agricultural land occupation ratio). Node 2:1980 (part of the building changed to brick-tile houses, the road started to harden, and the agricultural land changed to industrial land). Node 3:2020 (modern landing, high-rise building increase, perfect road network, centralized agricultural land).
And extracting the building type, height and distribution range of each historical development node, and generating a historical building three-dimensional model by combining building geometric data in the entity three-dimensional model. And generating three-dimensional models (such as soil roads, gravel roads and asphalt roads) of the road network of different historical development nodes according to the historical road vector data. Historical land type space data (e.g., cultivated land, woodland, water) is converted into texture or elevation changes of the three-dimensional terrain surface. Three-dimensional models of historical buildings are generated by using 3D modeling software (such as SketchUp, blender) or GIS tools (such as ARCGIS CITYENGINE), and materials and colors are adjusted to reflect historical characteristics. The geometrical shape and texture of the three-dimensional model of the road are adjusted according to the road type (such as width and material quality), and the distribution of the historical agricultural land is simulated by modifying the terrain elevation or overlaying vegetation texture. And fusing the building, road and agricultural land models with the aggregated solid three-dimensional models (such as terrain and water systems) to form a complete three-dimensional model of the historical development node. And integrating the three-dimensional model of the historical development node with the key environment and the ecological simulation scene by using a Virtual Reality (VR) development platform (such as Unity and Unreal Engine) or a three-dimensional GIS platform (such as Cesium and ARCGIS EARTH) to generate a digital twin model of the historical development node.
FIG. 4 is a schematic flow diagram of a multi-modal data fusion-based digital twinning modeling system, as shown in FIG. 4, according to some embodiments of the present disclosure, which may include a multi-modal data acquisition module, a digital twinning creation module, and a development analysis module.
The system comprises a multi-mode data acquisition module, a storage module and a storage module, wherein the multi-mode data acquisition module is used for acquiring current multi-mode data for performing aggregation and digitization, and the current multi-mode data for performing aggregation and digitization at least comprises space vector data, images, laser scanning data, meteorological data, vegetation data and text data;
The digital twin establishing module is used for establishing a solid three-dimensional model of aggregation according to current multi-mode data for carrying out aggregation digitization and establishing an aggregation current digital twin model;
the development analysis module is used for determining a plurality of historical development nodes for aggregation according to current multi-mode data for aggregation digitization;
the digital twin establishing module is also used for establishing a three-dimensional model of the landing historical development nodes according to the predicted landing multiple historical development nodes and the landing entity three-dimensional model, and establishing a landing digital twin model of the historical development nodes.
The system for modeling the digital twin of the convergence based on the multi-modal data fusion can be used for executing the method for modeling the digital twin of the convergence based on the multi-modal data fusion, and will not be repeated here.
Finally, it should be understood that the embodiments described in this specification are merely illustrative of the principles of the embodiments of this specification. Other variations are possible within the scope of this description. Thus, by way of example, and not limitation, alternative configurations of embodiments of the present specification may be considered as consistent with the teachings of the present specification. Accordingly, the embodiments of the present specification are not limited to only the embodiments explicitly described and depicted in the present specification.