The method and device of augmented reality is carried out for plane map
Technical field
This application involves cartographic information display field, in particular to a kind of method for carrying out augmented reality for plane map
And device.
Background technique
With the fast development of augmented reality research and deepening continuously for map and geographic information application, increasing
While strong reality becomes research hotspot, people are just attempting display enhancing and ground that augmented reality is applied to plane map
Manage information visualization.However, applying at present for the augmented reality of plane map, there are still many problems, for example, for increasing
The virtual three-dimensional model of strong reality is relatively simple, lacks corresponding semantic description, does not merge virtual three-dimensional model effectively
To real scene.
Therefore, there is an urgent need to a kind of methods and dress that augmented reality is carried out for plane map for being able to solve the above problem
It sets to improve the validity and identification of each map constitution element expression in plane map.
Summary of the invention
The purpose of the present invention is to provide a kind of method and devices that augmented reality is carried out for plane map.
According to an aspect of of the present present invention, a kind of method that wind carries out augmented reality for plane map, the method packet are provided
It includes: video image of the captured in real-time about the plane map;It is identified from the video image of shooting and the plane map
The associated default label of interior map constitution element;Based on the default label identified, obtain in the plane map
The corresponding three dimensional virtual models of map constitution element and semantic description information;Member will be constituted with the map in the plane map
The corresponding three dimensional virtual models of element and semantic description information trace are registered in the video image of shooting, in the video of shooting
Virtual three dimensional space map of the building about the plane map in image.
Preferably, the default label is existing with enhancing corresponding with the map constitution element in the plane map
The two dimensional code of real information.
Preferably, the coding region of the two dimensional code includes the first coding region and second code area domain, wherein described the
One coding region is used to record the coding of augmented reality information corresponding with the map constitution element in the plane map, institute
Second code area domain is stated for recording the volume of semantic description information corresponding with the map constitution element in the plane map
Code.
Preferably, it is identified in the video image from shooting related to the map constitution element in the plane map
The step of default label of connection includes: to be decoded to the first coding region of the two dimensional code, to obtain and the plane earth
The corresponding augmented reality information of map constitution element in figure, and the augmented reality information based on acquisition, from preset three
Three dimensional virtual models corresponding with the map constitution element in the plane map, institute are extracted in dimension dummy model database
Stating three dimensional virtual models database purchase has each augmented reality information and three-dimensional corresponding with each augmented reality information
Dummy model;The second code area domain of the two dimensional code is decoded, is constituted with obtaining with the map in the plane map
The corresponding semantic description information of element.
According to another aspect of the present invention, a kind of device that augmented reality is carried out for plane map, described device packet are provided
It includes: video capture unit, video image of the captured in real-time about the plane map;Tag recognition unit, from the video of shooting
Default label associated with the map constitution element in the plane map is identified in image;Model acquiring unit, is based on
The default label identified obtains three dimensional virtual models corresponding with the map constitution element in the plane map and semanteme
Description information;Augmented reality unit, by three dimensional virtual models corresponding with the map constitution element in the plane map and
Semantic description information trace is registered in the video image of shooting, to construct in the video image of shooting about the plane earth
The virtual three dimensional space map of figure.
Preferably, the default label is existing with enhancing corresponding with the map constitution element in the plane map
The two dimensional code of real information.
Preferably, the coding region of the two dimensional code includes the first coding region and second code area domain, wherein described the
One coding region is used to record the coding of augmented reality information corresponding with the map constitution element in the plane map, institute
Second code area domain is stated for recording the volume of semantic description information corresponding with the map constitution element in the plane map
Code.
Preferably, the model acquiring unit includes: model extraction subelement, to the first coding region of the two dimensional code
It is decoded, to obtain augmented reality information corresponding with the map constitution element in the plane map, and is based on obtaining
The augmented reality information taken is extracted from preset three dimensional virtual models database and is constituted with the map in the plane map
The corresponding three dimensional virtual models of element, the three dimensional virtual models database purchase have each augmented reality information and with it is each
The corresponding three dimensional virtual models of a augmented reality information;Semanteme obtains subelement, to the second code area domain of the two dimensional code
It is decoded, to obtain semantic description information corresponding with the map constitution element in the plane map.
According to another aspect of the present invention, a kind of computer readable storage medium being stored with computer program is provided, institute is worked as
Computer program is stated when being executed by processor, realizes the method for carrying out augmented reality for plane map as elucidated before.
According to another aspect of the present invention, a kind of computer equipment is provided, the computer equipment includes: processor;Storage
Device is stored with computer program, when the computer program is executed by processor, realizes and is directed to plane earth as elucidated before
The method that figure carries out augmented reality.
The method and device provided by the present invention for carrying out augmented reality for plane map can effectively improve plane
The validity and identification of each map constitution element expression in map.
Detailed description of the invention
By the description carried out with reference to the accompanying drawing, the purpose of the present invention and feature be will become apparent, in which:
Fig. 1 is the stream for showing the method that augmented reality is carried out for plane map of an exemplary embodiment of the present invention
Cheng Tu;
Fig. 2 is the knot for showing the device that augmented reality is carried out for plane map of an exemplary embodiment of the present invention
Structure block diagram;
Fig. 3 be show an exemplary embodiment of the present invention have it is opposite with the map constitution element in plane map
The schematic diagram of the two dimensional code (QR code) of augmented reality (AR) information answered.
Specific embodiment
The present general inventive concept is, using including augmented reality information relevant to map constitution element and semanteme
The label of description information constructs virtual three dimensional space map to be directed to plane map, so as to improve map structure each in plane map
The validity and identification expressed at element.
Hereinafter, with reference to the accompanying drawings to the embodiment that the present invention will be described in detail.
Fig. 1 is the stream for showing the method that augmented reality is carried out for plane map of an exemplary embodiment of the present invention
Cheng Tu.
In step 110, the video figure by video capture unit (such as camera) captured in real-time about plane map
Picture.
Here, as an example, plane map can be traditional plane paper map, in addition to this, it is also possible to other
The map of some any plane forms that can be taken.
Next, in the step 120, being identified from the video image of shooting and the map constitution element in plane map
Associated default label.
In general, may include various map constitution elements in plane map, as an example, map constitution element here can
To be the landmarks entity such as park, mansion, restaurant, supermarket, school.
In an alternative embodiment, it may be selected that there is enhancing corresponding with the map constitution element in plane map
The two dimensional code of real information is as default label associated with the map constitution element in plane map.In general, two dimensional code can
Including two parts of functional graphic and coding region, however, in this embodiment, as shown in figure 3, the code area of two dimensional code 300
Domain may include the first coding region 301 and second code area domain 302, wherein the first coding region 301 can be used for recording and plane
The coding of the corresponding augmented reality information of map constitution element in map, second code area domain 302 can be used for recording and putting down
The coding of the corresponding semantic description information of map constitution element in the map of face.Come below in conjunction with Fig. 3 in the embodiment
Two dimensional code 300 introduce in more detail.
Fig. 3 be show an exemplary embodiment of the present invention have it is opposite with the map constitution element in plane map
The schematic diagram of the two dimensional code 300 for the augmented reality information answered.
In the example depicted in fig. 3, the coding region of two dimensional code 300 may include two regions, one be for record with
First coding region 301 of the coding of the corresponding augmented reality information of map constitution element in plane map, the other is
For recording the second code area domain of the coding of semantic description information corresponding with the map constitution element in plane map
302.When it is implemented, the first coding region 301 shown in Fig. 3 can be divided into 7 × 7 grid, wherein black lattice indicates
0, white grid indicates 1, and augmented reality information corresponding with the map constitution element in plane map can be with binary
Data mode is coded in the grid inside the first coding region 301.It similarly, can be by second code area domain shown in Fig. 3
302 are divided into several grids, wherein black lattice indicates 0, and white grid indicates the map constitution element in 1, with plane map
Corresponding semantic description information can also be coded in the grid inside second code area domain 302 with binary data mode
In.
Here, augmented reality information corresponding with the map constitution element in plane map may include in plane map
Mark, title, type, pattern and the spatial position coordinate of the corresponding three dimensional virtual models of map constitution element etc..These
Information can be used to the drawing three-dimensional dummy model in the video image of shooting.
Correspondingly, in using embodiment of the two dimensional code 300 as default label, step 120 is directed to the knowledge of default label
Not may include following specific steps:
The video image of shooting is converted to gray level image from color image by step 1201, then from greyscale image transitions
For black white image;
Step 1202 carries out binaryzation to the video image of conversion using Gaussian smoothing filter and mean filter denoising
The methods of processing, then expanded using closed operation, burn into, to obtain the video image with square boundary;
Step 1203 carries out gradient correction (including angle correct and distortion correction) to the video image of processing, so as to
Position tracking and figure detection are carried out to video image;
The video image that step 1204, segmentation correct, and be rectangle data by each image recording after segmentation;
Step 1205 carries out outline to the rectangle data of record, to identify the figure with 300 feature of two dimensional code
Picture.
Next, in step 130, based on the default label identified, obtaining and constituting member with the map in plane map
The corresponding three dimensional virtual models of element and semantic description information.
In using embodiment of the two dimensional code 300 as default label, the parsing that step 130 is directed to default label may include
Following specific steps:
Step 1301 is decoded the first coding region 301 of the two dimensional code 300 identified, with acquisition and plane earth
The corresponding augmented reality information of map constitution element in figure, and the augmented reality information based on acquisition, from preset three
Three dimensional virtual models corresponding with the map constitution element in plane map, the three-dimensional are extracted in dimension dummy model database
Dummy model database purchase has each augmented reality information and three-dimensional mould corresponding with each augmented reality information
Type;
Step 1302 is decoded the second code area domain 302 of the two dimensional code 300 identified, with acquisition and plane earth
The corresponding semantic description information of map constitution element in figure.
Next, in step 140, by three dimensional virtual models corresponding with the map constitution element in plane map and
Semantic description information trace is registered in the video image of shooting, to construct in the video image of shooting about plane map
Virtual three dimensional space map.
When it is implemented, by three dimensional virtual models corresponding with the map constitution element in plane map and view can be passed through
The video image of frequency shooting unit captured in real-time carries out being registrated/being aligned (also referred to as three-dimensional Tracing Registration).In general, for three-dimensional
The three-dimensional Tracing Registration of dummy model relates generally to actual spatial coordinates system, Virtual Space coordinate system, eye coordinate and image
Four class of plane coordinate system, the purpose for carrying out three-dimensional Tracing Registration is in order to which these four types of coordinate systems are come together, to realize virtually
Object is merged with the seamless of real scene.Three-dimensional Tracing Registration based on computer vision be by a given width or several
Video image determines relative position and direction between each target in video camera and real world.
In one example, affine transformation can be used to realize three-dimensional Tracing Registration.Assuming that actual spatial coordinates system be [x,
Y, z, 1], Virtual Space coordinate system is [xv, yv, zv, 1], video camera space coordinates be [x ', y ', z ', 1], real space is sat
There is the coordinate transformation relation as shown in following formula (1) between mark system and Virtual Space coordinate system:
Exist as shown in following formula (2) between video camera space coordinates and actual spatial coordinates system and Virtual Space coordinate system
Coordinate transformation relation:
Coordinate conversion matrix U between actual spatial coordinates system and Virtual Space coordinate system4×4It is known, and images
Coordinate conversion matrix V between machine space coordinates and Virtual Space coordinate system4×4It can be solved by following formula (3):
Wherein, R3×3For spin matrix, T3×1For translation matrix, and spin matrix R3×3With translation matrix T3×1In it is each
A composition parameter can be obtained by video camera space coordinates relative to the position, direction and Attitude Calculation of three dimensional virtual models.Cause
This, formula (3) can be further broken into following formula (4):
Further, following formula (5) can be exported by above formula (2), (3), (4), with this determine video camera space coordinates with
Coordinate conversion between actual spatial coordinates system:
Similarly, it may be determined that the coordinate conversion between video camera space coordinates and Virtual Space coordinate system.
Therefore, the three-dimensional Tracing Registration in step 140 can be realized by the formula in above-mentioned affine transformation, thus clapping
The virtual three dimensional space map about plane map is constructed in the video image taken the photograph, and realizes three dimensional virtual models and real scene
Fusion (that is, i.e. by three dimensional virtual models and the direct Overlapping display of information in real scene).
In addition, during three dimensional virtual models are fused to real scene, it is also contemplated that three dimensional virtual models and true
The hiding relation of real field scape, the display factors such as reasonability and illumination consistency, with this set position that three dimensional virtual models show,
Angle and size, to reach Enhanced expressing effect.
Fig. 2 is the knot for showing the device that augmented reality is carried out for plane map of an exemplary embodiment of the present invention
Structure block diagram.
Referring to Fig. 2, device shown in Fig. 2 may include video capture unit 210, tag recognition unit 220, model acquisition list
Member 230 and augmented reality unit 240.Specifically, video capture unit 210 (such as camera) can be used for captured in real-time about
The video image of plane map;Tag recognition unit 220 can be used for identifying from the video image of shooting and plane map
The associated default label of interior map constitution element;Model acquiring unit 230 can be used for based on the default label identified,
Obtain three dimensional virtual models corresponding with the map constitution element in plane map and semantic description information;Augmented reality unit
240 can be used for three dimensional virtual models corresponding with the map constitution element in plane map and semantic description information trace
It is registered in the video image of shooting, to construct the virtual three dimensional space about plane map in the video image of shooting
Figure.
As previously mentioned, may be selected to have corresponding with the map constitution element in plane map in device shown in Fig. 2
Augmented reality information two dimensional code 300 as default label associated with the map constitution element in plane map.
As previously mentioned, the coding region of this two dimensional code 300 may include the first coding region in device shown in Fig. 2
301 and second code area domain 302, wherein the first coding region 301 is for recording and the map constitution element phase in plane map
The coding of corresponding augmented reality information, second code area domain 302 is for recording and the map constitution element phase in plane map
The coding of corresponding semantic description information.
As previously mentioned, model acquiring unit 230 may include model extraction subelement and semanteme in device shown in Fig. 2
It obtains subelement (having been not shown).Specifically, model extraction subelement can be used for the first code area to above-mentioned two dimensional code 300
Domain 301 is decoded, and to obtain augmented reality information corresponding with the map constitution element in plane map, and is based on obtaining
The augmented reality information taken extracts and the map constitution element in plane map from preset three dimensional virtual models database
Corresponding three dimensional virtual models, the three dimensional virtual models database purchase have each augmented reality information and with each enhancing
The corresponding three dimensional virtual models of real information;Semanteme, which obtains subelement, can be used for the second code area to above-mentioned two dimensional code 300
Domain 302 is decoded, to obtain semantic description information corresponding with the map constitution element in plane map.
By using above-mentioned implementation process, not only augmented reality can be applied to urban map Enhanced expressing,
And the validity and identification of each map constitution element expression of plane map can be improved.In addition, this utilization include with
The label of the relevant augmented reality information of map constitution element and semantic description information is empty for plane map building three-dimensional
Between the mode of map not only calculating speed is fast, but also can effectively meet the requirement of real-time of augmented reality.
In addition it is also necessary to explanation, although embodiment described here is mainly with two with enhancing identification information
The implementation process for plane map three-dimensional virtual object being fused in real scene is described for dimension code, it should be understood that
Be, embodiment described here the default label (for example, bar code etc.) of other forms can also be used realize it is above-mentioned will be three-dimensional
Dummy object is fused to the implementation process of the plane map in real scene.
An exemplary embodiment of the present invention also provides a kind of computer-readable storage medium for being stored with computer program
Matter.The computer-readable recording medium storage has makes processor execute determining wind according to the present invention when being executed by a processor
The computer program of the method for speed.The computer readable recording medium can be stored by any of the data of computer system reading
Data storage device.The example of computer readable recording medium include: read-only memory, random access memory, CD-ROM,
Tape, floppy disk, optical data storage devices and carrier wave (such as being transmitted through wired or wireless transmission path by the data of internet).
An exemplary embodiment of the present invention also provides a kind of computer equipment.The computer equipment include processor and
Memory.Memory is for storing computer program.The computer program is executed by processor so that processor executes basis
The computer program of the method for determination wind speed of the invention.
Although show and describing the application with reference to preferred embodiment, it will be understood by those skilled in the art that not
In the case where being detached from the spirit and scope that is defined by the claims, these embodiments can be carry out various modifications and
Transformation.