CN103968829B - Three-dimensional fix method for tracing based on virtual signage thing and system - Google Patents

Abstract

The invention discloses a three-dimensional space positioning and tracking method and system based on virtual markers, wherein the method includes: selecting a naked-eye three-dimensional display device, and fixing the projection component on the surface of the tracked object or on a positioning device, wherein the positioning The end of the device tracks the tracked object; designing the virtual marker and its corresponding naked-eye three-dimensional display image source; inputting the naked-eye three-dimensional display image source to the projection component to display the corresponding virtual marker in space, wherein the virtual marker The relative position of the projected part remains unchanged; the multi-view video of the scene to be shot is obtained, and the multi-view video includes the actual environment space and virtual markers; according to the multi-view video, the positioning information and motion information of the tracked object are processed . The present invention does not need to set real landmarks in space, but only needs to display virtual images of landmarks in space, and has the advantages of small space occupation, high operational flexibility and high positioning accuracy.

Description

Three-dimensional space positioning and tracking method and system based on virtual markers

technical field

The invention relates to space positioning technology, in particular to a virtual marker-based three-dimensional space positioning and tracking method and system.

Background technique

Traditional spatial positioning technologies are mainly near-infrared optical positioning and magnetic positioning. Taking POLARIS, a product of Northern Digital Inc (NDI), as an example, it mainly uses two cameras with near-infrared filters to shoot equipment equipped with marked reflective balls based on the principle of binocular vision, and obtains imaging images. The processing is performed to obtain the spatial position of the instrument. Ambient light can be suppressed after passing through the near-infrared filter, so there is basically no environmental information on the image, and only a clear image of the marked reflective ball can be obtained. The electromagnetic positioning system mainly includes three parts: the host, the transmitter and the receiver. Taking NDI’s product AURORA as an example, the transmitter emits a magnetic field into the space, and the receiver transmits the signal of the location to the host, and the host calculates and processes to get the receiver. Relative to the position and orientation of the emitter, the spatial positioning of the instrument is achieved. These two main spatial positioning technology products ignore the surrounding environmental factors and only collect the location information of the landmarks. But in some cases, in addition to obtaining the spatial position of the instrument, it is also hoped to obtain the environment information during the operation.

In order to obtain the intraoperative environmental information and the spatial position of the markers, companies such as Claron Technology Inc. have developed a new optical positioning system. This optical positioning technology is based on the principle of binocular vision. In the scene where the equipment is located, the corner points of the two checkerboards are extracted for spatial positioning of the equipment. Compared with the near-infrared optical positioning technology, this technology can obtain the environmental information except the spatial position of the instrument, and can better assist surgical navigation after combining the environmental information and the spatial position information of the instrument.

In order to detect the posture transformation of the entire surgical instrument in the current near-infrared optical positioning and visible light optical positioning systems, it is necessary to install three or more marked reflective balls or checkerboard markers that can ensure the degree of freedom in space. Therefore, the positioning device currently used needs to consider ensuring the position and degree of freedom of the landmarks during design and installation, so it is relatively complicated and has a large space volume, which will affect the surgical operation to a certain extent. Especially in places where the space is narrow, it is difficult to use larger volume positioning instruments. At the same time, the limited markers used for localization accuracy will be limited.

Contents of the invention

The present invention aims at solving one of the above technical problems at least to a certain extent or at least providing a useful commercial choice. Therefore, the object of the present invention is to propose a virtual marker-based three-dimensional space positioning and tracking method and system with narrow required space and high operational flexibility.

In order to achieve the above object, the method for positioning and tracking in three-dimensional space based on virtual markers according to an embodiment of the present invention includes the following steps: S1. Select a naked-eye three-dimensional display device, and fix the projection component in the naked-eye three-dimensional display device on the Positioning the surface of the tracking object or fixing it on the positioning device, wherein the end of the positioning device tracks the positioned tracking object; S2. Designing a virtual marker and a naked-eye three-dimensional display image source corresponding to the virtual marker; S3. Inputting the naked-eye three-dimensional display image source to the projection component to display the corresponding virtual marker in space, wherein the relative position of the virtual marker and the projection component remains unchanged; S4. The multi-view video of the shooting scene, the multi-view video includes the actual environment space and the virtual marker; S5. According to the multi-view video, process and obtain the positioning information and motion information of the positioned tracking object.

According to the three-dimensional space positioning and tracking method based on virtual markers in the above embodiment, compared with the prior art that needs to place at least three markers that are not completely collinear in real space, this method does not need to set real markers in space, Instead, it is only necessary to display virtual markers in space by means of projection components in the naked-eye three-dimensional display device. The three-dimensional space positioning and tracking method based on virtual markers of the present invention has the advantages of small space occupation, high operational flexibility, and high positioning accuracy.

The three-dimensional space positioning and tracking method based on virtual markers in the embodiment of the present invention may also have the following technical features:

In one embodiment of the present invention, the step S2 includes: S21. Designing the number, shape, and size of the virtual markers and the spatial positional relationship between the virtual markers and the projection component; S22. Designing the virtual The naked-eye three-dimensional display image source corresponding to the marker, wherein the algorithm for designing the naked-eye three-dimensional display image source is related to the type of the naked-eye three-dimensional display device.

In one embodiment of the present invention, when the surface of the positioned tracking object allows fixing a projection component, the number of the projection component is one, and the projection component is fixed on the surface of the positioned tracking object, and the virtual mark The number of objects is one.

In one embodiment of the present invention, when the surface of the tracked object does not allow fixing the projection components, the number of the projection components is at least three, and the at least three projection components are respectively fixed on the same positioning device , and ensure that the corresponding at least three virtual markers are not all collinear.

In one embodiment of the present invention, when the naked-eye three-dimensional display device is based on stereoscopic hologram technology, the projection component is a lens array; when the naked-eye three-dimensional display device is based on light field technology, the projection component is a lens array ; When the naked-eye three-dimensional display device is based on holographic technology, the projection component is an interference device.

In one embodiment of the present invention, the step S5 includes: S51. Extracting the feature points of the virtual marker; S52. Matching the feature points in different views of the multi-view video at a certain moment to obtain Match feature point pairs; S53. Perform three-dimensional space reconstruction according to the matching feature point pairs to obtain the spatial coordinate value of the virtual marker at a certain moment; S54. According to the relationship between the virtual marker and the positioned tracking object Spatial position relationship, obtain the spatial coordinate value of the positioned tracking object at a certain moment; S55. By comparing the spatial coordinate values of the positioned tracking object at different times, obtain the movement track of the positioned tracking object.

In order to achieve the above purpose, the three-dimensional space positioning and tracking system based on virtual markers according to the embodiment of the present invention includes the following parts: a naked-eye three-dimensional display device, and the projection component in the naked-eye three-dimensional display device is fixed on the surface of the tracked object or fixed on the positioning device, wherein the end of the positioning device tracks the positioned tracking object; a design module, which is connected to the input end of the projection component, and is used to design a virtual marker and the virtual marker A naked-eye three-dimensional display image source corresponding to an object, wherein the design module sends the naked-eye three-dimensional display image source to the projection component to display the virtual marker in the actual environment space, and the virtual marker and the The relative positions of the projection components remain unchanged; the multi-eye video acquisition module is used to obtain multi-view videos of the scene to be photographed, and the scene to be photographed includes the actual environment space and the virtual marker; the analysis and processing module, the The analysis and processing module is connected to the multi-view video acquisition module, and is used to process and obtain the positioning information and motion information of the positioned tracking object according to the multi-view video.

According to the three-dimensional space positioning and tracking system based on virtual markers in the above embodiment, compared with the prior art, at least three markers that are not completely collinear need to be placed in the actual space, the system does not need to set real markers in space, Instead, it is only necessary to display virtual markers in space by means of projection components in the naked-eye three-dimensional display device. The three-dimensional space positioning and tracking system based on virtual markers of the present invention has the advantages of small space occupation, high operational flexibility, and high positioning accuracy.

The three-dimensional space positioning and tracking system based on virtual markers in the embodiment of the present invention may also have the following technical features:

In an embodiment of the present invention, the design module includes: a first design module, configured to design the number, shape, and size of the virtual markers and the spatial positional relationship between the virtual markers and the projection component; The second design module is configured to design a naked-eye three-dimensional display image source corresponding to the virtual marker, wherein the algorithm for designing the naked-eye three-dimensional display image source is related to the type of the naked-eye three-dimensional display device.

In one embodiment of the present invention, when the surface of the positioned tracking object allows fixing the projection component, the number of the projection component is one, the projection component is fixed on the surface of the positioned tracking object, and the virtual The number of markers is one.

In one embodiment of the present invention, when the surface of the tracked object is not allowed to fix the projection components, the number of the projection components is at least three, and the at least three projection components are respectively fixed in the same position. on the instrument, and ensure that the corresponding at least three virtual markers are not all collinear by adjusting the position.

In one embodiment of the present invention, when the naked-eye three-dimensional display device is based on stereoscopic hologram technology, the projection component is a lens array; when the naked-eye three-dimensional display device is based on light field technology, the projection component is a lens array Array; when the naked-eye three-dimensional display device is based on holographic technology, the projection component is an interference device.

In one embodiment of the present invention, the analysis and processing module includes: a feature point extraction submodule for extracting feature points in the virtual marker; a feature point matching submodule for matching the feature point matching submodule with the The feature point extraction submodule is connected, and the feature point matching submodule is used to match the feature points in different views of the multi-view video at a certain moment to obtain a matching feature point pair; the spatial reconstruction submodule, the The space reconstruction submodule is connected to the feature point matching submodule, and the space reconstruction submodule is used to perform three-dimensional space reconstruction according to the matching feature point pairs to obtain the space coordinate value of the virtual marker at a certain moment; a positioning sub-module, the positioning sub-module is connected to the spatial reconstruction sub-module, and the positioning sub-module is used to obtain the spatial position relationship between the virtual marker and the positioned tracking object at a certain moment The spatial coordinate value of the positioned tracking object; the tracking submodule, the tracking submodule is connected to the positioning submodule, and the tracking submodule is used to compare the spatial coordinate values of the positioned tracking object at different times to obtain The trajectory of the tracked object.

Additional aspects and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

Description of drawings

The above and/or additional aspects and advantages of the present invention will become apparent and comprehensible from the description of the embodiments in conjunction with the following drawings, wherein:

FIG. 1 is a flowchart of a three-dimensional space positioning and tracking method based on virtual markers according to an embodiment of the present invention.

Figure 2a and Figure 2b are illustrations of markers of different shapes designed in the embodiment of the present invention.

Fig. 3 is a schematic diagram of a projecting component fixed on the surface of the tracked object according to the first embodiment.

Fig. 4 is a schematic diagram of a second embodiment where the surface of the tracked object cannot be fixed with projection components.

Fig. 5 is a structural block diagram of a three-dimensional spatial positioning system according to an embodiment of the present invention.

Fig. 6 is a schematic diagram of a three-dimensional space positioning system according to an embodiment of the present invention.

detailed description

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

In describing the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", " Orientation or position indicated by "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. The relationship is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, therefore It should not be construed as a limitation of the present invention.

In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, a feature defined as "first" and "second" may explicitly or implicitly include one or more of these features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

In the present invention, unless otherwise clearly specified and limited, terms such as "installation", "connection", "connection" and "fixation" should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection , or integrally connected; it may be mechanically connected or electrically connected; it may be directly connected or indirectly connected through an intermediary, and it may be the internal communication of two components. Those of ordinary skill in the art can understand the specific meanings of the above terms in the present invention according to specific situations.

Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments or portions of code comprising one or more executable instructions for implementing specific logical functions or steps of the process , and the scope of preferred embodiments of the invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including substantially concurrently or in reverse order depending on the functions involved, which shall It is understood by those skilled in the art to which the embodiments of the present invention pertain.

As shown in Figure 1, the present invention proposes a three-dimensional space positioning and tracking method based on virtual markers, which may include the following steps:

Step S1. Select a naked-eye three-dimensional display device, and fix the projection component in the naked-eye three-dimensional display device on the surface of the tracked object or on the positioning device, wherein the end of the positioning device tracks the positioned tracking object. In other words, placing the tip of the positioning instrument at the tracked location can be used to track that location.

The naked-eye three-dimensional display device refers to a device that is based on the principle of optical imaging and can display in space the image that the observer sees with the naked eye and has a three-dimensional sense after inputting the naked-eye three-dimensional display image source to the projection component. Wherein, the relative positional relationship between the three-dimensional image and the projection component in the naked-eye three-dimensional display device remains unchanged. The naked-eye three-dimensional display device is a technology known to those skilled in the art, and can be flexibly selected according to needs. For example, when the naked-eye three-dimensional display device is based on stereoscopic hologram technology, the projection component is a lens array. When the naked-eye three-dimensional display device is based on light field technology, the projection component is a lens array; when the naked-eye three-dimensional display device is based on holographic technology, the projection component is an interference device. It should be noted that other types of naked-eye three-dimensional display devices known to those skilled in the art may also be selected without changing the principle of the present invention.

When the surface of the positioned tracking object allows the projection component to be fixed, the projection component may be directly fixed on the surface of the positioned tracking object. The fixing method can be in the form of bonding, etc., and can be flexibly selected according to needs. When the surface of the object to be tracked does not allow the projection component to be fixed, the projection component can be fixed by means of a positioning device, the projection component is fixed on the positioning device and the end of the positioning device is tracked to the target. It should be noted that the positioning instrument should be rigid and not deformed during use, and its shape is not limited, and it may be a positioning probe or the like.

S2. Design a virtual marker and a naked-eye three-dimensional display image source corresponding to the virtual marker.

Specifically, step S2 includes the following steps:

S21. It is necessary to design the number, shape, and size of the virtual markers and the spatial position relationship between the virtual markers and the projection components. The shape of the virtual marker can be flexibly adjusted. For example, Fig. 2a and Fig. 2b respectively show two types of virtual markers, a planar checkerboard shape and a cubic checkerboard shape. Their surfaces are covered with a checkerboard texture pattern containing easily identifiable feature points. In addition, it may also be a spherical virtual marker. The spherical virtual marker does not need to be covered with a pattern. In the subsequent image processing, the outline of the ball—the circle—can be extracted, and then the position of the center of the circle can be calculated, and the center point of the circle can be used as a feature point. The size of the virtual marker can be flexibly adjusted according to the size of the application space. If multiple virtual markers need to be designed, their specific spatial positions can also be flexibly adjusted according to the specific conditions of the application, and it is only necessary to ensure that the requirements of the degree of freedom measurement are met. It should be emphasized that these virtual markers have a certain spatial position, so they have three-dimensional and three-dimensional features. Even if the virtual marker itself is a plane texture pattern or a combination of multiple plane texture patterns, it also has three-dimensional features because we have given it a certain spatial position. The virtual marker itself may include multiple feature points, and tracking multiple feature points can improve positioning accuracy.

S22. It is necessary to design a naked-eye three-dimensional display image source corresponding to the virtual marker. Wherein, the algorithm for designing the naked-eye 3D display image source is related to the type of the naked-eye 3D display device. For example, when the naked-eye three-dimensional display device is based on stereoscopic holographic technology, a computer program is written first, and the light emitted by a certain point in the three-dimensional space object is simulated through the lens array by volume rendering or surface rendering, and processed to obtain the naked-eye three-dimensional display image source.

In one embodiment of the present invention, when the surface of the tracked object allows fixing the projection component, the number of the projection component is one, the projection component is fixed on the surface of the tracked object, and the number of the virtual marker is one.

In one embodiment of the present invention, when the surface of the tracked object does not allow fixing projection components, the number of projection components is at least three, and at least three projection components are respectively fixed on the same positioning device, and the corresponding At least three virtual markers are not all collinear. Specifically, the multiple virtual markers may not all be collinear by adjusting the specific positions of each projection component during the installation process or by adjusting the content of the design during the design process.

S3. Input the naked-eye three-dimensional display image source to the projection component to display the corresponding virtual marker in space, and the relative position between the virtual marker and the projection component remains unchanged.

As shown in FIG. 3 , when the surface of the tracked object is allowed to fix the projection component, only one virtual marker is displayed in the space, and the virtual marker moves completely synchronously with the tracked object.

As shown in FIG. 4 , when the surface of the tracked object is not allowed to fix the projection component, at least three virtual markers are displayed in the space. The movement of these virtual markers is related to but not completely synchronized with the movement of the tracked object itself.

S4. Obtain a multi-view video of the scene to be photographed, where the multi-view video includes actual environment space and virtual markers. Wherein, the meaning of multi-view refers to at least two views.

Specifically, equipment such as a multi-eye camera may be used to obtain multi-view videos of the scene to be shot, and the scene to be shot includes virtual images of the actual environment space and at least three landmarks. Technicians can also flexibly select other devices to acquire multi-view video, without changing the principle of the present invention.

S5. According to the multi-view video, process and obtain the positioning information and motion information of the tracked object. Step S5 specifically includes the following steps:

S51. Extract feature points of the virtual marker.

For example, taking a virtual marker in the form of a planar checkerboard as an example, a corner point extraction algorithm can be used to extract feature points. The corner extraction algorithm may be a method such as Harris operator.

S52. Match feature points in different views of the multi-view video at a certain moment to obtain matching feature point pairs. This step is known to those skilled in the art, and will not be described in detail herein.

S53. Perform three-dimensional space reconstruction according to the matching feature point pairs to obtain the space coordinate value of the virtual marker at a certain moment. This step is known to those skilled in the art, and will not be described in detail herein.

S54. Obtain the spatial coordinate value of the positioned tracking object at a certain moment according to the spatial position relationship between the virtual marker and the positioned tracking object.

When the surface of the tracked object allows the projection component to be fixed, the coordinates of the projection component can be estimated based on the coordinates of the unique virtual marker and in combination with the positional relationship between the virtual marker and the projection component. The projection component is attached to the tracked object, so the coordinates of the tracked object are obtained.

At least three virtual markers are displayed in space when the surface of the tracked object does not allow for fixing the projection component. Since the projection component is fixed on the positioning instrument, that is to say, the distance between the projection component and the end of the positioning instrument is known and always constant, so the distance between the virtual marker and the end of the positioning instrument is known and always constant. Therefore, after the coordinates of at least three non-collinear virtual markers are known at a certain moment, the coordinates of the end of the positioning device can be obtained by combining the distance data from these virtual markers to the end of the positioning device. The end of the positioning device tracks the object to be positioned, so the coordinates of the object to be positioned are obtained.

S55. By comparing the spatial coordinate values of the tracked object at different times, the trajectory of the tracked object is obtained. This step is known to those skilled in the art, and will not be described in detail herein.

According to the three-dimensional space positioning and tracking method based on virtual markers in the above embodiment, compared with the prior art that needs to place at least three markers that are not completely collinear in real space, this method does not need to set real markers in space, Instead, it is only necessary to display virtual markers in space by means of projection components in the naked-eye three-dimensional display device. The three-dimensional space positioning and tracking method based on virtual markers of the present invention has the advantages of small space occupation, high operational flexibility, and high positioning accuracy.

FIG. 5 is a structural block diagram of a three-dimensional space positioning system according to an embodiment of the present invention, and FIG. 6 is a schematic diagram of the three-dimensional space positioning system according to an embodiment of the present invention. As shown in Figures 5 and 6, the present invention proposes a three-dimensional space positioning and tracking system based on virtual markers, which may include the following steps: a naked-eye three-dimensional display device 100, a design module 200, a multi-eye video acquisition module 300, and analysis and processing Module 400. in:

The projection component 101 in the naked-eye three-dimensional display device 100 is fixed on the surface of the object to be tracked or fixed on a positioning instrument, wherein the end of the positioning instrument tracks the object to be tracked. The type of the naked-eye three-dimensional display device 100 can be flexibly selected according to needs. When the naked-eye three-dimensional display device 100 is based on stereoholographic technology, the projection component 101 is a lens array; when the naked-eye three-dimensional display device 100 is based on light field technology, the projection component 101 is a lens array; when the naked-eye three-dimensional display device 100 is In the case of a holographic-based type, the projection component 101 is an interference device. It should be noted that other types of naked-eye three-dimensional display devices known to those skilled in the art may also be selected without changing the principle of the present invention.

The design module 200 is used to design a virtual marker and a naked-eye three-dimensional display image source corresponding to the virtual marker. The design module 200 is connected to the input end of the projection component 101 in the naked-eye three-dimensional display device 100 . The design module sends the naked-eye three-dimensional display image source to the projection component 101 to display the virtual marker in the actual environment space. Wherein, the relative position between the virtual marker and the projection component 101 remains unchanged all the time.

In one embodiment of the present invention, the design module 200 specifically includes: a first design module, which is used to design the number, shape, size and spatial position relationship between the virtual markers and the projection component 101; and the second design module A module for designing a naked-eye three-dimensional display image source corresponding to the virtual marker, wherein the algorithm for designing the naked-eye three-dimensional display image source is related to the type of naked-eye three-dimensional display device.

In one embodiment of the present invention, when the surface of the tracked object to be positioned allows fixing the projection component 101, the number of the projection component 101 is one, the projection component 101 is fixed on the surface of the tracked object to be positioned, and the number of virtual markers is One.

In one embodiment of the present invention, when the surface of the tracked object to be positioned does not allow fixing the projection components 101, the number of projection components 101 is at least three, and at least three projection components 101 are respectively fixed on the same positioning device, And by adjusting the positions, it is ensured that the corresponding at least three virtual markers are not all collinear.

The multi-view video acquisition module 300 is used to acquire multi-view video of the scene to be photographed, which includes the actual environment space and virtual markers. Wherein, multi-purpose means at least two eyes.

The analysis and processing module 400 is connected to the multi-view video acquisition module 300, and is used to process and obtain the positioning information and motion information of the tracked object according to the multi-view video.

In one embodiment of the present invention, the analysis and processing module 400 specifically includes: a feature point extraction submodule for extracting feature points in virtual markers; a feature point matching submodule, a feature point matching submodule and a feature point extraction submodule Connected, the feature point matching sub-module is used to match the feature points in different views of the multi-view video at a certain moment to obtain matching feature point pairs; the space reconstruction sub-module, the space reconstruction sub-module is connected with the feature point matching sub-module, the space The reconstruction sub-module is used to reconstruct the three-dimensional space according to the matching feature point pairs to obtain the spatial coordinate value of the virtual marker at a certain moment; the positioning sub-module is connected to the spatial reconstruction sub-module, and the positioning sub-module is used to The spatial position relationship between the object and the tracked object is obtained, and the spatial coordinate value of the tracked object at a certain moment is obtained; the tracking sub-module is connected with the positioning sub-module, and the tracking sub-module is used to compare the positioning at different times The space coordinate value of the tracked object is used to obtain the trajectory of the tracked object.

According to the three-dimensional space positioning and tracking system based on virtual markers in the above embodiment, compared with the prior art, at least three markers that are not completely collinear need to be placed in the actual space, the system does not need to set real markers in space, Instead, it is only necessary to display virtual markers in space by means of projection components in the naked-eye three-dimensional display device. The three-dimensional space positioning and tracking system based on virtual markers of the present invention has the advantages of small space occupation, high operational flexibility, and high positioning accuracy.

In the description of this specification, descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be construed as limitations to the present invention. Variations, modifications, substitutions, and modifications to the above-described embodiments are possible within the scope of the present invention.

Claims (12)

1. a three-dimensional fix method for tracing based on virtual signage thing, it is characterised in that comprise the following steps:

S1. select a kind of naked eye three-dimensional display device, the projection part in described naked eye three-dimensional display device is fixed on and is positioned Follow the trail of the surface of thing or be fixed on locating apparatus, wherein, described in the end tracking of described locating apparatus, being positioned tracking thing；

S2. design virtual signage thing and described virtual signage thing corresponding naked eye three-dimensional display image source；

S3. described projection part is inputted described naked eye three-dimensional display image source, to demonstrate the virtual signage of correspondence in space Thing, wherein, described virtual signage thing remains constant with the relative position of described projection part；

S4. obtaining the multi-view video of scene to be taken the photograph, described multi-view video includes actual environment space and described virtual signage Thing；And

S5. according to described multi-view video, process and described in obtaining, be positioned location information and the movable information following the trail of thing.

2. three-dimensional fix method for tracing as claimed in claim 1, it is characterised in that described step S2 includes:

S21. the space of the number of described virtual signage thing, shape, size and described virtual signage thing and described projection part is designed Position relationship；And

S22. design the naked eye three-dimensional display image source that described virtual signage thing is corresponding, wherein, design described naked eye three-dimensional display figure The algorithm of image source is relevant with described naked eye three-dimensional display device type.

3. method as claimed in claim 2, it is characterised in that allow fixing projection part when being positioned the surface following the trail of thing Time, the number of described projection part is one, described projection part be fixed on described in be positioned follow the trail of thing surface, described void The number intending mark is one.

4. method as claimed in claim 2, it is characterised in that do not allow fixing Projection Division when being positioned the surface following the trail of thing During part, the number of described projection part is at least three, and projection part described at least three is separately fixed at same described fixed On the apparatus of position, and ensure the whole conllinear of virtual signage thing described in corresponding at least three.

5. the method for claim 1, it is characterised in that

When naked eye three-dimensional display device is based on 3 D full-figure technique type, described projection part is lens arra；

When naked eye three-dimensional display device be based on light field poly-talented time, described projection part is lens arra；

When naked eye three-dimensional display device is based on holographic technique type, described projection part is interference device.

6. the method for claim 1, it is characterised in that described step S5 includes:

S51. the characteristic point of described virtual signage thing is extracted；

S52. the described characteristic point in the different views of multi-view video described in a certain moment is mated, obtain matching characteristic point Right；

S53. according to described matching characteristic point to carrying out three dimensions reconstruction, the space of the described virtual signage thing in a certain moment is obtained Coordinate figure；

S54. according to described virtual signage thing and the described spatial relation being positioned tracking thing, the described quilt in a certain moment is obtained The spatial value of location tracking thing；And

S55. by being positioned the spatial value following the trail of thing described in comparing the most in the same time, obtain described in be positioned the fortune following the trail of thing Dynamic track.

7. a three-dimensional fix tracing system based on virtual signage thing, it is characterised in that include with lower part:

Naked eye three-dimensional display device, the projection part in described naked eye three-dimensional display device be fixed on be positioned follow the trail of thing surface or Person is fixed on locating apparatus, wherein, is positioned tracking thing described in the end tracking of described locating apparatus；

Design module, described design module is connected with the input of described projection part, is used for designing virtual signage thing and described void Intending the naked eye three-dimensional display image source that mark is corresponding, wherein, described naked eye three-dimensional display image source is sent by described design module To described projection part to demonstrate described virtual signage thing in actual environment space, described virtual signage thing and described Projection Division The relative position of part remains constant；

Many mesh video acquisition module, for obtaining the multi-view video of scene to be taken the photograph, described in scene to be taken the photograph include that actual environment is empty Between and described virtual signage thing；And

Analysis and processing module, described analysis and processing module is connected with described many mesh video acquisition module, for according to described multi views Video, processes and is positioned location information and the movable information following the trail of thing described in obtaining.

8. system as claimed in claim 7, it is characterised in that described design module includes:

First design module, for designing the number of described virtual signage thing, shape, size and described virtual signage thing with described The spatial relation of projection part；And

Second design module, for designing the naked eye three-dimensional display image source that described virtual signage thing is corresponding, wherein, design is described The algorithm of naked eye three-dimensional display image source is relevant with described naked eye three-dimensional display device type.

9. system as claimed in claim 8, it is characterised in that allow fixing projection part when being positioned the surface following the trail of thing Time, the number of described projection part is one, described projection part be fixed on described in be positioned follow the trail of thing surface, described void The number intending mark is one.

10. system as claimed in claim 8, it is characterised in that do not allow fixing Projection Division when being positioned the surface following the trail of thing During part, the number of described projection part is at least three, and projection part described at least three is separately fixed at same described fixed On the apparatus of position, and ensure the inwhole conllinear of virtual signage thing described in corresponding at least three by adjusting position.

11. systems as claimed in claim 7, it is characterised in that

When naked eye three-dimensional display device is based on 3 D full-figure technique type, described projection part is lens arra；

When naked eye three-dimensional display device be based on light field poly-talented time, described projection part is lens arra；

When naked eye three-dimensional display device is based on holographic technique type, described projection part is interference device.

12. systems as claimed in claim 7, it is characterised in that described analysis and processing module includes:

Feature point extraction submodule, for extracting the characteristic point in described virtual signage thing；

Feature Points Matching submodule, described Feature Points Matching submodule is connected with described feature point extraction submodule, described characteristic point Matched sub-block, for mating the described characteristic point in the different views of multi-view video described in a certain moment, is mated Feature point pairs；

Space reconstruction submodule, described space reconstruction submodule is connected with described Feature Points Matching submodule, described space reconstruction Module is used for according to described matching characteristic point carrying out three dimensions reconstruction, obtains the space of the described virtual signage thing in a certain moment Coordinate figure；

Locator module, described locator module is connected with described space reconstruction submodule, and described locator module is for according to institute State virtual signage thing and the described spatial relation being positioned tracking thing, obtain described in a certain moment, being positioned the sky following the trail of thing Between coordinate figure；And

Following the trail of submodule, described tracking submodule is connected with described locator module, and described tracking submodule is for by ratio less Be positioned described in the same time follow the trail of thing spatial value, obtain described in be positioned follow the trail of thing movement locus.