CN104280886A - Microscopic system and microscopic method based on in-situ three-dimensional enhanced display - Google Patents

Abstract

The invention discloses a microscopic system and a microscopic method based on in-situ stereo enhanced display, enabling users to observe entities clearly and continuously within a larger field of view. The microscopic system in the embodiment of the present invention includes: a physical observation device, which is aimed at the physical area, and is used to allow the user to obtain real-time and direct visual images of the physical objects; Content-related naked-eye true 3D images; in-situ stereoscopic display device, the in-situ stereoscopic display device is respectively connected with the physical observation device and the naked-eye 3D display device, and is used for in-situ stereoscopic fusion of the real visual image and the naked-eye true 3D image for display to the user. The present invention has the advantages of simple structure, large operation flexibility, meeting different requirements, realizing high-resolution stereo fusion display, or realizing portability and miniaturization of the system, and the like.

Description

Microscopic system and microscopic method based on in-situ stereo enhanced display

technical field

The invention relates to microscopic technology, in particular to a microscopic system and a microscopic method based on in-situ stereo enhanced display.

Background technique

Microsystems are used more and more widely in research fields such as clinical medicine, biology, and pharmacy. Traditional microscopic displays are obtained directly through a microscope. Take Zeiss’ OPMI PENTERO 900 surgical microscope and AxioImager 2 upright research microscope platform as an example. The observation system consists of the main objective lens, binocular lens tube and zoom unit. By matching eyepieces and objective lenses with different magnifications, different physical magnifications can be achieved. Multiples and working distance requirements. At the same time, the in-mirror projection function allows the user to fuse and display certain additional image information in the entity to guide the user's operation. However, the diameter of the field of view of this microscope is very small, and it can only be observed in a straight line, and it is easy to produce dead angles for observation. At the same time, the additional image information that can be projected and displayed in the mirror is very limited. Although the existing flat display method can provide the trial user with valuable and various required image information, it lacks intuition.

In order to solve problems such as field of view and image information display, people have done a lot of research on 3D head-mounted displays. For example, the University of North Carolina uses a see-through head-mounted display to make a stereoscopic "stethoscope" and a head-mounted device for endoscopic surgery planning and preoperative simulation. The head-mounted display places micro-displays in the respective fields of view of the wearer's left and right eyes. Based on the principle that the left and right eyes can produce stereo vision when observing different images, the entity visual image is obtained by using a high-definition stereo camera device that can automatically zoom, and the observation entity and The three-dimensional image data of the entity is combined to present a real-time image of the augmented reality fusion display in the head-mounted display. The report pointed out that under the three-dimensional display, the user's understanding accuracy and operation accuracy have been improved, and the operation time can also be shortened. However, there are obvious limitations in the development of head-mounted displays. For example, when the resolution requirements for in-situ fusion images increase, the size of the microdisplay will increase accordingly, and the overall size and weight of the device will also increase; The wearable display will cause inconsistency in the user's vergence and focus adjustment. In order to ensure realistic three-dimensional imaging, head-mounted displays have problems such as large size, inconsistency between observation time frame and focus adjustment, etc., which are prone to visual fatigue, dizziness and discomfort when used for a long time. When the human eye observes a real object, the convergence and focus adjustment are consistent. When watching a stereoscopic image, the eye adjustment is on the screen, but the convergence is on the virtual stereoscopic image synthesized by parallax. Therefore, long-term viewing will inevitably lead to visual fatigue.

It can be seen from the above that although the current microscopic system can directly observe the entity, the field of view is small, and it is difficult to enhance the display of the 3D image in situ; while the head-mounted display can perform high-definition 3D image fusion, but it is difficult to guarantee the user's accuracy at the same time. The comfort and the clarity of the fusion image are prone to discomfort. At present, there is no microscopic display system that not only satisfies the in-situ enhanced display of three-dimensional stereoscopic images, but also ensures a sufficient field of view, and cannot provide users with real, intuitive, three-dimensional enhanced display of in-situ fused microscopic information of entities.

Contents of the invention

The present invention aims to solve one of the technical problems in the related art at least to a certain extent. Therefore, the object of the present invention is to propose a microscopic system and microscopic method based on in-situ stereo enhanced display with simple structure and high operational flexibility.

The microscopic system based on in-situ stereo enhanced display according to an embodiment of the present invention may include: a physical observation device, which is aimed at the physical area, and is used for allowing the user to directly obtain the physical visual image in real time; A three-dimensional display device, configured to provide a naked-eye true three-dimensional image related to the content of the solid visual image; an in-situ stereoscopic display device, the in-situ stereoscopic display device is respectively connected to the physical observation device and the naked-eye three-dimensional display device, It is used for displaying to the user after in-situ stereoscopic fusion of the solid visual image and the naked-eye true three-dimensional image.

In addition, the microscope system according to the above-mentioned embodiments of the present invention may also have the following additional technical features:

Optionally, the naked-eye three-dimensional display device includes a naked-eye real three-dimensional image source and a projection component, and the projection component is used to project and display a corresponding naked-eye real three-dimensional image in space according to the naked-eye real three-dimensional image source, wherein , when the naked-eye three-dimensional display device is based on stereoholographic technology, the projection component is a lens array; when the naked-eye three-dimensional display device is based on light field technology, the projection component is a lens array; when the naked-eye three-dimensional display device is a lens array In the type based on holographic technology, the projection means is an interference device.

Optionally, the in-situ stereoscopic display device includes: a first fixing part, used for fixing the projection part in the naked-eye three-dimensional display device; a second fixing part, used for fixing the entity viewing device; a transflective device, the relative positions of the transflective device and the first fixing part and the second fixing part meet the following conditions: the solid visual image and the naked-eye true three-dimensional image The in-situ stereoscopic perspective fusion is performed and displayed to the user, wherein the solid visual image is displayed to the user after passing through the transflective device, and the naked-eye true three-dimensional image is reflected by the transflective device displayed to the user.

Optionally, in the in-situ stereoscopic display device, between the naked-eye true three-dimensional image and the transflective device further includes: a spatial image fusion unit and a reflective unit with controllable optical path, so as to realize scalable and compact Fusion projection structure.

Optionally, it also includes: at least one of an entity image enlargement unit, a true three-dimensional image scaling unit, and a fusion image scaling unit, wherein the entity image enlargement unit is used to enlarge the entity before performing in-situ stereo perspective fusion The size of the visual image, the true three-dimensional image scaling unit is used to scale the size of the naked-eye true three-dimensional image before in-situ stereo perspective fusion, and the fusion image scaling unit is used to scale the entity visual image and the naked-eye The image size of the true 3D image after in-situ stereoscopic fusion.

Optionally, it further includes: an aberration correction processing module, the aberration correction processing module is used for correcting aberrations.

Optionally, in the in-situ stereoscopic display device, after registering the entity visual image and the naked-eye true three-dimensional image, the in-situ display is performed and then displayed to the user, wherein the registration method is based on marker points registration or registration without landmarks.

According to the embodiment of the present invention, the microscopic system based on the in-situ stereo enhanced display may include the following steps: A. Acquiring the entity visual image in real time and directly; B. Providing a naked-eye true three-dimensional image related to the content of the entity visual image C. Displaying the entity visual image and the naked-eye true three-dimensional image to the user after in-situ stereoscopic fusion.

In addition, the microscope system according to the above-mentioned embodiments of the present invention may also have the following additional technical features:

Optionally, the step B specifically includes: providing a naked-eye true three-dimensional image source, and projecting and displaying a corresponding naked-eye true three-dimensional image in space through a projection component, wherein, when the naked-eye three-dimensional display method is based on stereoscopic holographic technology , the projection component is a lens array; when the naked-eye three-dimensional display method is based on light field technology, the projection component is a lens array; when the naked-eye three-dimensional display method is based on holographic technology, the projection component is an interference device.

Optionally, in the step C, the solid visual image and the naked-eye true three-dimensional image are fused in situ through a transflective device and then displayed to the user, wherein the solid visual image is transmitted through The transflective device is then displayed to the user, and the naked-eye true three-dimensional image is displayed to the user after being reflected by the transflective device.

Optionally, an expandable and compact fusion projection structure is used for optical path design.

Optionally, at least one of the following steps is also included: enlarging the size of the solid visual image before performing in-situ stereo perspective fusion; scaling the size of the naked-eye true three-dimensional image before performing in-situ stereo perspective fusion; scaling The image size after in-situ stereo perspective fusion of the solid visual image and the naked-eye true three-dimensional image.

Optionally, a step of performing aberration correction processing is also included.

Optionally, in the step C, the entity visual image and the naked-eye real three-dimensional image are registered and displayed to the user after in-situ display, wherein the registration method is registration based on landmark points or Registration without landmarks.

The microscopic system and microscopic method based on in-situ stereo enhanced display according to the embodiments of the present invention enable users to perform microscopic observation and manipulation of entities within a larger field of view while providing observers with true three-dimensional The in-situ enhanced display image of naked-eye three-dimensional entities can achieve clear, intuitive and continuous observation of the combination of internal and external entities with the assistance of perspective fusion images. The present invention also has the advantages of simple structure, great operational flexibility, meeting different needs, realizing high-resolution stereo fusion display, or realizing portability and miniaturization of the system.

Description of drawings

Fig. 1 is a structural block diagram of a microscope system based on in-situ stereo enhanced display according to an embodiment of the present invention.

Fig. 2 is a schematic structural diagram of an in-situ stereoscopic microscope device according to an embodiment of the present invention.

Fig. 3 is a schematic structural diagram of a scalable and compact fusion projection according to an embodiment of the present invention.

Fig. 4a is a schematic diagram of a microscope system in which a true 3D image scaling unit reduces the size of a naked-eye true 3D image; Fig. 4b is a schematic diagram of a microscope system in which a true 3D image scaling unit enlarges the size of a naked-eye true 3D image.

Fig. 5a is a schematic diagram of a microscope system based on in-situ stereo enhancement display without aberration processing; Fig. 5b is a schematic diagram of a microscope system based on in-situ stereo enhancement display with aberration reduction processing.

Fig. 6 is a flowchart of a microscopic method based on in-situ stereo enhanced display according to an embodiment of the present invention.

Detailed ways

Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

The microscopic system based on in-situ stereo enhanced display according to the embodiment of the first aspect of the present invention is shown in FIG.

The entity observation device 10 is aimed at the entity area, and is used for allowing the user to directly obtain entity visual images in real time. The so-called solid visual image refers to an image within the range of eyesight during use. For example, if the patient's head needs surgical treatment during the operation, the hair, skin, bones, blood vessels, organs (visible after craniotomy) and other contents of the patient's head directly observed by the doctor with the naked eye belong to entity vision the content of the image. The simplest entity observation device 10 can be one or more optical lenses, which are aimed at the entity area so that the user can directly observe with the naked eye to obtain the entity visual image. Preferably, the entity observation device 10 may also use a combination of a display screen and an image acquisition device to provide the observer with real-time images of the entity being observed. Of course, the entity observation device 10 may also be equipped with accessory structures such as a shading cover, a dustproof transparent cover, etc., which will not be described in detail herein.

The naked-eye three-dimensional display device 20 is used to provide naked-eye true three-dimensional images related to the content of the entity visual image. There are two meanings here: first, the image provided by the naked-eye three-dimensional display device 20 is a three-dimensional virtual image formed in space that looks three-dimensional to the naked eye. The naked-eye three-dimensional display device 20 refers to a device based on the principle of optical imaging, capable of displaying in space a three-dimensional image seen by a user with naked eyes. Second, the content of the naked-eye true 3D image is related to the content of the physical visual image. For example, during a certain operation, the solid area is the patient's head, and the solid visual image is the image of the head seen intuitively. At this time, the content of the naked-eye true three-dimensional image provided by the naked-eye three-dimensional display device 20 should also be an image related to the head. rather than images related to other locations such as the chest cavity.

It should be noted that the naked-eye true three-dimensional image is used to provide users (such as doctors) with rich data information, which is convenient for users to make analysis and judgments. The naked-eye true 3D image source used to generate the naked-eye true 3D image can be volume data generated by methods such as computer tomography (CT) and computer 3D modeling, or it can be imaging such as X-ray transmission and optical coherence tomography (OCT). The two-dimensional tomographic image produced by the method can also be the surface texture image produced by methods such as camera and structured light. In addition, the content of the naked-eye true 3D image can be obtained before the operation or in real time during the operation, or it can be the result of real-time registration and fusion of the images before and during the operation.

It should be noted that the naked-eye three-dimensional display device 20 includes a naked-eye true three-dimensional image source 21 and a projection component 22 . The naked-eye true three-dimensional image image source 21 is the component that provides the content source of the naked-eye true three-dimensional image. The storage device of the original image corresponding to the image (for example, a hard disk storing two-dimensional ultrasound images) can be flexibly selected by those skilled in the art according to needs, which is not limited herein. The projection component 22 is used for projecting and displaying a naked-eye true three-dimensional image in space according to the naked-eye true three-dimensional image source 21 . For example, a two-dimensional flat light-emitting display screen is used as the naked-eye true three-dimensional image source 21, so that the light-emitting display screen presents a pre-designed pattern, and then the light emitted by the light-emitting display screen is projected in space after passing through the projection component 22 A real three-dimensional image with naked eyes can be produced. The technical details of the naked-eye three-dimensional display device 20 are known to those skilled in the art, and can be flexibly selected according to needs. When the naked-eye three-dimensional display device 20 is based on stereoscopic hologram technology, the projection component 22 is a lens array. When the naked-eye three-dimensional display device 20 is based on light field technology, the projection component 22 is a lens array. When the naked-eye three-dimensional display device 20 is based on holographic technology, the projection component 22 is an interference device. It should be noted that other types of naked-eye three-dimensional display devices known to those skilled in the art may also be selected without changing the principle of the present invention. And, it should be noted that the algorithm for designing the original image corresponding to the naked-eye true 3D image is related to the type of the naked-eye 3D display device. For example, when the naked-eye three-dimensional display device 20 is based on stereoscopic holographic technology, it is necessary to write a computer program first, and use volume rendering or surface rendering to simulate the light emitted by a certain point in a three-dimensional space object through the lens array, and process it to obtain a naked-eye true three-dimensional image. the corresponding original image. In order to realize the dynamic real-time update of the naked-eye true 3D image during the operation, parallel processing methods such as GPU can be used to accelerate the rendering of the original image corresponding to the naked-eye true 3D image.

The in-situ stereoscopic display device 30 is respectively connected with the physical observation device 10 and the naked-eye three-dimensional display device 20, and is used for in-situ fusion of the physical visual image and the naked-eye real three-dimensional image to display to the user. The in-situ stereoscopic display device 30 is mainly composed of a series of optical elements (lenses, mirrors, transflective devices, etc.) brackets and more. The specific form of the in-situ three-dimensional display device 30 is not limited in the present invention, it is only necessary to ensure that the user can see the real visual image and the naked-eye real three-dimensional image at the same time for the in-situ displayed image.

FIG. 2 shows a schematic structural diagram of an in-situ stereoscopic display device 30 according to a specific embodiment of the present invention. As shown in FIG. 2 , the in-situ stereoscopic display device 30 includes: a first fixing part 31 , a second fixing part 32 and a transflective device 33 . The first fixing member 31 is used to fix the projection component in the naked-eye three-dimensional display device 20 . The second fixing member 32 is used for fixing the physical observation device 10 . The relative positions of the transflective device 33 and the physical observation device 10, the first fixing part 31 and the second fixing part 32 meet the following conditions: the solid visual image and the naked-eye true three-dimensional image are stereoscopically seen in situ at the transflective device 33 After fusion, it is displayed to the user, wherein the solid visual image is displayed to the user after passing through the transflective device 33 , and the naked-eye true three-dimensional image is displayed to the user after being reflected by the transflective device 33 . It should be noted that it should be ensured that the optical path from the transflective device 33 to the entity is consistent with the optical path from the naked-eye three-dimensional display device 20 to the transflective device 33 . Preferably, the transflective device 33 may be a liquid crystal screen type transflective device. The liquid crystal screen type transflective device can realize controllable transmittance and reflectance. The LCD screen itself can also be used as a display screen. In specific applications, it can be used as an additional flat-screen display to increase the amount of information in the system's in-situ image, or as a touch screen to collect user operations and enhance the interaction between the system and users.

In the in-situ three-dimensional display device 30 of the embodiment shown in FIG. 2 , the optical path propagation between the naked-eye three-dimensional display device 20 and the transflective device 33 is completed by means of linear projection, which causes the device to occupy a relatively large space, causing problems when the space is limited. It eliminates the inconvenience of device control and user operation. In view of this situation, the applicant also proposes an in-situ stereoscopic display device using an expandable and compact fusion projection structure, which will be described in detail below with reference to FIG. 3 .

In the in-situ stereoscopic display device 30 of the microscopic system based on the in-situ stereoscopic enhanced display in one embodiment of the present invention, a space image fusion unit 34 and a light A controllable reflection unit 35, as shown in FIG. 3 . Among them: the spatial image fusion structure 34 is based on the multi-transflective device mechanism, and realizes the spatial fusion of several independent and side-by-side three-dimensional naked-eye stereoscopic images through optical hardware at the front end of the projection device, and realizes the registration and fusion of simple multi-modal image information, Improve the richness of information displayed by the system; at the same time, the image fusion based on optical hardware is very fast, so it can be applied to occasions with high requirements on imaging speed. The reflective structure 35 with controllable optical path is used to reduce the length of the optical axis and make the system more compact. In this embodiment, by adopting the zoom lens array design in the optical path and the reflective mirror group with adjustable spacing, it is possible to ensure the consistency of the optical path and expand the visual range while reducing the volume of the system.

In an embodiment of the present invention, the microscope system based on in-situ stereo enhanced display may further include at least one of a solid image scaling unit 41 , a true 3D image scaling unit 42 and a fusion image scaling unit 43 . Wherein: the entity image enlargement unit 41 is used to enlarge the size of the entity visual image before the in-situ stereo perspective fusion, for example, the entity image enlargement unit 41 can be a lens or a combination of lenses positioned between the entity region and the transflective device 33 . The true 3D image scaling unit 42 is used for scaling the size of the naked-eye true 3D image before in-situ stereoscopic fusion. For example, the true three-dimensional image scaling unit 42 may include a lens or lens combination located between the projection component and the transflective device 33 . The fused image scaling unit 43 is used to scale the image size of the solid visual image and the naked-eye true 3D image after the in-situ stereo perspective fusion. For example, the fused image scaling unit 43 may comprise a lens or combination of lenses positioned between the user's eye and the transflective device 33 . It can be known from the above that the user can obtain the best observation effect by comprehensively adjusting the respective zoom factors of the entity image zoom unit 41 , the true 3D image zoom unit 42 and the fused image zoom unit 43 .

It should be noted that, in addition to the lens or lens combination, the solid image zoom unit 41 , the true three-dimensional image zoom unit 42 and the fused image zoom unit 43 also have matching operating devices. The control and operating device is used to move the lens or lens group to a proper working position. The operating device can be flexibly set, and can be automatically operated in the form of a motor, or manually operated in the form of a guide rail, etc.

It should be noted that the true 3D image scaling unit 42 has the most significant significance for scaling the naked-eye true 3D image. An example will be described below with reference to FIG. 4a and FIG. 4b.

In an embodiment of the present invention, the true 3D image scaling unit 42 may reduce the size of the naked-eye true 3D image. As shown in Figure 4a, the initial naked-eye true three-dimensional image is marked as A, and the size of A is reduced by the true three-dimensional image scaling unit 42, which is equivalent to forming a reduced-sized virtual image A' in space. The transflective device 33 reflects into the user's eyes. In this embodiment, the original large-size naked-eye true 3D image can be reduced and concentrated to a small size and then fused with the solid visual image, which reduces the resolution requirements for the original naked-eye true 3D image source and improves the clarity of the fused image degree.

In another embodiment of the present invention, the true 3D image scaling unit 42 may enlarge the size of the naked-eye true 3D image. As shown in Figure 4b, the initial naked-eye true three-dimensional image is marked as B, and the size of B is enlarged by the true three-dimensional image scaling unit 42, which is equivalent to forming a virtual image B' of enlarged size in space, and finally B' passes half The transflective device 33 is reflected and reflected into the user's eyes. In this embodiment, the original small-sized naked-eye real three-dimensional image can be enlarged and expanded to a large size and then merged with the solid visual image, which can reduce the area requirement of the projection component in the naked-eye three-dimensional display device, and is conducive to the hardware miniaturization of the entire microscope system change.

In an embodiment of the present invention, the microscope system based on in-situ stereo enhanced display further includes: an aberration correction processing module 50, which is used for eliminating aberrations. Due to the imperfection of the actual optical components, it will lead to defects such as blurring and deformation of the imaging. As shown in Figure 5a. After the original naked-eye true 3D image C is reduced, C' with aberrations is generated, so that the fusion image observed by the user has obvious aberrations. At this time, the imaging system needs to be corrected for aberrations. Here, the computer simulation technology can be used to quantitatively measure the aberration value of the optical system. There are generally two ways to correct the aberration: ① optimize the initial optical design parameters based on the aberration value, readjust the focal length of each optical component in the relevant zoom unit, Radius, spacing and other parameters to achieve the minimum aberration. ②Adjust the naked-eye stereoscopic display image based on the aberration value, so that the aberration of the naked-eye stereoscopic image after being formed by the lens group is minimized. Depending on the specific application, one of the aberration correction methods can be selected, or the two methods can be used in combination. Figure 5b provides an embodiment, on the basis of Figure 5a, the lens in the true three-dimensional image scaling unit 42 is replaced by a doublet, and a naked-eye real three-dimensional image D with pre-deformation compensation is provided, and the D is obtained after the doublet. The aberration-free virtual image D', the user finally observes a three-dimensional virtual fusion image without aberration.

In the in-situ stereoscopic display device 30 , when performing image fusion, it is necessary to register the solid visual image and the naked-eye real three-dimensional image, so as to ensure that the image observed by the user can be correctly fused in the original position. There are two ways of registration: registration based on landmarks and registration without landmarks. When using a registration algorithm based on landmark points, the registration relationship with the stereoscopic image can be calculated using optimization theory based on automatically or manually determined landmarks. At the same time, the intraoperative positioning equipment is required to continuously track the physical target during the operation, so as to realize the real-time update of the registration information when the target moves. When the registration algorithm without marker points is used, the reconstruction method of the 3D scene can be used, and the registration relationship with the stereoscopic image is calculated using optimization theory based on the 3D surface contour information. At the same time, it is necessary to use the intraoperative positioning device to continuously track the physical target during the operation, so as to realize the real-time update of the registration information when the target moves. Based on the above registration results with or without marker points, the optimal spatial coordinate transformation relationship between the two is calculated, and the size of the stereoscopic image display is adjusted based on the size of the physical target. This step is known to those skilled in the art, and will not be described in detail herein.

It should be noted that if the magnification of the system in use changes, or the pose of the physical object changes, it is necessary to re-register the stereo fusion image and the physical target entity, and update the display content of the stereo fusion image. When the magnification factor changes, the displayed image is scaled according to the magnification; when the position of the physical target changes, it is necessary to use the intraoperative positioning device to continuously track the physical target in use, and perform registration again according to the target surface marker points or surface contour information. The registration results update the displayed image again.

To sum up, the microscope system based on the in-situ stereo enhanced display in the embodiment of the present invention enables the user to clearly and continuously observe the solid and stereo fusion images within a larger field of view, and has the advantages of simple structure, It has the advantages of large operation flexibility, can meet different needs, can realize high-resolution stereo fusion display, or can realize the portability and miniaturization of the system.

The microscopic method based on in-situ stereo enhanced display according to the embodiment of the second aspect of the present invention, as shown in FIG. 6 , includes the following steps:

A. Real-time and direct acquisition of physical visual images;

B. Provide naked-eye true three-dimensional images related to the content of physical visual images;

C. The in-situ stereoscopic fusion of the solid visual image and the naked-eye true three-dimensional image is displayed to the user.

In one embodiment of the present invention, step B specifically includes: providing a naked-eye true three-dimensional image source, and projecting and displaying a corresponding naked-eye true three-dimensional image in space through a projection component. There are many methods for naked-eye 3D display, which can be flexibly selected according to needs. When the naked-eye three-dimensional display method is based on stereoscopic holographic technology, the projection component is a lens array; when the naked-eye three-dimensional display method is based on light field technology, the projection component is a lens array; when the naked-eye three-dimensional display method is based on holographic technology , the projection component is an interference device.

In one embodiment of the present invention, in step C, the solid visual image and the naked-eye true three-dimensional image are fused in situ through a transflective device and then displayed to the user. Among them, the solid visual image is displayed to the user after passing through the transflective device, and the naked-eye true three-dimensional image is displayed to the user after being reflected by the transflective device.

In one embodiment of the present invention, an expandable and compact fusion projection structure is used for optical path design. In this way, the volume of the equipment can be reduced, and the limitation on the size of the installation site can be reduced.

In an embodiment of the present invention, the image can also be zoomed. Specifically, at least one of the following steps may be included: enlarging the size of the entity visual image before performing in-situ stereoscopic perspective fusion; scaling the size of the naked-eye real three-dimensional image before performing in-situ stereoscopic perspective fusion; scaling the entity visual image and the naked-eye The image size of the true 3D image after in-situ stereoscopic fusion. It can be seen from the above that the user can obtain the best observation effect by comprehensively controlling the respective zoom factors of the solid visual image, the naked-eye true 3D image and the fused image.

In an embodiment of the present invention, further comprising: performing aberration correction processing. After the aberration correction process, the user can finally observe a three-dimensional virtual fusion image without aberration.

In one embodiment of the present invention, the entity visual image and the naked-eye real three-dimensional image are registered and then displayed to the user in situ, wherein the registration method is registration based on landmark points or registration without landmark points. allow.

In summary, the microscopic method based on in-situ stereo enhanced display in the embodiment of the present invention enables users to observe entities and stereo fusion images clearly and continuously within a larger field of view, and has the advantages of simple process, It has the advantages of large operation flexibility, can meet different needs, can realize high-resolution stereo fusion display, or can realize the portability and miniaturization of the system.

In describing the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", " Back", "Left", "Right", "Vertical", "Horizontal", "Top", "Bottom", "Inner", "Outer", "Clockwise", "Counterclockwise", "Axial", The orientation or positional relationship indicated by "radial", "circumferential", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying the referred device or element Must be in a particular orientation, be constructed in a particular orientation, and operate in a particular orientation, and therefore should not be construed as limiting the invention.

In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

In the present invention, unless otherwise clearly specified and limited, terms such as "installation", "connection", "connection" and "fixation" should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection , or integrated; it may be mechanically connected or electrically connected; it may be directly connected or indirectly connected through an intermediary, and it may be the internal communication of two components or the interaction relationship between two components, unless otherwise specified limit. Those of ordinary skill in the art can understand the specific meanings of the above terms in the present invention according to specific situations.

In the present invention, unless otherwise clearly specified and limited, the first feature may be in direct contact with the first feature or the first and second feature may be in direct contact with the second feature through an intermediary. touch. Moreover, "above", "above" and "above" the first feature on the second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. "Below", "beneath" and "beneath" the first feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is less horizontally than the second feature.

Any process or method descriptions in flowcharts or otherwise described herein may be understood to represent modules, segments or portions of code comprising one or more executable instructions for implementing specific logical functions or steps of the process , and the scope of preferred embodiments of the invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including substantially concurrently or in reverse order depending on the functions involved, which shall It is understood by those skilled in the art to which the embodiments of the present invention pertain.

In the description of this specification, descriptions referring to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or characteristic is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples and features of different embodiments or examples described in this specification without conflicting with each other.

Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention, those skilled in the art can make the above-mentioned The embodiments are subject to changes, modifications, substitutions and variations.

Claims (14)

1. strengthen a microscopic system for display based on original position solid, it is characterized in that, comprising:

Physical inspection device, described physical inspection device aims at entity area, obtains stereopsis image in real time, directly for allowing user;

Naked eye three-dimensional display device, for providing the bore hole true 3-D view relevant to described stereopsis picture material;

Original position 3 d display device, described original position 3 d display device is connected with described naked eye three-dimensional display device with described physical inspection device respectively, after described stereopsis image and the true 3-D view of described bore hole are carried out original position volume rendering fusion, be shown to user.

2. microscopic system according to claim 1, it is characterized in that, described naked eye three-dimensional display device comprises the true three-dimensional image source of bore hole and projection part, described projection part be used for according to described bore hole true three-dimensional image source in space Projection Display go out the corresponding true 3-D view of bore hole, wherein

When naked eye three-dimensional display device is based on 3 D full-figure technique type, described projection part is lens arra;

When naked eye three-dimensional display device is poly-talented based on light field, described projection part is lens arra;

When naked eye three-dimensional display device is based on holographic technique type, described projection part is interference device.

3. microscopic system according to claim 2, is characterized in that, described original position 3 d display device comprises:

First fixture, for the projection part in fixing described naked eye three-dimensional display device;

Second fixture, for fixing described physical inspection device;

Semi-transparent semi-reflecting device, the relative position of described semi-transparent semi-reflecting device and described first fixture and described second fixture meets following condition: described stereopsis image and the true 3-D view of described bore hole are shown to user after described semi-transparent semi-reflecting device place carries out original position volume rendering fusion, wherein, described stereopsis image is shown to user through after described semi-transparent semi-reflecting device, and the true 3-D view of described bore hole is shown to user after described semi-transparent semi-reflecting device reflection.

4. microscopic system according to claim 3, it is characterized in that, in described original position 3 d display device, also comprise between the true 3-D view of described bore hole and described semi-transparent semi-reflecting device: aerial image integrated unit and the controlled reflector element of light path, merge projection structure to realize easily extensible close-coupled.

5. microscopic system according to claim 3, it is characterized in that, also comprise: solid images amplifying unit, in true 3-D view unit for scaling and fused images unit for scaling one of at least, wherein, described solid images amplifying unit is for amplifying the size of the described stereopsis image before carrying out original position volume rendering fusion, described true 3-D view unit for scaling carries out the size of the true 3-D view of described bore hole before original position volume rendering fusion for convergent-divergent, described fused images unit for scaling carries out the picture size after original position volume rendering fusion for stereopsis image described in convergent-divergent and the true 3-D view of described bore hole.

6. microscopic system according to claim 1, is characterized in that, also comprises: aberration correction processing module, and described aberration correction processing module is used for aberration correction.

7. microscopic system according to claim 1, it is characterized in that, in described original position 3 d display device, be shown to user after carrying out home position manifestation after described stereopsis image and the true 3-D view of described bore hole are carried out registration, wherein the mode of registration is the registration of registration based on monumented point or no marks point.

8. strengthen a microscopic method for display based on original position solid, it is characterized in that, comprise the following steps:

A. in real time, directly stereopsis image is obtained;

B., the true 3-D view of bore hole relevant to described stereopsis picture material is provided;

C. user is shown to after described stereopsis image and the true 3-D view of described bore hole being carried out original position volume rendering fusion.

9. microscopic method according to claim 8, is characterized in that, described step B specifically comprises: provide bore hole true three-dimensional image source, by projection part in space Projection Display go out the corresponding true 3-D view of bore hole, wherein,

When naked eye three-dimensional display packing is based on 3 D full-figure technique type, described projection part is lens arra;

When naked eye three-dimensional display packing is poly-talented based on light field, described projection part is lens arra;

When naked eye three-dimensional display packing is based on holographic technique type, described projection part is interference device.

10. microscopic method according to claim 9, it is characterized in that, in described step C, user is shown to after described stereopsis image and the true 3-D view of described bore hole being carried out original position volume rendering fusion by semi-transparent semi-reflecting device, wherein, described stereopsis image is shown to user through after described semi-transparent semi-reflecting device, and the true 3-D view of described bore hole is shown to user after described semi-transparent semi-reflecting device reflection.

11. microscopic methods according to claim 10, is characterized in that, adopt easily extensible close-coupled to merge projection structure when carrying out light path design.

12. microscopic methods according to claim 10, is characterized in that, in also comprising the following steps one of at least:

Amplify the size of the described stereopsis image before carrying out original position volume rendering fusion;

Convergent-divergent carries out the size of the true 3-D view of described bore hole before original position volume rendering fusion;

Stereopsis image described in convergent-divergent and the true 3-D view of described bore hole carry out the picture size after original position volume rendering fusion.

13. microscopic methods according to claim 8, is characterized in that, also comprise step: carry out aberration correction process.

14. microscopic methods according to claim 8, it is characterized in that, in described step C, be shown to user after carrying out home position manifestation after described stereopsis image and the true 3-D view of described bore hole are carried out registration, wherein the mode of registration is the registration of registration based on monumented point or no marks point.