CN119279510A - A near-infrared two-zone multi-channel in vivo macroscopic stereoscopic imaging system and method - Google Patents
A near-infrared two-zone multi-channel in vivo macroscopic stereoscopic imaging system and method Download PDFInfo
- Publication number
- CN119279510A CN119279510A CN202411488793.6A CN202411488793A CN119279510A CN 119279510 A CN119279510 A CN 119279510A CN 202411488793 A CN202411488793 A CN 202411488793A CN 119279510 A CN119279510 A CN 119279510A
- Authority
- CN
- China
- Prior art keywords
- channel
- image
- near infrared
- stereoscopic
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/0059—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
- A61B5/0071—Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by measuring fluorescence emission
Landscapes
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Biophysics (AREA)
- Pathology (AREA)
- Engineering & Computer Science (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Physics & Mathematics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
Abstract
The invention discloses a near infrared two-region multichannel living body macroscopic stereoscopic imaging system and method. The invention integrates the visible light and infrared imaging technology, adopts a coaxial double-channel lens, an illumination and excitation light source and a three-dimensional display system, and can realize the capture of high-quality three-dimensional images across different spectrum intervals. The system uses a coaxial double-channel lens for light splitting to realize synchronous acquisition of near infrared and visible light signals in the same field of view, enhances image quality by using a specific image processing algorithm, integrates image information of different spectrums by using an image fusion technology, and presents a final three-dimensional imaging result by using a three-dimensional display system and a naked eye 3D display technology. In addition, the depth information is accurately calculated by adopting a binocular depth estimation algorithm, and more specific target depth information is further provided for assisting operation. The technical scheme is beneficial to improving the utilization of living body macroscopic three-dimensional information in a plurality of fields such as medical diagnosis, biological scientific research, industrial detection and the like.
Description
Technical Field
The invention belongs to the field of biomedical images applying optics, and relates to a near infrared two-region multichannel living body macroscopic stereoscopic imaging system and method.
Background
Near infrared two-region (NIR-II, 900-1880 nm wave band) fluorescence imaging is an advanced optical living body imaging method, because biological tissues have low scattering and moderate absorption characteristics on photons in the wave band, compared with the traditional visible light (360-760 nm) and near infrared one-region (760-900 nm) imaging technology, the NIR-II imaging effectively reduces light scattering interference in the two-dimensional planar array imaging process due to the characteristics, and therefore imaging quality is remarkably improved.
Multichannel imaging technology is an imaging technology that has emerged with the development of optical, computer vision, and image processing technologies. Compared with single wavelength window or single mode image capturing in traditional imaging, the multi-channel imaging technology can obtain more dimensional information, and provides a richer data base for analysis of complex scenes.
Stereoscopic imaging techniques originate from the natural stereoscopic vision system of humans, which is capable of observing the same scene from slightly different angles through two eyes, thereby resolving depth information of the scene, capturing images of objects or the scene through two or more angles, and reconstructing the three-dimensional structure of the scene using parallax between the images. Stereoscopic imaging techniques have been widely used in many fields of machine vision, remote sensing, medical imaging, and the like.
The existing near infrared two-region imaging system for biomedical imaging is mostly a single-channel plane imaging system, namely the imaging system only has a fluorescent channel and lacks a visible light channel, so that tissues except fluorescent markers cannot be imaged, and the imaging cannot guide a user to carry out biological operation well because of the lack of depth resolution capability.
Disclosure of Invention
The invention aims to provide a near infrared two-region multichannel living body macroscopic stereoscopic imaging system and method aiming at the defects of the existing near infrared two-region imaging technology. The method is used for solving the defects that the current imaging system cannot analyze the target depth information and lacks a visible light channel, and realizing calculation and three-dimensional display of the target depth information based on a two-channel system.
In a first aspect, the present invention provides a near infrared two-zone multichannel living macro stereoscopic imaging system, comprising:
the system comprises a dual-channel image acquisition device, an illumination and excitation light source and a three-dimensional display system;
the illumination and excitation light source is used for observing biological tissues, fluorescent substances in the biological tissues are excited by the excitation light source so as to emit fluorescent signals in a near infrared two-region, and meanwhile, the parts except the fluorescent substances reflect shadowless lamp signals serving as illumination light sources;
In each set of the two-channel image acquisition equipment, near infrared two-region fluorescent and reflective shadowless lamp signals are split and filtered by the coaxial two-channel lens and then synchronously imaged in the near infrared camera and the visible light camera respectively with the same field of view;
The stereoscopic display system is used for processing images output by the near infrared camera and the visible light camera so as to obtain a multi-channel stereoscopic image with depth information.
In a second aspect, the present invention provides a real-time multichannel living body macroscopic stereoscopic imaging method, adopting the imaging system, comprising the steps of:
Step 1, fluorescent substances in biological tissues are excited by an excitation light source to emit fluorescent signals in a near infrared two-region, and meanwhile, the parts except the fluorescent substances reflect shadowless lamp signals serving as illumination light sources;
In each set of two-channel image acquisition equipment, near-infrared two-region fluorescence and reflected shadowless lamp signals are subjected to light splitting and filtering through a two-channel lens and then respectively imaged on a near-infrared camera and a visible light camera synchronously with the view field;
Step 3, real-time image signals of the four cameras are transmitted into a computer and are respectively a fluorescence left view F L, a fluorescence right view F R, a visible light left view V L and a visible light right view V R, and structural information of a near-fluorescence left and right view F L、FR is enhanced by utilizing an image enhancement algorithm;
Step4, performing image registration on the two groups of obtained double-channel left and right views F L、VL and F R、VR respectively, endowing a fluorescent image with pseudo color, and fusing the fluorescent image with a visible light image to obtain a double-channel image FV L、FVR;
and the registered left and right double-channel images FV L、FVR are sent into a naked eye 3D display, and the control of the propagation directions of the left and right images is realized through a cylindrical lens, so that the left and right eyes of a viewer receive different pictures, and parallax is formed to obtain a stereoscopic image.
Compared with the prior art, the invention has the remarkable advantages that:
1) Compared with a conventional single-channel imaging system, the coaxial double-channel lens is used for collecting visible light signals, and simultaneously imaging tissues except for fluorescent markers, so that more abundant biological tissue information is provided;
2) Compared with the conventional monocular camera which can only provide two-dimensional image information, the method enables a plurality of cameras to work in a synchronous mode, and can generate and display multichannel stereoscopic images with depth information in real time by combining an image processing algorithm and a naked eye 3D display technology. The depth perception is provided for a user, and more accurate positioning of the target tissue structure is realized;
3) The system has accurate depth calculation capability in two channels, and can provide accurate depth information of a target object.
Drawings
Fig. 1 is a schematic view of the optical path structure of the present invention.
Fig. 2 is a flow chart of a real-time multichannel macroscopic in-vivo imaging method.
Fig. 3 shows an embodiment of a dual channel left view FV L (a), a dual channel right view FV R (b), a mixed output left and right view image (c) (where green is a near infrared two-region fluorescence signal), and a dual channel depth image (d) (where pseudo-color indicates depth size).
Detailed Description
The invention is described in further detail below with reference to the accompanying drawings.
Referring to fig. 1, the real-time multichannel living body macroscopic stereoscopic imaging system based on near infrared two regions and visible light provided by the invention comprises visible light cameras 1 and 6, infrared cameras 2 and 5, coaxial double-channel lenses 3 and 7, an illumination and excitation light source 8 and a stereoscopic display system 9.
Further exemplary, the illumination and excitation light sources include one of an illumination light source, a shadowless lamp, and an excitation light source, a laser or an LED light source.
Further exemplary, the visible light camera specifically employs the same type of visible light sensitive industrial camera.
Further exemplary, the infrared camera is embodied as an indium gallium arsenide deep refrigeration camera that is sensitive to the same model of infrared band.
Further exemplary, the dual channel lens employs a coaxial dual channel lens that is broadband antireflection to visible to near infrared light (360-1700 nm). The visible light channel of the lens collects signals of 360-700 nm, the near infrared channel collects signals of more than 1000 nm, and the lens is correspondingly optimized for various aberrations of the two channels.
Further exemplary, the stereoscopic display system includes a computer and a stereoscopic display device. Computers are particularly large bandwidth computers that can support multiple high definition video inputs. The stereoscopic display device specifically adopts a naked eye 3D display, the display is provided with a multi-lens screen, and the control of the propagation directions of left and right images is realized by using vertically arranged cylindrical lenses, so that different pictures are received by left and right eyes of a viewer, and parallax is formed to obtain a stereoscopic image.
With reference to fig. 2, the real-time multichannel living body macroscopic stereoscopic imaging method provided by the invention comprises the following steps:
in step 1, fluorescent substances in the biological tissue 4 are excited by an excitation light source to emit fluorescent signals in a near infrared region, and at the same time, the parts other than the fluorescent substances reflect shadowless lamp signals serving as illumination light sources.
And 2, two sets of double-channel image acquisition equipment which are arranged in parallel at a fixed distance b acquire four paths of signals at the same time.
In each set of double-channel image acquisition equipment, near-infrared two-region fluorescence and reflected shadowless lamp signals are subjected to light splitting and filtering through a double-channel lens and then are respectively imaged on an infrared camera and a visible light camera sensor synchronously with the same view field;
Step 3, real-time image signals of the four cameras are transmitted into a computer, namely a fluorescence left view F L, a fluorescence right view F R, a visible light left view V L and a visible light right view V R, and structural information of a near-fluorescence left and right view F L、FR is enhanced by utilizing an image enhancement algorithm based on Hessian characteristics and Frangi filtering;
And 4, respectively carrying out image registration on the two groups of obtained double-channel left and right views F L、VL and F R、VR, endowing a fluorescent image with proper pseudo color, and fusing the fluorescent image with a visible light image to obtain a double-channel image FV L、FVR. Because parallax exists between the left and right views, the registered left and right double-channel views FV L、FVR are sent into the naked eye 3D display, the control of the propagation directions of the left and right images is realized through the cylindrical lenses, and the left and right eyes of a viewer can receive different pictures to form parallax so as to obtain a stereoscopic image;
And 5, performing matching operation on the FV L、FVR by using a binocular global matching algorithm, calculating to obtain a dual-channel parallax image, performing weighted least square filtering on the dual-channel parallax image, and performing connected domain detection to remove a small noise region in the parallax image, which appears due to mismatching, so as to obtain a processed parallax image FV D.
And obtaining a double-channel specific depth image of the target based on calibrated visible and infrared camera parameters and a fixed distance b between cameras, and endowing the double-channel specific depth image with proper pseudo color to represent depth output.
Example mice were dehaired in advance and orbital injected with indocyanine green (ICG, FDA certified with fluorescence center wavelength 810 nm, but fluorescence tail above 900 nm was still available for clear near infrared two-zone imaging) in 200 μl (concentration 0.2 mg/mL) of aqueous solution.
During imaging, a mouse is placed under a double-channel lens, irradiated by 793 nm excitation light, and two sets of double-channel collecting systems are arranged in parallel at a fixed distance b, and four paths of signals are collected. In each set of double-channel collecting system, near infrared two-region fluorescence and reflected visible light signals are split by a double-channel lens and imaged on an infrared camera and a visible light camera sensor synchronously with the same field of view respectively, and real-time image signals of four cameras are transmitted into a computer and are respectively a fluorescence left view F L, a fluorescence right view F R, a visible light left view V L and a visible light right view V R.
And (3) respectively carrying out image registration on the obtained two groups of double-channel left and right views F L、VL and F R、VR, endowing a fluorescent image with proper pseudo color, and fusing the fluorescent image with a visible light image to obtain a double-channel image FV L、FVR. And matching the FV L、FVR by using a binocular global matching algorithm, calculating to obtain a double-channel parallax image, carrying out weighted least square filtering on the double-channel parallax image, and carrying out connected domain detection to remove a small noise region in the parallax image, which appears due to mismatching, so as to obtain a processed parallax image FV D. Based on calibrated visible and infrared camera parameters and a fixed distance b between cameras, obtaining a double-channel depth image of the whole body and liver of the mouse, respectively endowing proper pseudo-colors to represent the depth and outputting the depth image, as shown in figure 3.
The invention discloses a real-time multichannel living body macroscopic stereoscopic imaging system based on near infrared two regions and visible light and an imaging method thereof. The system integrates a visible light camera, a near infrared camera, a coaxial double-channel lens, an illumination and excitation light source and a stereoscopic display system, can capture high-quality stereoscopic images in different spectrum intervals, and provides depth perception for users. The system has wide application prospect in clinical fluorescence operation navigation.
The above-described embodiments are intended to illustrate the present invention, not to limit it, and any modifications and variations made thereto are within the spirit of the invention and the scope of the appended claims.
Claims (10)
1. A near infrared two-zone multichannel living body macroscopic stereoscopic imaging system, characterized in that the system comprises:
the system comprises a dual-channel image acquisition device, an illumination and excitation light source and a three-dimensional display system;
the illumination and excitation light source is used for observing biological tissues, fluorescent substances in the biological tissues are excited by the excitation light source so as to emit fluorescent signals in a near infrared two-region, and meanwhile, the parts except the fluorescent substances reflect shadowless lamp signals serving as illumination light sources;
In each set of the two-channel image acquisition equipment, near infrared two-region fluorescent and reflective shadowless lamp signals are split and filtered by the coaxial two-channel lens and then synchronously imaged in the near infrared camera and the visible light camera respectively with the same field of view;
The stereoscopic display system is used for processing images output by the near infrared camera and the visible light camera so as to obtain a multi-channel stereoscopic image with depth information.
2. The near infrared two-zone multichannel living body macroscopic stereoscopic imaging system of claim 1, wherein the illumination and excitation light source comprises an illumination light source and an excitation light source, wherein the illumination light source is a shadowless lamp, and the excitation light source is one of a laser or an LED light source.
3. The near infrared two-zone multichannel living macro stereoscopic imaging system of claim 1, wherein the visible camera is a visible-light sensitive industrial camera.
4. The near infrared two-region multichannel living body macroscopic stereoscopic imaging system of claim 1, wherein the near infrared camera is an indium gallium arsenic depth refrigeration camera sensitive to near infrared band.
5. The near infrared two-region multi-channel living body macroscopic stereoscopic imaging system according to claim 3 or 4, wherein the coaxial dual-channel lens is a coaxial dual-channel lens for broadband anti-reflection of visible light to near infrared light, the visible light channel of the lens collects signals of 360-700 nm, the near infrared channel collects signals of more than 1000 nm, and the lens is correspondingly optimized for various aberrations of the two channels.
6. The near infrared two-area multichannel living body macroscopic stereoscopic imaging system according to claim 1 is characterized by comprising a computer and stereoscopic display equipment, wherein the computer specifically adopts a large-bandwidth computer capable of supporting multipath high-definition video input, the stereoscopic display equipment specifically adopts an naked eye 3D display, the display is provided with a multi-lens screen, and the control of the propagation directions of left and right images is realized by using vertically arranged cylindrical lenses, so that different pictures are received by left and right eyes of a viewer to form parallax so as to obtain stereoscopic images.
7. A real-time multichannel living macro stereoscopic imaging method employing the imaging system of any one of claims 1 to 6, characterized by comprising the steps of:
Step 1, fluorescent substances in biological tissues are excited by an excitation light source to emit fluorescent signals in a near infrared two-region, and meanwhile, the parts except the fluorescent substances reflect shadowless lamp signals serving as illumination light sources;
In each set of two-channel image acquisition equipment, near-infrared two-region fluorescence and reflected shadowless lamp signals are subjected to light splitting and filtering through a two-channel lens and then respectively imaged on a near-infrared camera and a visible light camera synchronously with the view field;
Step 3, real-time image signals of the four cameras are transmitted into a computer and are respectively a fluorescence left view F L, a fluorescence right view F R, a visible light left view V L and a visible light right view V R, and structural information of a near-fluorescence left and right view F L、FR is enhanced by utilizing an image enhancement algorithm;
Step4, performing image registration on the two groups of obtained double-channel left and right views F L、VL and F R、VR respectively, endowing a fluorescent image with pseudo color, and fusing the fluorescent image with a visible light image to obtain a double-channel image FV L、FVR;
and the registered left and right double-channel images FV L、FVR are sent into a naked eye 3D display, and the control of the propagation directions of the left and right images is realized through a cylindrical lens, so that the left and right eyes of a viewer receive different pictures, and parallax is formed to obtain a stereoscopic image.
8. The method of claim 1, wherein the structural information of the near-fluorescence left-right view F L、FR is enhanced in step 3 by using an image enhancement algorithm based on Hessian features and Frangi filtering.
9. The method for real-time multi-channel living body macroscopic stereoscopic imaging according to claim 1, further comprising the steps of 5, performing matching operation on left and right dual-channel images FV L、FVR, calculating to obtain a dual-channel parallax image, performing weighted least square filtering on the dual-channel parallax image, performing connected domain detection to remove a small noise region in the parallax image due to mismatching, and obtaining a processed parallax image FV D;
And obtaining a double-channel specific depth image of the target based on calibrated camera parameters and a fixed distance between cameras, and endowing the double-channel specific depth image with pseudo color to represent the depth and outputting the pseudo color.
10. The method for real-time multi-channel living body macroscopic stereoscopic imaging according to claim 1, wherein the binocular global matching algorithm is utilized to perform matching operation on the left and right double-channel images FV L、FVR and calculate a double-channel parallax map.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411488793.6A CN119279510A (en) | 2024-10-24 | 2024-10-24 | A near-infrared two-zone multi-channel in vivo macroscopic stereoscopic imaging system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202411488793.6A CN119279510A (en) | 2024-10-24 | 2024-10-24 | A near-infrared two-zone multi-channel in vivo macroscopic stereoscopic imaging system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN119279510A true CN119279510A (en) | 2025-01-10 |
Family
ID=94166779
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202411488793.6A Pending CN119279510A (en) | 2024-10-24 | 2024-10-24 | A near-infrared two-zone multi-channel in vivo macroscopic stereoscopic imaging system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN119279510A (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110025830A1 (en) * | 2009-07-31 | 2011-02-03 | 3Dmedia Corporation | Methods, systems, and computer-readable storage media for generating stereoscopic content via depth map creation |
| CN114947752A (en) * | 2022-06-28 | 2022-08-30 | 浙江大学 | Macroscopic two-channel living body imaging system based on visible light and near-infrared two-zone fluorescence |
| CN115530995A (en) * | 2021-06-29 | 2022-12-30 | 南京微纳视界医疗科技有限公司 | Medical imaging apparatus |
| CN117653020A (en) * | 2022-08-22 | 2024-03-08 | 南京微纳视界医疗科技有限公司 | Three-dimensional imaging device, surgical navigation system and three-dimensional imaging method |
-
2024
- 2024-10-24 CN CN202411488793.6A patent/CN119279510A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110025830A1 (en) * | 2009-07-31 | 2011-02-03 | 3Dmedia Corporation | Methods, systems, and computer-readable storage media for generating stereoscopic content via depth map creation |
| CN115530995A (en) * | 2021-06-29 | 2022-12-30 | 南京微纳视界医疗科技有限公司 | Medical imaging apparatus |
| CN114947752A (en) * | 2022-06-28 | 2022-08-30 | 浙江大学 | Macroscopic two-channel living body imaging system based on visible light and near-infrared two-zone fluorescence |
| CN117653020A (en) * | 2022-08-22 | 2024-03-08 | 南京微纳视界医疗科技有限公司 | Three-dimensional imaging device, surgical navigation system and three-dimensional imaging method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106236006B (en) | 3D optical molecular image laparoscope imaging systems | |
| CN104783767B (en) | Device and method for detecting human body microcirculation by means of orthogonal polarization spectral imaging | |
| US20210195128A1 (en) | Solid-state image capturing device, information processing device, information processing system, information processing method, and computer program | |
| US12193786B2 (en) | Apparatus and method for image-guided interventions with hyperspectral imaging | |
| EP3559741B1 (en) | Three-dimensional image reconstruction using multi-layer data acquisition | |
| CN109222865B (en) | Multimode imaging endoscope system | |
| CN110638527A (en) | Augmented Reality Surgical Microscopic Imaging System Based on Optical Coherence Tomography | |
| KR20160089355A (en) | Device for non-invasive detection of predetermined biological structures | |
| CN112656369B (en) | AR technology-based near-infrared fluorescent parathyroid gland identification enhancement method | |
| JP2014023609A (en) | Imaging observation device | |
| CN102846312B (en) | A multi-dimensional imaging detector and its data inversion method | |
| JP7183616B2 (en) | IMAGING DEVICE, SIGNAL PROCESSING DEVICE, SIGNAL PROCESSING METHOD AND PROGRAM | |
| CN114947752A (en) | Macroscopic two-channel living body imaging system based on visible light and near-infrared two-zone fluorescence | |
| CN107518879A (en) | A kind of fluoroscopic imaging device and method | |
| CN104055489A (en) | Blood vessel imaging device | |
| CN109496316A (en) | Image recognition system | |
| JP2012143363A (en) | Image processing apparatus | |
| CN109752377B (en) | A spectroscopic dual-mode projection tomographic tissue vascular imaging device and method | |
| CN109738879A (en) | Active laser detection apparatus | |
| CN119279510A (en) | A near-infrared two-zone multi-channel in vivo macroscopic stereoscopic imaging system and method | |
| CN119342363B (en) | Multispectral depth camera and depth measurement method | |
| CN115177199A (en) | Image processing system, surgical system, and image processing method | |
| KR101133503B1 (en) | Integrated optical and x-ray ct system and method of reconstructing the data thereof | |
| EP4006615A1 (en) | Imaging system | |
| CN110115557B (en) | A hyperspectral endoscopic imaging device and imaging method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |