[go: up one dir, main page]

CN108120680B - Method and device for removing stray light in microscopic imaging based on prior photoelectric characteristics - Google Patents

Method and device for removing stray light in microscopic imaging based on prior photoelectric characteristics Download PDF

Info

Publication number
CN108120680B
CN108120680B CN201711373195.4A CN201711373195A CN108120680B CN 108120680 B CN108120680 B CN 108120680B CN 201711373195 A CN201711373195 A CN 201711373195A CN 108120680 B CN108120680 B CN 108120680B
Authority
CN
China
Prior art keywords
photo
stray light
camera
light source
exposure time
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201711373195.4A
Other languages
Chinese (zh)
Other versions
CN108120680A (en
Inventor
范静涛
陈熙
戴琼海
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tsinghua University
Original Assignee
Tsinghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tsinghua University filed Critical Tsinghua University
Priority to CN201711373195.4A priority Critical patent/CN108120680B/en
Publication of CN108120680A publication Critical patent/CN108120680A/en
Application granted granted Critical
Publication of CN108120680B publication Critical patent/CN108120680B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/01Arrangements or apparatus for facilitating the optical investigation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/14Picture signal circuitry for video frequency region
    • H04N5/21Circuitry for suppressing or minimising disturbance, e.g. moiré or halo
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10056Microscopic image
    • G06T2207/10061Microscopic image from scanning electron microscope
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20024Filtering details
    • G06T2207/20032Median filtering

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Biochemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Theoretical Computer Science (AREA)
  • Solid State Image Pick-Up Elements (AREA)
  • Studio Devices (AREA)

Abstract

本发明公开了一种基于光电特性先验的显微成像的杂散光去除方法及装置,其中方法包括:布置暗室并在相机支持的曝光时间参数区间上以对数间断取点,拍摄第一照片;在暗室中仅打开仪器光源,在物镜下方使用光陷阱吸收光,以相机曝光时间和光源亮度组合,在光源亮度上均匀取点,且在相机曝光时间上述取点相同,拍摄第二照片;将第二照片减去第一照片以消除相机本身像感器对实验结果的干扰,进而获取杂散光在相机曝光时间和光源亮度上的变化规律;在对特定的一张图分析杂散光强度在空间上的优化分布模型时,根据变化规律得到采集照片中每个像素的杂散光数值,以去除杂散光的影响。该方法可以减少算法的时间复杂度,有效提高实验效率和图片清晰度。

The invention discloses a stray light removal method and device for microscopic imaging based on prior photoelectric characteristics, wherein the method includes: arranging a darkroom and taking points with logarithmic intervals on the exposure time parameter interval supported by the camera, and taking the first photo ;Only turn on the light source of the instrument in the darkroom, use light traps under the objective lens to absorb light, use the combination of camera exposure time and light source brightness, uniformly take points on the light source brightness, and take the same point at the camera exposure time, and take the second photo; Subtract the first photo from the second photo to eliminate the interference of the camera's own image sensor on the experimental results, and then obtain the change law of stray light in the exposure time of the camera and the brightness of the light source; analyze the intensity of stray light in a specific picture. When optimizing the distribution model in space, the stray light value of each pixel in the collected photos is obtained according to the change law, so as to remove the influence of stray light. This method can reduce the time complexity of the algorithm, and effectively improve the experimental efficiency and picture clarity.

Description

基于光电特性先验的显微成像的杂散光去除方法及装置Method and device for removing stray light in microscopic imaging based on prior photoelectric characteristics

技术领域technical field

本发明涉及计算机视觉技术和图像处理技术领域,特别涉及一种基于光电特性先验的显微成像的杂散光去除方法及装置。The invention relates to the fields of computer vision technology and image processing technology, in particular to a stray light removal method and device for microscopic imaging based on prior photoelectric characteristics.

背景技术Background technique

多维多尺度高分辨率计算摄像仪器是目前世界上最大的兼顾“宽视场、高分辨率、高帧率”的计算摄像仪器,通过高维耦合信息的可逆计算,解耦重建拍到的原始图像,可以实现多维多尺度高分辨率的连续观测。该仪器要求拍摄出来的照片在达到帧率要求的同时具有较低的信噪比,可以显示更多的细节,以期望能够清晰地观察到细胞层次的生命运动,从而帮助实验人员抓住生命过程中的有效信息,不仅节约了操作时间(生物实验的最大难度在于其难以复现,因为生物样本是非常珍贵的),而且可以提高后续解耦重建图像的精度,从而为生命科学领域提供巨大的支持。The multi-dimensional, multi-scale and high-resolution computational imaging instrument is currently the world's largest computational imaging instrument that takes into account "wide field of view, high resolution, and high frame rate". Images can realize multi-dimensional, multi-scale and high-resolution continuous observation. The instrument requires that the photos taken by the instrument have a lower signal-to-noise ratio while meeting the frame rate requirements, and can display more details, in order to clearly observe the life movement at the cell level, thereby helping experimenters to grasp the life process The effective information in it not only saves operation time (the biggest difficulty in biological experiments is that it is difficult to reproduce, because biological samples are very precious), but also can improve the accuracy of subsequent decoupling and reconstruction images, thus providing huge benefits for the field of life sciences. support.

此外,该计算摄像仪器拍摄的样本主要为荧光样本,荧光蛋白在波长为488的光激发下发出荧光,但是荧光强度(有效信息)非常低,相对的,其他干扰就比较高。因此,减弱成像过程中仪器固有特性对样本成像的干扰至关重要,其中影响最大的是非成像光线(主要是光源光线)在光路中的非正常传递到达像感器表面的杂散光。然而,相关技术的实验效率低,图片清晰度差,算法较为复杂,有待解决。In addition, the samples taken by the computational imaging instrument are mainly fluorescent samples. The fluorescent protein emits fluorescence under the excitation of light with a wavelength of 488, but the fluorescence intensity (effective information) is very low, and other interferences are relatively high. Therefore, it is very important to reduce the interference of the inherent characteristics of the instrument on the imaging of the sample during the imaging process. The most influential is the stray light that is not normally transmitted by the non-imaging light (mainly the light source light) in the optical path and reaches the surface of the image sensor. However, the experimental efficiency of related technologies is low, the picture definition is poor, and the algorithm is relatively complicated, which needs to be solved.

发明内容Contents of the invention

本发明旨在至少在一定程度上解决相关技术中的技术问题之一。The present invention aims to solve one of the technical problems in the related art at least to a certain extent.

为此,本发明的一个目的在于提出一种基于光电特性先验的显微成像的杂散光去除方法,该方法可以减少算法的时间复杂度,有效提高实验效率,有效提高图片清晰度。Therefore, an object of the present invention is to propose a method for removing stray light in microscopic imaging based on photoelectric characteristics prior, which can reduce the time complexity of the algorithm, effectively improve the experimental efficiency, and effectively improve the picture clarity.

本发明的另一个目的在于提出一种基于光电特性先验的显微成像的杂散光去除方装置。Another object of the present invention is to propose a device for removing stray light in microscopic imaging based on prior photoelectric characteristics.

为达到上述目的,本发明一方面实施例提出了一种基于光电特性先验的显微成像的杂散光去除方法,包括以下步骤:步骤A1:布置暗室并在相机支持的曝光时间参数区间上以对数间断取点,拍摄第一照片;步骤A2:在所述暗室中仅打开仪器光源,在物镜下方使用光陷阱吸收光,以相机曝光时间和光源亮度组合,在所述光源亮度上均匀取点,且在所述相机曝光时间上与所述步骤A1取点相同,拍摄第二照片;步骤A3:将所述第二照片减去所述第一照片以消除相机本身像感器对实验结果的干扰,进而获取杂散光在所述相机曝光时间和所述光源亮度上的变化规律;步骤A4:在对特定的一张图分析杂散光强度在空间上的优化分布模型时,根据所述变化规律得到采集照片中每个像素的杂散光数值,以去除杂散光的影响。In order to achieve the above-mentioned purpose, an embodiment of the present invention proposes a stray light removal method based on photoelectric characteristic priori microscopic imaging, including the following steps: Step A1: Arranging a darkroom and setting the exposure time parameter range supported by the camera with Take logarithmically intermittent points and take the first photo; step A2: only turn on the light source of the instrument in the darkroom, use light traps under the objective lens to absorb light, combine the exposure time of the camera with the brightness of the light source, and uniformly obtain the brightness of the light source point, and take the same point as the step A1 on the camera exposure time, take a second photo; step A3: subtract the first photo from the second photo to eliminate the camera’s own image sensor to the experimental results interference, and then obtain the change law of stray light in the exposure time of the camera and the brightness of the light source; Step A4: When analyzing the optimal distribution model of stray light intensity in space for a specific picture, according to the change Regularly obtain the stray light value of each pixel in the collected photos to remove the influence of stray light.

本发明实施例的基于光电特性先验的显微成像的杂散光去除方法,针对成像仪器的光路在一段时间内空间位置不变的性质,事先采集在不同光源亮度和不同相机曝光时间下的杂散光数据,进行分析,将复杂的建模计算放在平时空余时间内完成,降低了实时算法的时间复杂度,可以实现根据拍摄照片时的相关参数,实时生成对应的杂散光空间分布模型,在图像存储之前就去除杂散光的影响,从而减少了算法的时间复杂度,不仅有效提高了实验效率,而且有效提高了图片清晰度。The stray light removal method of microscopic imaging based on photoelectric characteristics prior in the embodiment of the present invention aims at the property that the optical path of the imaging instrument does not change in space within a period of time, and collects the stray light under different light source brightness and different camera exposure time Astigmatism data is analyzed, and complex modeling calculations are completed in the usual spare time, which reduces the time complexity of the real-time algorithm, and can generate the corresponding stray light spatial distribution model in real time according to the relevant parameters when taking photos. The influence of stray light is removed before image storage, thereby reducing the time complexity of the algorithm, not only effectively improving the experimental efficiency, but also effectively improving the image clarity.

另外,根据本发明上述实施例的基于光电特性先验的显微成像的杂散光去除方法还可以具有以下附加的技术特征:In addition, the stray light removal method based on photoelectric characteristic prior microscopic imaging according to the above-mentioned embodiments of the present invention may also have the following additional technical features:

进一步地,在本发明的一个实施例中,所述步骤A3,进一步包括:对所述第一照片和所述第二照片进行中值滤波,并取每张图片的中间二分之一大小的平均灰度值作为照片的灰度值,分别在只考虑所述相机曝光时间和所述光源亮度的情况下对一组照片进行线性拟合,以根据两组拟合系数得到平均灰度值关于所述相机曝光时间和所述光源亮度的二元一次函数。Further, in one embodiment of the present invention, the step A3 further includes: performing a median filter on the first photo and the second photo, and taking the middle half of the size of each picture The average gray value is used as the gray value of the photo, and a group of photos are linearly fitted when only the camera exposure time and the brightness of the light source are considered, so as to obtain the average gray value according to two sets of fitting coefficients. A binary linear function of the camera exposure time and the brightness of the light source.

进一步地,在本发明的一个实施例中,所述步骤A4,进一步包括:步骤1:对照片的每一行单独取出以进行分段线性拟合,并采用阈值的方法分为三段,阈值为照片的平均灰度值乘以修正系数,通过穷尽搜索的方法得到峰值信噪比的局部极大值,对每段进行多项式拟合;步骤2:对所述照片的每一列单独取出,并重复所述步骤1的操作;步骤3:取两次模型的均值作为最终的杂散光强度在空间上的优化分布模型,并结合所述变化规律得到所述杂散光数值。Further, in one embodiment of the present invention, the step A4 further includes: Step 1: Separately extract each line of the photo for piecewise linear fitting, and divide it into three sections by using a threshold method, the threshold is The average gray value of the photo is multiplied by the correction coefficient, and the local maximum value of the peak signal-to-noise ratio is obtained by exhaustive search, and polynomial fitting is performed on each segment; Step 2: Take out each column of the photo separately, and repeat The operation of the step 1; step 3: taking the mean value of the two models as the final optimal distribution model of the stray light intensity in space, and combining the change rule to obtain the stray light value.

进一步地,在本发明的一个实施例中,将一元高次多项式转化为多元线性函数,并使用最小二乘法精确拟合多项式系数,实现多项式拟合。Furthermore, in one embodiment of the present invention, the polynomial of high degree in one variable is transformed into a multivariate linear function, and the least square method is used to accurately fit the coefficients of the polynomial to realize polynomial fitting.

进一步地,在本发明的一个实施例中,所述峰值信噪比的计算公式为:Further, in one embodiment of the present invention, the formula for calculating the peak signal-to-noise ratio is:

其中,MSE是模型与原图的均方误差,n是图像的位数。Among them, MSE is the mean square error between the model and the original image, and n is the number of bits of the image.

为达到上述目的,本发明另一方面实施例提出了一种基于光电特性先验的显微成像的杂散光去除装置,包括:第一拍摄模块,用于布置暗室并在相机支持的曝光时间参数区间上以对数间断取点,拍摄第一照片;第二拍摄模块,用于在所述暗室中仅打开仪器光源,在物镜下方使用光陷阱吸收光,以相机曝光时间和光源亮度组合,在所述光源亮度上均匀取点,且在所述相机曝光时间上与所述步骤A1取点相同,拍摄第二照片;消除模块,用于将所述第二照片减去所述第一照片以消除相机本身像感器对实验结果的干扰,进而获取杂散光在所述相机曝光时间和所述光源亮度上的变化规律;处理模块,用于在对特定的一张图分析杂散光强度在空间上的优化分布模型时,根据所述变化规律得到采集照片中每个像素的杂散光数值,以去除杂散光的影响。In order to achieve the above purpose, another embodiment of the present invention proposes a stray light removal device for microscopic imaging based on photoelectric characteristics a priori, including: a first shooting module for arranging a darkroom and setting the exposure time parameters supported by the camera Take points with logarithmic intervals on the interval, and take the first photo; the second shooting module is used to turn on only the light source of the instrument in the dark room, and use a light trap to absorb light under the objective lens, and combine the exposure time of the camera with the brightness of the light source, and then The brightness of the light source is evenly selected, and the exposure time of the camera is the same as that of the step A1, and the second photo is taken; the elimination module is used to subtract the first photo from the second photo to obtain Eliminate the interference of the camera's own image sensor on the experimental results, and then obtain the change law of stray light in the exposure time of the camera and the brightness of the light source; the processing module is used to analyze the intensity of stray light in a specific image in space When optimizing the distribution model above, the stray light value of each pixel in the collected photos is obtained according to the change rule, so as to remove the influence of stray light.

本发明实施例的基于光电特性先验的显微成像的杂散光去除装置,针对成像仪器的光路在一段时间内空间位置不变的性质,事先采集在不同光源亮度和不同相机曝光时间下的杂散光数据,进行分析,将复杂的建模计算放在平时空余时间内完成,降低了实时算法的时间复杂度,可以实现根据拍摄照片时的相关参数,实时生成对应的杂散光空间分布模型,在图像存储之前就去除杂散光的影响,从而减少了算法的时间复杂度,不仅有效提高了实验效率,而且有效提高了图片清晰度。The stray light removal device for microscopic imaging based on photoelectric characteristics prior in the embodiment of the present invention aims at the property that the optical path of the imaging instrument does not change in spatial position within a period of time, and collects the stray light under different light source brightness and different camera exposure time in advance. Astigmatism data is analyzed, and complex modeling calculations are completed in the usual spare time, which reduces the time complexity of the real-time algorithm, and can generate the corresponding stray light spatial distribution model in real time according to the relevant parameters when taking photos. The influence of stray light is removed before image storage, thereby reducing the time complexity of the algorithm, not only effectively improving the experimental efficiency, but also effectively improving the image clarity.

另外,根据本发明上述实施例的基于光电特性先验的显微成像的杂散光去除装置还可以具有以下附加的技术特征:In addition, the stray light removal device based on photoelectric characteristic prior microscopic imaging according to the above-mentioned embodiments of the present invention may also have the following additional technical features:

进一步地,在本发明的一个实施例中,所述消除模块还用于对所述第一照片和所述第二照片进行中值滤波,并取每张图片的中间二分之一大小的平均灰度值作为照片的灰度值,分别在只考虑所述相机曝光时间和所述光源亮度的情况下对一组照片进行线性拟合,以根据两组拟合系数得到平均灰度值关于所述相机曝光时间和所述光源亮度的二元一次函数。Further, in one embodiment of the present invention, the elimination module is also used to perform median filtering on the first photo and the second photo, and take the average value of the middle half size of each picture The gray value is used as the gray value of the photo, and a group of photos is linearly fitted when only the camera exposure time and the brightness of the light source are considered, so as to obtain the average gray value according to the two sets of fitting coefficients. A quadratic linear function of the exposure time of the camera and the brightness of the light source.

进一步地,在本发明的一个实施例中,所述处理模块还用于对照片的每一行单独取出以进行分段线性拟合,并采用阈值的方法分为三段,阈值为照片的平均灰度值乘以修正系数,通过穷尽搜索的方法得到峰值信噪比的局部极大值,对每段进行多项式拟合,并对所述照片的每一列单独取出,并重复上述的操作,以及取两次模型的均值作为最终的杂散光强度在空间上的优化分布模型,并结合所述变化规律得到所述杂散光数值。Further, in one embodiment of the present invention, the processing module is also used to separately extract each line of the photo for piecewise linear fitting, and divide it into three segments by using a threshold method, where the threshold is the average gray value of the photo Degree value multiplied by the correction coefficient, the local maximum value of peak signal-to-noise ratio is obtained by exhaustive search method, polynomial fitting is performed on each segment, and each column of the photo is taken out separately, and the above operation is repeated, and the obtained The mean value of the two models is used as the final optimal distribution model of the stray light intensity in space, and the value of the stray light is obtained by combining the change rule.

进一步地,在本发明的一个实施例中,将一元高次多项式转化为多元线性函数,并使用最小二乘法精确拟合多项式系数,实现多项式拟合。Furthermore, in one embodiment of the present invention, the polynomial of high degree in one variable is transformed into a multivariate linear function, and the least square method is used to accurately fit the coefficients of the polynomial to realize polynomial fitting.

进一步地,在本发明的一个实施例中,所述峰值信噪比的计算公式为:Further, in one embodiment of the present invention, the formula for calculating the peak signal-to-noise ratio is:

其中,MSE是模型与原图的均方误差,n是图像的位数。Among them, MSE is the mean square error between the model and the original image, and n is the number of bits of the image.

本发明附加的方面和优点将在下面的描述中部分给出,部分将从下面的描述中变得明显,或通过本发明的实践了解到。Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention.

附图说明Description of drawings

本发明上述的和/或附加的方面和优点从下面结合附图对实施例的描述中将变得明显和容易理解,其中:The above and/or additional aspects and advantages of the present invention will become apparent and easy to understand from the following description of the embodiments in conjunction with the accompanying drawings, wherein:

图1为根据本发明一个实施例的基于光电特性先验的显微成像的杂散光去除方法的流程图;1 is a flowchart of a stray light removal method based on photoelectric characteristic prior microscopic imaging according to an embodiment of the present invention;

图2为根据本发明一个具体实施例的基于光电特性先验的显微成像的杂散光去除方法的流程图;2 is a flow chart of a stray light removal method based on photoelectric characteristic prior microscopic imaging according to a specific embodiment of the present invention;

图3为根据本发明一个实施例的基于光电特性先验的显微成像的杂散光去除装置的结构示意图。FIG. 3 is a schematic structural diagram of a stray light removal device based on photoelectric characteristic prior microscopic imaging according to an embodiment of the present invention.

具体实施方式Detailed ways

下面详细描述本发明的实施例,所述实施例的示例在附图中示出,其中自始至终相同或类似的标号表示相同或类似的元件或具有相同或类似功能的元件。下面通过参考附图描述的实施例是示例性的,旨在用于解释本发明,而不能理解为对本发明的限制。Embodiments of the present invention are described in detail below, examples of which are shown in the drawings, wherein the same or similar reference numerals designate the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary and are intended to explain the present invention and should not be construed as limiting the present invention.

下面参照附图描述根据本发明实施例提出的基于光电特性先验的显微成像的杂散光去除方法及装置,首先将参照附图描述根据本发明实施例提出的基于光电特性先验的显微成像的杂散光去除方法。The stray light removal method and device based on photoelectric characteristic prior microscopic imaging according to the embodiments of the present invention will be described below with reference to the accompanying drawings. Stray light removal methods for imaging.

图1是本发明一个实施例的基于光电特性先验的显微成像的杂散光去除方法的流程图。FIG. 1 is a flow chart of a method for removing stray light in microscopic imaging based on prior photoelectric characteristics according to an embodiment of the present invention.

如图1所示,该基于光电特性先验的显微成像的杂散光去除方法包括以下步骤:As shown in Figure 1, the stray light removal method based on photoelectric characteristic prior microscopic imaging includes the following steps:

步骤A1:布置暗室并在相机支持的曝光时间参数区间上以对数间断取点,拍摄第一照片。Step A1: Arrange the darkroom and take the logarithmic intervals on the exposure time parameter interval supported by the camera, and take the first photo.

可以理解的是,如图2所示,布置暗室,杜绝任何光源,在相机支持的曝光时间参数区间上以对数间断取点,拍摄照片。It can be understood that, as shown in Figure 2, the darkroom is arranged to eliminate any light source, and the photos are taken with logarithmic intervals on the exposure time parameter interval supported by the camera.

例如,布置暗室,杜绝任何光源,任何光源包括可见光和不可见光,比如红外摄像头的红外发射装置,因此需在暗室外控制相机进行拍照。在相机支持的曝光时间参数区间上以对数间断取点,拍摄照片(这些照片主要是数值变化较小的暗电流噪声)。For example, arrange a dark room to avoid any light source, including visible light and invisible light, such as the infrared emitting device of an infrared camera, so it is necessary to control the camera to take pictures outside the dark room. Take points with logarithmic intervals on the exposure time parameter interval supported by the camera, and take photos (these photos are mainly dark current noise with small numerical changes).

步骤A2:在暗室中仅打开仪器光源,在物镜下方使用光陷阱吸收光,以相机曝光时间和光源亮度组合,在光源亮度上均匀取点,且在相机曝光时间上与步骤A1取点相同,拍摄第二照片。Step A2: Only turn on the light source of the instrument in the darkroom, use a light trap to absorb light under the objective lens, and use the combination of camera exposure time and light source brightness to uniformly take points on the light source brightness, and the camera exposure time is the same as the point taken in step A1. Take a second photo.

可以理解的是,如图2所示,在暗室中仅打开仪器光源,在物镜下方使用光陷阱吸收光,以相机曝光时间和光源亮度组合,在光源亮度上均匀取点,在相机曝光时间上与A1取点相同,拍摄照片。It can be understood that, as shown in Figure 2, only the instrument light source is turned on in the darkroom, light traps are used under the objective lens to absorb light, and the exposure time of the camera is combined with the brightness of the light source to uniformly take points on the brightness of the light source, while the camera exposure time Take the same point as A1, take a photo.

具体而言,在暗室中仅打开仪器光源,在物镜下方使用光陷阱吸收射出来的光。该仪器中使用了二向色镜和数十种透镜,因此杂散光主要来自于光源而不是样本反射的回去的光。以相机曝光时间和光源亮度组合,在光源亮度上均匀取点,在相机曝光时间上与A1取点相同,拍摄照片(这些照片既包含杂散光又包含如A1的暗电流噪声)。Specifically, only the instrument light source is turned on in a darkroom, and light traps are used below the objective lens to absorb the emitted light. Dichroic mirrors and dozens of lenses are used in this instrument, so stray light comes mainly from the light source rather than light reflected back from the sample. Combining the exposure time of the camera and the brightness of the light source, the brightness of the light source is evenly selected, and the exposure time of the camera is the same as that of A1, and the photos are taken (these photos contain both stray light and dark current noise like A1).

步骤A3:将第二照片减去第一照片以消除相机本身像感器对实验结果的干扰,进而获取杂散光在相机曝光时间和光源亮度上的变化规律。Step A3: Subtract the first photo from the second photo to eliminate the interference of the camera's own image sensor on the experimental results, and then obtain the change law of stray light in the exposure time of the camera and the brightness of the light source.

可以理解的是,如图2所示,用步骤A2照片减去步骤A1照片可以消除相机本身像感器对实验结果的干扰,进而分析杂散光在相机曝光时间和光源亮度上的变化规律。It can be understood that, as shown in Figure 2, subtracting the photo in step A1 from the photo in step A2 can eliminate the interference of the camera's own image sensor on the experimental results, and then analyze the change law of stray light in the exposure time of the camera and the brightness of the light source.

进一步地,在本发明的一个实施例中,步骤A3,进一步包括:对第一照片和第二照片进行中值滤波,并取每张图片的中间二分之一大小的平均灰度值作为照片的灰度值,分别在只考虑相机曝光时间和光源亮度的情况下对一组照片进行线性拟合,以根据两组拟合系数得到平均灰度值关于相机曝光时间和光源亮度的二元一次函数。Further, in one embodiment of the present invention, step A3 further includes: performing median filtering on the first photo and the second photo, and taking the average gray value of the middle half of each picture as the photo The gray value of the gray value, respectively, in the case of only considering the exposure time of the camera and the brightness of the light source, a linear fitting is performed on a group of photos, so as to obtain the binary primary value of the average gray value with respect to the exposure time of the camera and the brightness of the light source according to the two sets of fitting coefficients function.

可以理解的是,如图2所示,步骤A3中对A2-A1后的照片首先进行中值滤波(相机在高曝光条件下拍摄的照片会有阶跃噪点),再取每张图片的中间二分之一大小的平均灰度值作为该照片的灰度值,分别在只考虑相机曝光时间和光源亮度的情况下对一组照片进行线性拟合,最后根据两组拟合系数可以得到一个平均灰度值关于相机曝光时间和光源亮度的二元一次函数。It is understandable that, as shown in Figure 2, in step A3, the photos after A2-A1 are firstly subjected to median filtering (the photos taken by the camera under high exposure conditions will have step noise), and then the middle of each picture is taken. The average gray value of half the size is used as the gray value of the photo, and a group of photos are linearly fitted in the case of only considering the camera exposure time and the brightness of the light source, and finally according to two sets of fitting coefficients, a Binary linear function of average gray value with respect to camera exposure time and light source brightness.

具体而言,相机在高曝光条件下拍摄的照片会有阶跃噪点,即改点的灰度值远比四周的点大得多。因此首先进行中值滤波可以得到非常好的效果。可以认为相机像感器中间位置基本能接受到外界射进来的光线,故取处理后的每张图片的中间二分之一大小的平均灰度值作为该照片的灰度值。分别在只考虑相机曝光时间和光源亮度的情况下对一组照片进行线性拟合,最后根据多组拟合系数可以得到一个平均灰度值关于相机曝光时间和光源亮度的二元一次函数。Specifically, the photos taken by the camera under high exposure conditions will have step noise, that is, the gray value of the changed point is much larger than that of the surrounding points. Therefore, performing median filtering first can get very good results. It can be considered that the middle position of the camera image sensor can basically receive the light from the outside world, so the average gray value of the middle half of each processed picture is taken as the gray value of the photo. A group of photos are linearly fitted while only considering the exposure time of the camera and the brightness of the light source. Finally, a binary linear function of the average gray value with respect to the exposure time of the camera and the brightness of the light source can be obtained according to multiple sets of fitting coefficients.

步骤A4:在对特定的一张图分析杂散光强度在空间上的优化分布模型时,根据变化规律得到采集照片中每个像素的杂散光数值,以去除杂散光的影响。Step A4: When analyzing the optimal distribution model of stray light intensity in space for a specific picture, the stray light value of each pixel in the collected photo is obtained according to the variation law, so as to remove the influence of stray light.

可以理解的是,如图2所示,本发明实施例可以针对特定的一张图分析杂散光强度在空间上的优化分布模型,结合步骤A3的杂散光与相机曝光时间、光源亮度的变化规律可以得到采集照片中每个像素的杂散光数值,进而可以精确并且实时地去除杂散光的影响。It can be understood that, as shown in FIG. 2, the embodiment of the present invention can analyze the optimal distribution model of stray light intensity in space for a specific picture, and combine the stray light in step A3 with the change law of camera exposure time and light source brightness. The stray light value of each pixel in the collected photos can be obtained, and then the influence of stray light can be removed accurately and in real time.

进一步地,在本发明的一个实施例中,步骤A4,进一步包括:步骤1:对照片的每一行单独取出以进行分段线性拟合,并采用阈值的方法分为三段,阈值为照片的平均灰度值乘以修正系数,通过穷尽搜索的方法得到峰值信噪比的局部极大值,对每段进行多项式拟合;步骤2:对照片的每一列单独取出,并重复步骤1的操作;步骤3:取两次模型的均值作为最终的杂散光强度在空间上的优化分布模型,并结合变化规律得到杂散光数值。Further, in one embodiment of the present invention, step A4 further includes: Step 1: Separately extract each line of the photo for piecewise linear fitting, and divide it into three segments by using a threshold method, where the threshold value is The average gray value is multiplied by the correction coefficient, and the local maximum value of the peak signal-to-noise ratio is obtained by exhaustive search, and polynomial fitting is performed on each segment; Step 2: Take out each column of the photo separately, and repeat the operation of Step 1 ; Step 3: Take the mean value of the two models as the final optimal distribution model of stray light intensity in space, and combine the change rule to obtain the stray light value.

可选地,在本发明的一个实施例中,将一元高次多项式转化为多元线性函数,并使用最小二乘法精确拟合多项式系数,实现多项式拟合。Optionally, in one embodiment of the present invention, the polynomial of higher degree in one variable is transformed into a multivariate linear function, and the least square method is used to accurately fit the coefficients of the polynomial to realize polynomial fitting.

可以理解的是,步骤A4中不是直接找到一个符合模型的二元函数,而是按行按列分别进行拟合,分为三步。步骤1:把照片的每一行单独取出,进行分段线性拟合,采用阈值的方法分为三段,该阈值为照片的平均灰度值乘以一个修正系数,通过穷尽搜索的方法找到最佳拟合效果,即PSNR(峰值信噪比)的局部极大值,对每段进行多项式拟合,这种拟合实质上是一个多元线性回归问题,假设Y=b0+b1X1+b2X2+b3X3+b4X4+…其中X1=x,X2=x2,X3=x3,X4=x4…Xn=xn这样就把一元高次多项式转化为了多元线性函数,可以使用最小二乘法精确拟合多项式系数。步骤2:把照片的每一列单独取出,重复步骤1的操作。步骤3:为了尽可能的保证拟合模型的连续性,取两次模型的均值作为最终的杂散光强度在空间上的优化分布模型。结合A3的杂散光与相机曝光时间、光源亮度的变化规律可以得到采集照片中每个像素的杂散光数值,进而可以精确并且实时地去除杂散光的影响。It can be understood that in step A4, instead of directly finding a binary function that fits the model, the fitting is performed by row and column respectively, which is divided into three steps. Step 1: Take out each line of the photo separately, perform piecewise linear fitting, and divide it into three sections by thresholding. The threshold is the average gray value of the photo multiplied by a correction coefficient, and find the best one by exhaustive search. The fitting effect, that is, the local maximum value of PSNR (peak signal-to-noise ratio), performs polynomial fitting on each segment. This fitting is essentially a multiple linear regression problem, assuming Y=b 0 +b 1 X 1 + b 2 X 2 +b 3 X 3 +b 4 X 4 +… where X 1 =x, X 2 =x 2 , X 3 =x 3 , X 4 =x 4 ...X n =x n Degree polynomials are transformed into multivariate linear functions, and polynomial coefficients can be accurately fitted using the least squares method. Step 2: Take out each column of the photo separately, and repeat the operation of step 1. Step 3: In order to ensure the continuity of the fitting model as much as possible, take the mean value of the two models as the final optimized distribution model of stray light intensity in space. Combining the stray light of A3 with the camera exposure time and the change law of light source brightness, the stray light value of each pixel in the collected photos can be obtained, and then the influence of stray light can be removed accurately and in real time.

可选地,在本发明的一个实施例中,峰值信噪比的计算公式为:Optionally, in one embodiment of the present invention, the formula for calculating the peak signal-to-noise ratio is:

其中,MSE是模型与原图的均方误差,n是图像的位数。Among them, MSE is the mean square error between the model and the original image, and n is the number of bits of the image.

具体而言,针对特定的一张图分析杂散光强度在空间上的优化分布模型。通常相机为了保证能够100%采集进入相机的光,像感器都会做的比实际入射光面积更大一些,因此实际拍摄的每张照片边缘部分的灰度值会有一个剧烈下降。想要直接找到一个契合的二元函数模型非常困难,步骤1:把照片的每一行单独取出,进行分段线性拟合,采用阈值的方法分为三段,该阈值为照片的平均灰度值乘以一个修正系数ξ,可以在实验中调节修正系数来达到最佳拟合效果,对每段进行多项式拟合,这种拟合实质上是一个多元线性回归问题,Specifically, the optimal distribution model of stray light intensity in space is analyzed for a specific image. Generally, in order to ensure that the camera can capture 100% of the light entering the camera, the image sensor will be made larger than the actual incident light area, so the gray value of the edge of each photo actually taken will drop sharply. It is very difficult to directly find a suitable binary function model. Step 1: Take out each row of the photo separately, perform piecewise linear fitting, and divide it into three sections by thresholding. The threshold is the average gray value of the photo Multiplied by a correction coefficient ξ, the correction coefficient can be adjusted in the experiment to achieve the best fitting effect, and polynomial fitting is performed on each segment. This fitting is essentially a multiple linear regression problem.

假设Y=b0+b1X1+b2X2+b3X3+b4X4+…其中X1=x,X2=x2,X3=x3,X4=x4…Xn=xn这样就把一元高次多项式转化为了多元线性函数,可以使用最小二乘法精确拟合多项式系数。步骤2:把照片的每一列单独取出,重复步骤1的操作。步骤3:为了尽可能的保证拟合模型的连续性,取两次模型的均值作为最终的杂散光强度在空间上的优化分布模型,计算模型的PSNR,通过穷尽搜索的方法找到局部最优解ξ使生成模型的PSNR最大。结合A3的杂散光与相机曝光时间、光源亮度的变化规律可以得到采集照片中每个像素的杂散光数值,进而可以精确并且实时地去除杂散光的影响。Suppose Y=b 0 +b 1 X 1 +b 2 X 2 +b 3 X 3 +b 4 X 4 +... where X 1 =x, X 2 =x 2 , X 3 =x 3 , X 4 =x 4 ...X n =x n In this way, the one-variable high-degree polynomial is converted into a multivariate linear function, and the least square method can be used to accurately fit the polynomial coefficients. Step 2: Take out each column of the photo separately, and repeat the operation of step 1. Step 3: In order to ensure the continuity of the fitting model as much as possible, take the mean value of the two models as the final optimal distribution model of stray light intensity in space, calculate the PSNR of the model, and find the local optimal solution by exhaustive search ξ maximizes the PSNR of the generative model. Combining the stray light of A3 with the camera exposure time and the change law of light source brightness, the stray light value of each pixel in the collected photos can be obtained, and then the influence of stray light can be removed accurately and in real time.

其中MSE是模型与原图的均方误差,n是图像的位数,本发明使用的图像是16位。Wherein MSE is the mean square error of the model and the original image, n is the number of digits of the image, and the image used in the present invention is 16 bits.

综上,本发明实施例可以有效降低成像过程中到达像感器表面的非成像光线(指杂散光)对最终成像的影响,大幅提高了图像的清晰度,不仅可以提高信息的有效性,而且可以提高后续图像处理算法的精度。本发明应用于多维多尺度高分辨率计算摄像仪器(可以理解为一台巨大的显微镜),该仪器的光源、物镜、相机阵列全都是固定的,可以认为光路在长时间内是固定不变的。因此在特定情况下杂散光在像感器表面的空间分布和强度也是固定不变的。事先采集在不同光源亮度和不同相机曝光时间下的杂散光数据,对其进行分析,建立杂散光的空间分布模型,找到杂散光强度与光源亮度和相机曝光时间的关系,就可以实现根据拍摄照片时的相关参数,实时生成对应的杂散光空间分布模型,在图像存储之前就去除杂散光的影响,提高了实验效率,有效提高了图片清晰度。To sum up, the embodiment of the present invention can effectively reduce the influence of non-imaging light (referred to as stray light) reaching the surface of the image sensor during the imaging process on the final imaging, greatly improving the clarity of the image, not only improving the effectiveness of information, but also The accuracy of subsequent image processing algorithms can be improved. The present invention is applied to a multi-dimensional, multi-scale and high-resolution computing imaging instrument (which can be understood as a huge microscope). The light source, objective lens, and camera array of the instrument are all fixed, and it can be considered that the optical path is fixed for a long time . Therefore, the spatial distribution and intensity of stray light on the surface of the image sensor is also fixed under certain conditions. Collect stray light data under different light source brightness and different camera exposure time in advance, analyze it, establish the spatial distribution model of stray light, find the relationship between stray light intensity, light source brightness and camera exposure time, and then realize the The relevant parameters of time can be used to generate the corresponding stray light spatial distribution model in real time, and the influence of stray light can be removed before image storage, which improves the experimental efficiency and effectively improves the picture clarity.

另外,本发明实施例创新性地提出先行找出仪器的光电特性,在很大程度上减少了算法的时间复杂度,从而可以实现实时消除高清显微相机拍摄图像的杂散光。In addition, the embodiment of the present invention innovatively proposes to find out the photoelectric characteristics of the instrument in advance, which greatly reduces the time complexity of the algorithm, so that the stray light of the image captured by the high-definition microscope camera can be eliminated in real time.

根据本发明实施例提出的基于光电特性先验的显微成像的杂散光去除方法,针对成像仪器的光路在一段时间内空间位置不变的性质,事先采集在不同光源亮度和不同相机曝光时间下的杂散光数据,进行分析,将复杂的建模计算放在平时空余时间内完成,降低了实时算法的时间复杂度,可以实现根据拍摄照片时的相关参数,实时生成对应的杂散光空间分布模型,在图像存储之前就去除杂散光的影响,从而减少了算法的时间复杂度,不仅有效提高了实验效率,而且有效提高了图片清晰度。According to the stray light removal method of microscopic imaging based on photoelectric characteristics prior proposed by the embodiment of the present invention, aiming at the property that the optical path of the imaging instrument does not change in spatial position within a period of time, it is collected in advance under different light source brightness and different camera exposure time Analyze the stray light data, and complete the complex modeling calculation in the usual spare time, which reduces the time complexity of the real-time algorithm, and can generate the corresponding stray light spatial distribution model in real time according to the relevant parameters when taking photos , the influence of stray light is removed before the image is stored, thereby reducing the time complexity of the algorithm, not only effectively improving the efficiency of the experiment, but also effectively improving the clarity of the image.

其次参照附图描述根据本发明实施例提出的基于光电特性先验的显微成像的杂散光去除装置。Next, a stray light removal device based on photoelectric property prior microscopic imaging proposed according to an embodiment of the present invention will be described with reference to the accompanying drawings.

图3是本发明一个实施例的基于光电特性先验的显微成像的杂散光去除装置的结构示意图。FIG. 3 is a schematic structural diagram of a stray light removal device based on photoelectric characteristic prior microscopic imaging according to an embodiment of the present invention.

如图3所示,该基于光电特性先验的显微成像的杂散光去除装置10包括:第一拍摄模块100、第二拍摄模块200、消除模块300和处理模块400。As shown in FIG. 3 , the stray light removal device 10 based on photoelectric characteristic prior microscopic imaging includes: a first photographing module 100 , a second photographing module 200 , an elimination module 300 and a processing module 400 .

其中,第一拍摄模块100用于布置暗室并在相机支持的曝光时间参数区间上以对数间断取点,拍摄第一照片。第二拍摄模块200用于在暗室中仅打开仪器光源,在物镜下方使用光陷阱吸收光,以相机曝光时间和光源亮度组合,在光源亮度上均匀取点,且在相机曝光时间上与步骤A1取点相同,拍摄第二照片。消除模块300用于将第二照片减去第一照片以消除相机本身像感器对实验结果的干扰,进而获取杂散光在相机曝光时间和光源亮度上的变化规律。处理模块400用于在对特定的一张图分析杂散光强度在空间上的优化分布模型时,根据变化规律得到采集照片中每个像素的杂散光数值,以去除杂散光的影响。本发明实施例的装置10实时生成对应的杂散光空间分布模型,在图像存储之前就去除杂散光的影响,从而减少了算法的时间复杂度,不仅有效提高了实验效率,而且有效提高了图片清晰度。Wherein, the first photographing module 100 is used for arranging a dark room and taking points at logarithmic intervals on the exposure time parameter interval supported by the camera, and taking the first photograph. The second photographing module 200 is used to turn on only the light source of the instrument in the darkroom, use light traps under the objective lens to absorb light, combine the exposure time of the camera with the brightness of the light source, uniformly take points on the brightness of the light source, and use the exposure time of the camera as in step A1 Take the same point and take a second photo. The elimination module 300 is used to subtract the first photo from the second photo to eliminate the interference of the camera's own image sensor on the experimental results, and then obtain the change law of stray light on the exposure time of the camera and the brightness of the light source. The processing module 400 is used to obtain the stray light value of each pixel in the captured photo according to the change law when analyzing the optimal distribution model of stray light intensity in space for a specific image, so as to remove the influence of stray light. The device 10 of the embodiment of the present invention generates the corresponding stray light spatial distribution model in real time, and removes the influence of stray light before image storage, thereby reducing the time complexity of the algorithm, not only effectively improving the experimental efficiency, but also effectively improving the clarity of the picture Spend.

进一步地,在本发明的一个实施例中,消除模块300还用于对第一照片和第二照片进行中值滤波,并取每张图片的中间二分之一大小的平均灰度值作为照片的灰度值,分别在只考虑相机曝光时间和光源亮度的情况下对一组照片进行线性拟合,以根据两组拟合系数得到平均灰度值关于相机曝光时间和光源亮度的二元一次函数。Further, in one embodiment of the present invention, the elimination module 300 is also used to perform median filtering on the first photo and the second photo, and take the average gray value of the middle half of each picture as the photo The gray value of the gray value, respectively, in the case of only considering the exposure time of the camera and the brightness of the light source, a linear fitting is performed on a group of photos, so as to obtain the binary primary value of the average gray value with respect to the exposure time of the camera and the brightness of the light source according to the two sets of fitting coefficients function.

进一步地,在本发明的一个实施例中,处理模块400还用于对照片的每一行单独取出以进行分段线性拟合,并采用阈值的方法分为三段,阈值为照片的平均灰度值乘以修正系数,通过穷尽搜索的方法得到峰值信噪比的局部极大值,对每段进行多项式拟合,并对照片的每一列单独取出,并重复上述的操作,以及取两次模型的均值作为最终的杂散光强度在空间上的优化分布模型,并结合变化规律得到杂散光数值。Further, in one embodiment of the present invention, the processing module 400 is also used to separately extract each line of the photo for piecewise linear fitting, and divide it into three segments by using a threshold method, where the threshold is the average gray level of the photo Value multiplied by the correction coefficient, the local maximum value of peak signal-to-noise ratio is obtained by exhaustive search method, polynomial fitting is performed on each segment, and each column of the photo is taken out separately, and the above operation is repeated, and the model is taken twice The mean value of the stray light intensity is used as the optimal distribution model of the final stray light intensity in space, and the stray light value is obtained by combining the change rule.

进一步地,在本发明的一个实施例中,将一元高次多项式转化为多元线性函数,并使用最小二乘法精确拟合多项式系数,实现多项式拟合。Furthermore, in one embodiment of the present invention, the polynomial of high degree in one variable is transformed into a multivariate linear function, and the least square method is used to accurately fit the coefficients of the polynomial to realize polynomial fitting.

进一步地,在本发明的一个实施例中,峰值信噪比的计算公式为:Further, in one embodiment of the present invention, the calculation formula of peak signal-to-noise ratio is:

其中,MSE是模型与原图的均方误差,n是图像的位数。Among them, MSE is the mean square error between the model and the original image, and n is the number of bits of the image.

需要说明的是,前述对基于光电特性先验的显微成像的杂散光去除方法实施例的解释说明也适用于该实施例的基于光电特性先验的显微成像的杂散光去除装置,此处不再赘述。It should be noted that the foregoing explanations on the embodiment of the stray light removal method based on photoelectric characteristic prior microscopic imaging are also applicable to the stray light removal device based on photoelectric characteristic prior microscopic imaging of this embodiment, here No longer.

根据本发明实施例提出的基于光电特性先验的显微成像的杂散光去除装置,针对成像仪器的光路在一段时间内空间位置不变的性质,事先采集在不同光源亮度和不同相机曝光时间下的杂散光数据,进行分析,将复杂的建模计算放在平时空余时间内完成,降低了实时算法的时间复杂度,可以实现根据拍摄照片时的相关参数,实时生成对应的杂散光空间分布模型,在图像存储之前就去除杂散光的影响,从而减少了算法的时间复杂度,不仅有效提高了实验效率,而且有效提高了图片清晰度。According to the stray light removal device for microscopic imaging based on photoelectric characteristics prior proposed by the embodiment of the present invention, aiming at the property that the optical path of the imaging instrument does not change in spatial position within a period of time, it is collected in advance under different light source brightness and different camera exposure time Analyze the stray light data, and complete the complex modeling calculation in the usual spare time, which reduces the time complexity of the real-time algorithm, and can generate the corresponding stray light spatial distribution model in real time according to the relevant parameters when taking photos , the influence of stray light is removed before the image is stored, thereby reducing the time complexity of the algorithm, not only effectively improving the efficiency of the experiment, but also effectively improving the clarity of the image.

在本发明的描述中,需要理解的是,术语“中心”、“纵向”、“横向”、“长度”、“宽度”、“厚度”、“上”、“下”、“前”、“后”、“左”、“右”、“竖直”、“水平”、“顶”、“底”“内”、“外”、“顺时针”、“逆时针”、“轴向”、“径向”、“周向”等指示的方位或位置关系为基于附图所示的方位或位置关系,仅是为了便于描述本发明和简化描述,而不是指示或暗示所指的装置或元件必须具有特定的方位、以特定的方位构造和操作,因此不能理解为对本发明的限制。In describing the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", " Back", "Left", "Right", "Vertical", "Horizontal", "Top", "Bottom", "Inner", "Outer", "Clockwise", "Counterclockwise", "Axial", The orientation or positional relationship indicated by "radial", "circumferential", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying the referred device or element Must be in a particular orientation, be constructed in a particular orientation, and operate in a particular orientation, and therefore should not be construed as limiting the invention.

此外,术语“第一”、“第二”仅用于描述目的,而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此,限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。在本发明的描述中,“多个”的含义是至少两个,例如两个,三个等,除非另有明确具体的限定。In addition, the terms "first" and "second" are used for descriptive purposes only, and cannot be interpreted as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

在本发明中,除非另有明确的规定和限定,术语“安装”、“相连”、“连接”、“固定”等术语应做广义理解,例如,可以是固定连接,也可以是可拆卸连接,或成一体;可以是机械连接,也可以是电连接;可以是直接相连,也可以通过中间媒介间接相连,可以是两个元件内部的连通或两个元件的相互作用关系,除非另有明确的限定。对于本领域的普通技术人员而言,可以根据具体情况理解上述术语在本发明中的具体含义。In the present invention, unless otherwise clearly specified and limited, terms such as "installation", "connection", "connection" and "fixation" should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection , or integrated; it may be mechanically connected or electrically connected; it may be directly connected or indirectly connected through an intermediary, and it may be the internal communication of two components or the interaction relationship between two components, unless otherwise specified limit. Those of ordinary skill in the art can understand the specific meanings of the above terms in the present invention according to specific situations.

在本发明中,除非另有明确的规定和限定,第一特征在第二特征“上”或“下”可以是第一和第二特征直接接触,或第一和第二特征通过中间媒介间接接触。而且,第一特征在第二特征“之上”、“上方”和“上面”可是第一特征在第二特征正上方或斜上方,或仅仅表示第一特征水平高度高于第二特征。第一特征在第二特征“之下”、“下方”和“下面”可以是第一特征在第二特征正下方或斜下方,或仅仅表示第一特征水平高度小于第二特征。In the present invention, unless otherwise clearly specified and limited, the first feature may be in direct contact with the first feature or the first and second feature may be in direct contact with the second feature through an intermediary. touch. Moreover, "above", "above" and "above" the first feature on the second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. "Below", "beneath" and "beneath" the first feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is less horizontally than the second feature.

在本说明书的描述中,参考术语“一个实施例”、“一些实施例”、“示例”、“具体示例”、或“一些示例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或者特点包含于本发明的至少一个实施例或示例中。在本说明书中,对上述术语的示意性表述不必须针对的是相同的实施例或示例。而且,描述的具体特征、结构、材料或者特点可以在任一个或多个实施例或示例中以合适的方式结合。此外,在不相互矛盾的情况下,本领域的技术人员可以将本说明书中描述的不同实施例或示例以及不同实施例或示例的特征进行结合和组合。In the description of this specification, descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific examples", or "some examples" mean that specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the described specific features, structures, materials or characteristics may be combined in any suitable manner in any one or more embodiments or examples. In addition, those skilled in the art can combine and combine different embodiments or examples and features of different embodiments or examples described in this specification without conflicting with each other.

尽管上面已经示出和描述了本发明的实施例,可以理解的是,上述实施例是示例性的,不能理解为对本发明的限制,本领域的普通技术人员在本发明的范围内可以对上述实施例进行变化、修改、替换和变型。Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and those skilled in the art can make the above-mentioned The embodiments are subject to changes, modifications, substitutions and variations.

Claims (6)

1.一种基于光电特性先验的显微成像的杂散光去除方法,其特征在于,包括以下步骤:1. A stray light removal method based on photoelectric characteristic priori microscopic imaging, is characterized in that, comprises the following steps: 步骤A1:布置暗室并在相机支持的曝光时间参数区间上以对数间断取点,拍摄第一照片;Step A1: Arrange the darkroom and take the logarithmically intermittent points on the exposure time parameter interval supported by the camera, and take the first photo; 步骤A2:在所述暗室中仅打开仪器光源,在物镜下方使用光陷阱吸收光,以相机曝光时间和光源亮度组合,在所述光源亮度上均匀取点,且在所述相机曝光时间上与所述步骤A1取点位置相同,拍摄第二照片;Step A2: Turn on only the light source of the instrument in the darkroom, use a light trap to absorb light under the objective lens, combine the exposure time of the camera with the brightness of the light source, take points evenly on the brightness of the light source, and compare the exposure time of the camera with the brightness of the light source. The step A1 takes the same point position, and takes the second photo; 步骤A3:将所述第二照片减去所述第一照片以消除相机本身像感器对实验结果的干扰,进而获取杂散光在所述相机曝光时间和所述光源亮度上的变化规律,其中,所述步骤A3,进一步包括:对所述第一照片和所述第二照片进行中值滤波,并取每张图片的中间二分之一大小的平均灰度值作为照片的灰度值,分别在只考虑所述相机曝光时间和所述光源亮度的情况下对一组照片进行线性拟合,以根据两组拟合系数得到平均灰度值关于所述相机曝光时间和所述光源亮度的二元一次函数;以及Step A3: Subtract the first photo from the second photo to eliminate the interference of the camera's own image sensor on the experimental results, and then obtain the change law of stray light on the exposure time of the camera and the brightness of the light source, wherein , the step A3 further includes: performing median filtering on the first photo and the second photo, and taking the average gray value of the middle half of each picture as the gray value of the photo, Carry out linear fitting to a group of photos under the condition of only considering the exposure time of the camera and the brightness of the light source, so as to obtain the average gray value with respect to the exposure time of the camera and the brightness of the light source according to two sets of fitting coefficients. Binary linear functions; and 步骤A4:在对特定的一张图分析杂散光强度在空间上的优化分布模型时,根据所述变化规律得到采集照片中每个像素的杂散光数值,以去除杂散光的影响,其中,所述步骤A4,进一步包括:步骤1:对照片的每一行单独取出以进行分段线性拟合,并采用阈值的方法分为三段,阈值为照片的灰度值乘以修正系数,通过穷尽搜索的方法得到峰值信噪比的局部极大值,对每段进行多项式拟合;步骤2:对所述照片的每一列单独取出,并重复所述步骤1的操作;步骤3:取两次模型的均值作为最终的杂散光强度在空间上的优化分布模型,并结合所述变化规律得到所述杂散光数值。Step A4: When analyzing the optimal distribution model of stray light intensity in space for a specific picture, the stray light value of each pixel in the collected photo is obtained according to the change rule, so as to remove the influence of stray light, wherein, the Said step A4, further comprising: Step 1: each row of the photo is taken out separately to carry out piecewise linear fitting, and adopts the method of threshold value to be divided into three sections, and the threshold value is the gray value of the photo multiplied by the correction coefficient, and through exhaustive search The method obtains the local maximum value of peak signal-to-noise ratio, and performs polynomial fitting on each segment; Step 2: Take out each column of the photo separately, and repeat the operation of Step 1; Step 3: Take the model twice The mean value of is used as the optimal distribution model of the final stray light intensity in space, and the stray light value is obtained in combination with the change rule. 2.根据权利要求1所述的基于光电特性先验的显微成像的杂散光去除方法,其特征在于,将一元高次多项式转化为多元线性函数,并使用最小二乘法精确拟合多项式系数,实现多项式拟合。2. the method for removing stray light based on photoelectric characteristic prior microscopic imaging according to claim 1, is characterized in that, one yuan high degree polynomial is converted into multivariate linear function, and uses least square method to accurately fit polynomial coefficient, Implements polynomial fitting. 3.根据权利要求1或2所述的基于光电特性先验的显微成像的杂散光去除方法,其特征在于,所述峰值信噪比的计算公式为:3. The method for removing stray light based on photoelectric characteristic prior microscopic imaging according to claim 1 or 2, wherein the calculation formula of the peak signal-to-noise ratio is: 其中,MSE是模型与原图的均方误差,n是图像的位数。Among them, MSE is the mean square error between the model and the original image, and n is the number of bits of the image. 4.一种基于光电特性先验的显微成像的杂散光去除装置,其特征在于,包括:4. A stray light removal device based on photoelectric characteristic prior microscopic imaging, characterized in that, comprising: 第一拍摄模块,用于布置暗室并在相机支持的曝光时间参数区间上以对数间断取点,拍摄第一照片;The first shooting module is used to arrange the darkroom and take the logarithmically intermittent points on the exposure time parameter interval supported by the camera to take the first photo; 第二拍摄模块,用于在所述暗室中仅打开仪器光源,在物镜下方使用光陷阱吸收光,以相机曝光时间和光源亮度组合,在所述光源亮度上均匀取点,且在所述相机曝光时间上与所述第一拍摄模块取点位置相同,拍摄第二照片;The second shooting module is used to turn on only the light source of the instrument in the dark room, use a light trap to absorb light under the objective lens, combine the exposure time of the camera with the brightness of the light source, uniformly take points on the brightness of the light source, and The exposure time is the same as that of the first shooting module, and the second photo is taken; 消除模块,用于将所述第二照片减去所述第一照片以消除相机本身像感器对实验结果的干扰,进而获取杂散光在所述相机曝光时间和所述光源亮度上的变化规律,其中,所述消除模块还用于对所述第一照片和所述第二照片进行中值滤波,并取每张图片的中间二分之一大小的平均灰度值作为照片的灰度值,分别在只考虑所述相机曝光时间和所述光源亮度的情况下对一组照片进行线性拟合,以根据两组拟合系数得到平均灰度值关于所述相机曝光时间和所述光源亮度的二元一次函数;以及The elimination module is used to subtract the first photo from the second photo to eliminate the interference of the image sensor of the camera itself on the experimental results, and then obtain the change law of stray light on the exposure time of the camera and the brightness of the light source , wherein the elimination module is also used to perform median filtering on the first photo and the second photo, and take the average gray value of the middle half of each picture as the gray value of the photo , linearly fitting a group of photos under the condition of only considering the camera exposure time and the light source brightness respectively, so as to obtain the average gray value with respect to the camera exposure time and the light source brightness according to two sets of fitting coefficients binary linear function of ; and 处理模块,用于在对特定的一张图分析杂散光强度在空间上的优化分布模型时,根据所述变化规律得到采集照片中每个像素的杂散光数值,以去除杂散光的影响,其中,所述处理模块还用于对照片的每一行单独取出以进行分段线性拟合,并采用阈值的方法分为三段,阈值为照片的灰度值乘以修正系数,通过穷尽搜索的方法得到峰值信噪比的局部极大值,对每段进行多项式拟合,并对所述照片的每一列单独取出,并重复上述的操作,以及取两次模型的均值作为最终的杂散光强度在空间上的优化分布模型,并结合所述变化规律得到所述杂散光数值。The processing module is used to obtain the stray light value of each pixel in the collected photo according to the change law when analyzing the optimal distribution model of stray light intensity in space for a specific picture, so as to remove the influence of stray light, wherein , the processing module is also used to separately extract each line of the photo to perform piecewise linear fitting, and divide it into three sections by using a threshold method, the threshold is the gray value of the photo multiplied by the correction coefficient, and through the method of exhaustive search Get the local maximum value of peak signal-to-noise ratio, perform polynomial fitting on each segment, and take out each column of the photo separately, and repeat the above operation, and take the mean value of the two models as the final stray light intensity in The distribution model is optimized in space, and the value of stray light is obtained in combination with the change rule. 5.根据权利要求4所述的基于光电特性先验的显微成像的杂散光去除装置,其特征在于,将一元高次多项式转化为多元线性函数,并使用最小二乘法精确拟合多项式系数,实现多项式拟合。5. The stray light removing device based on photoelectric characteristic prior microscopic imaging according to claim 4, is characterized in that, the one-dimensional high-order polynomial is converted into a multivariate linear function, and the polynomial coefficients are accurately fitted using the least squares method, Implements polynomial fitting. 6.根据权利要求4或5所述的基于光电特性先验的显微成像的杂散光去除装置,其特征在于,所述峰值信噪比的计算公式为:6. The stray light removal device based on photoelectric characteristic prior microscopic imaging according to claim 4 or 5, wherein the calculation formula of the peak signal-to-noise ratio is: 其中,MSE是模型与原图的均方误差,n是图像的位数。Among them, MSE is the mean square error between the model and the original image, and n is the number of bits of the image.
CN201711373195.4A 2017-12-19 2017-12-19 Method and device for removing stray light in microscopic imaging based on prior photoelectric characteristics Active CN108120680B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201711373195.4A CN108120680B (en) 2017-12-19 2017-12-19 Method and device for removing stray light in microscopic imaging based on prior photoelectric characteristics

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201711373195.4A CN108120680B (en) 2017-12-19 2017-12-19 Method and device for removing stray light in microscopic imaging based on prior photoelectric characteristics

Publications (2)

Publication Number Publication Date
CN108120680A CN108120680A (en) 2018-06-05
CN108120680B true CN108120680B (en) 2019-11-22

Family

ID=62229300

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201711373195.4A Active CN108120680B (en) 2017-12-19 2017-12-19 Method and device for removing stray light in microscopic imaging based on prior photoelectric characteristics

Country Status (1)

Country Link
CN (1) CN108120680B (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109872315B (en) * 2019-02-20 2021-04-20 中国科学院国家天文台 Method for detecting stray light uniformity of optical astronomical telescope in real time
CN110415226A (en) * 2019-07-23 2019-11-05 Oppo广东移动通信有限公司 Stray light measurement method, device, electronic equipment and storage medium
CN112067254B (en) * 2020-08-13 2021-12-28 广东弘景光电科技股份有限公司 A kind of optical system ghost image measurement method and system
CN113824875B (en) * 2021-08-11 2023-05-09 北京京仪仪器仪表研究总院有限公司 Fruit and vegetable shooting method based on modulated light source brightness interval and shooting interval
CN113686878B (en) * 2021-09-03 2024-02-09 太原理工大学 Multi-stage joint detection method and system for surface defects of special steel bar
CN113962915B (en) * 2021-10-20 2022-04-29 哈尔滨工业大学 Adaptive Nonlinear Hyperdynamic Image Synthesis Method under Non-Uniform Illumination
CN114972101B (en) * 2022-06-02 2025-06-13 上海圭目机器人有限公司 A jitter elimination method for road surface depth images taken by line laser 3D camera
CN118329391B (en) * 2024-04-16 2025-06-03 中电海康集团有限公司 AR optical waveguide stray light testing method and system
CN118781007B (en) * 2024-06-03 2025-04-01 大湾区大学(筹) Image stray light removal method, device, equipment and storage medium

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103679653A (en) * 2013-11-29 2014-03-26 北京空间机电研究所 System and method for eliminating stray light of satellite images
WO2014186544A1 (en) * 2013-05-15 2014-11-20 The Administrators Of The Tulane Educational Fund Microscopy of a tissue sample using structured illumination
CN104200430A (en) * 2014-08-20 2014-12-10 北京空间机电研究所 Image region stray light eliminating device based on moment matching and method thereof
CN105812661A (en) * 2016-03-16 2016-07-27 浙江大学 Digital camera uniformity correction method based on standard light box and gray card
EP2815251B1 (en) * 2012-02-15 2017-03-22 Heptagon Micro Optics Pte. Ltd. Time of flight camera with stripe illumination

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2815251B1 (en) * 2012-02-15 2017-03-22 Heptagon Micro Optics Pte. Ltd. Time of flight camera with stripe illumination
WO2014186544A1 (en) * 2013-05-15 2014-11-20 The Administrators Of The Tulane Educational Fund Microscopy of a tissue sample using structured illumination
CN103679653A (en) * 2013-11-29 2014-03-26 北京空间机电研究所 System and method for eliminating stray light of satellite images
CN104200430A (en) * 2014-08-20 2014-12-10 北京空间机电研究所 Image region stray light eliminating device based on moment matching and method thereof
CN105812661A (en) * 2016-03-16 2016-07-27 浙江大学 Digital camera uniformity correction method based on standard light box and gray card

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
Asymmetric Microlens Design with Tilted Elements for Removal of Ghost Images and Stray Light;J. D. Rogers;《OSA/FIO》;20051231;第FWU3页 *
Blind deconvolution subject to sparse representation for fluorescence microscopy;YU Wang等;《Optics Communications》;20120815;第60-68页 *
天基可见光探测相机光学系统设计及其杂散光分析;尚玲;《中国优秀硕士学位论文全文数据库》;20130615;全文 *

Also Published As

Publication number Publication date
CN108120680A (en) 2018-06-05

Similar Documents

Publication Publication Date Title
CN108120680B (en) Method and device for removing stray light in microscopic imaging based on prior photoelectric characteristics
CN108898567B (en) Image noise reduction method, device and system
Chen et al. Reblur2deblur: Deblurring videos via self-supervised learning
CN108012080B (en) Image processing method, image processing device, electronic equipment and computer readable storage medium
CN106846463B (en) Three-dimensional reconstruction method and system of microscopic image based on deep learning neural network
JP5470959B2 (en) Multi-frame reconstruction method, system and software
CN107509031B (en) Image processing method, image processing device, mobile terminal and computer readable storage medium
Raghavendra et al. Comparative evaluation of super-resolution techniques for multi-face recognition using light-field camera
JP6094370B2 (en) Dictionary learning for non-coherent sampling
JP6104010B2 (en) Image processing apparatus, imaging apparatus, image processing method, image processing program, and storage medium
CN106331513B (en) The acquisition methods and system of a kind of high-quality skin image
CN111145134B (en) Block effect-based microlens light field camera full-focus image generation algorithm
CN104410784A (en) Light field collecting control method and light field collecting control device
CN110490196B (en) Subject detection method and device, electronic device, computer-readable storage medium
CN110349163A (en) Image processing method and apparatus, electronic device, computer-readable storage medium
Bauer et al. Automatic estimation of modulation transfer functions
CN110248101A (en) Focusing method and device, electronic equipment and computer readable storage medium
CN112819742B (en) Event field synthetic aperture imaging method based on convolutional neural network
Yang et al. Latency correction for event-guided deblurring and frame interpolation
CN112651911A (en) High dynamic range imaging generation method based on polarization image
CN101726829A (en) Method for automatically focusing zoom lens
CN118333914B (en) High dynamic range light field data white image correction method, system and medium
Li et al. Learning to synthesize photorealistic dual-pixel images from RGBD frames
CN118784807A (en) Projector automatic focusing method, device, projector, equipment and medium
JP2015163915A (en) Image processor, imaging device, image processing method, program, and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant