CN113099209A

CN113099209A - Non-visual field imaging device and method based on photomultiplier tube array

Info

Publication number: CN113099209A
Application number: CN202110312078.7A
Authority: CN
Inventors: 戴琼海; 裴承全; 张安科; 方璐; 索津莉; 乔晖; 吴嘉敏; 袁文帅
Original assignee: Tsinghua University
Current assignee: Tsinghua University
Priority date: 2021-03-24
Filing date: 2021-03-24
Publication date: 2021-07-09

Abstract

The present application proposes a photomultiplier tube array-based non-field of view imaging device and method, which relates to the technical field of computational photography, including: a photomultiplier tube PMT array, a laser, a time-dependent counting device, and a computational reconstruction device; a PMT array The target object is photographed with the time-dependent counting device as a non-visual field imaging acquisition camera; wherein, each PMT is connected to a time-to-digital converter TDC; the laser is used as an excitation source, and the electrical synchronization signal output by the laser is synchronized with the time-dependent counting device; The calculation and reconstruction device obtains the time and the number of photons recorded by the PMT array as the data to be reconstructed, uses an inversion reconstruction algorithm to process the data to be reconstructed, and reconstructs a non-field of view imaging target image. As a result, high-efficiency and high-quality non-field-of-view imaging can be realized.

Description

Non-visual field imaging device and method based on photomultiplier tube array

Technical Field

The application relates to the technical field of computational photography, in particular to a non-visual field imaging device and method based on a photomultiplier tube array.

Background

In recent years, computational photography has become an international leading research hotspot in the fields of cross vision, graphics, photography, signal processing and the like, and how to make full use of computational methods to continuously promote new imaging devices has attracted wide attention. In conventional imaging systems, one point on the object plane corresponds to one point on the image plane, i.e. it is assumed that the speed of propagation of light is infinite. If the propagation velocity of light in the imaging system is limited, the imaging system is no longer point-to-point correspondence, but rather a point of the object plane corresponds to a hyperbola with the image plane. By fully utilizing the principle, a meaningful novel imaging method can be obtained by adopting a calculation imaging technology.

The non-vision field imaging fully utilizes the assumption that the light speed is limited, hidden objects at the corners of urban streets and in houses can be detected by adopting the principle of elliptical tomography, the hidden objects can be imaged by bypassing the corners or barriers, the positioning of the targets in the areas outside the sight line is realized, and the improvement of the imaging performance by fully utilizing the means of calculating the camera has very important significance. The Non-line-of-vision imaging (Non-line-of-vision imaging) technique is the biggest difference from the conventional optical imaging technique in that it can image a hidden object in an area invisible to human eyes. Conventional optical imaging techniques image objects that can be seen through a detector, while non-field-of-view imaging techniques image objects that are specifically intended for hidden objects. The key technology of the technology is that laser is irradiated on a middle interface to perform diffuse reflection, light is transmitted to a hidden object after being subjected to one or more times of diffuse reflection, the flight time and the intensity of different reflected photons are recorded by a detector with ultrahigh time resolution, and the hidden object is finally reconstructed by a reverse time projection algorithm. In recent years, non-visual field imaging technology has been rapidly developed with the continuous maturity of laser imaging technology, semiconductor detector technology, and computational imaging technology.

In the related art, a novel algorithm for understanding the Time image analysis of a scene is realized by using a Time of Flight (TOF) camera and multipath analysis; the reflection characteristic of the window glass is researched, laser is used for active illumination, PMD Tec is used as a receiving detector, the influence of different reflection angles of the window on reflection imaging and transmission images is compared, and the result shows that the higher the incident angle is, the stronger and clearer the reflection image is; processing image information confused by diffuse reflection by using a time-of-flight measurement technique and a computational reconstruction algorithm; the non-visual field three-dimensional reconstruction is realized by adopting a single-point SPAD detector and a laser scanning mode; the non-visual field imaging is realized by adopting a wave optical method and a single-point SPAD detector, the characteristics of a non-visual field imaging system are analyzed, the SPAD is used as the detector, and the non-visual field imaging is obtained by combining the single-pixel imaging principle; the scheme of adopting reverse pinhole imaging achieves the aim of realizing non-visual field target imaging by adopting a common camera. In recent years, the study and exploration of more and more scholars are attracted in the field of non-visual field imaging.

Disclosure of Invention

The present application is directed to solving, at least to some extent, one of the technical problems in the related art.

Therefore, a first objective of the present application is to provide a non-visual field imaging apparatus based on a photomultiplier tube array, so as to achieve data acquisition for transient imaging of a non-visual field object, and fully combine an imaging model of the non-visual field to achieve three-dimensional reconstruction of the non-visual field object by using a reconstruction algorithm of inverse light cone decomposition, deep learning and inverse time projection.

A second object of the present application is to propose a non-visual field imaging method based on a photomultiplier tube array.

To achieve the above object, an embodiment of a first aspect of the present application proposes a non-visual-field imaging device based on a photomultiplier tube array, including:

the device comprises a photomultiplier PMT array, a laser, a time correlation counting device and a calculation reconstruction device;

the PMT array and the time-dependent counting device are used as a collection camera for non-visual field imaging to shoot a target object; wherein, each PMT is connected with a time-to-digital converter TDC;

the laser is used as an excitation source, and an electric synchronization signal output by the laser synchronizes the time-dependent counting device;

and the computing and reconstructing device acquires the time and the photon number recorded by the PMT array as data to be reconstructed, and an inversion reconstruction algorithm is adopted to process the data to be reconstructed so as to reconstruct an imaging target image of a non-visual field.

The non-visual field imaging device based on the photomultiplier tube array comprises the photomultiplier tube PMT array, a laser, a time correlation counting device and a calculation reconstruction device; the PMT array and the time correlation counting device are used as a collection camera for non-visual field imaging to shoot a target object; each PMT is connected with a time-to-digital converter TDC; the laser is used as an excitation source, and an electric synchronous signal output by the laser synchronizes with the time-related counting device; and the computing and reconstructing device acquires the time and photon number recorded by the PMT array as data to be reconstructed, and the data to be reconstructed are processed by adopting an inversion reconstruction algorithm to reconstruct an imaging target image of a non-visual field. Thereby, high-efficiency high-quality non-visual field imaging can be realized.

Optionally, in an embodiment of the present application, the PMT array is a 32 × 32 PMT array, each PMT corresponds to one pixel, and each pixel is connected to one TDC.

Optionally, in an embodiment of the present application, an electrical synchronization signal of the laser is used as a start instruction of the TDC, and after the PMTs receive photons, the TDCs perform time sampling on the PMTs to obtain the number of photons received by each PMT;

and controlling to shoot a laser spot, and acquiring the data to be reconstructed with the number of 32 x 32 photons changing along with time through the PMT array.

Optionally, in one embodiment of the present application, 32 × 32 PMTs are embedded in a square structure, and each PMT receives information about the time-dependent change of photons reflected from the diffusely reflecting plate.

Optionally, in an embodiment of the present application, an electrical pulse output by each PMT is used as an input of the time-dependent counting device, and the time-dependent counting device is composed of a carry chain and a delay line, and each PMT is configured with one counting module.

Optionally, in one embodiment of the present application, the inverse reconstruction algorithm includes, but is not limited to, one or more of a forward model of elliptical tomography, an inverse cone of solution, and a deep learning reconstruction algorithm.

To achieve the above object, an embodiment of a second aspect of the present application provides a non-visual-field imaging method based on a photomultiplier tube array, including:

setting up a time-dependent counting device of PMT arrays and target number channels, wherein each PMT is provided with a counting channel, and pulse signals output by each PMT are connected to the input end of a TDC in the counting device;

configuring a laser with a corresponding wave band and software for recording photon number, wherein the software is used for recording the photon number received by the PMT with the target number and storing the recorded photon number and corresponding time information as a data format;

the laser emits laser pulses, and data of time and photon number recorded on the PMT of the target number are acquired and serve as data to be reconstructed;

and processing the data to be reconstructed by adopting an inversion reconstruction algorithm to reconstruct an imaging target image of a non-visual field.

According to the non-visual field imaging method based on the photomultiplier tube array, the PMT array and a time-dependent counting device of target number channels are set up, each PMT is provided with one counting channel, and pulse signals output by each PMT are connected to the input end of a TDC in the counting device; configuring a laser with a corresponding wave band and software for recording photon number, wherein the software is used for recording the photon number received by the PMT with the target number and storing the recorded photon number and corresponding time information as a data format; the laser emits laser pulses, and data of time and photon number recorded on a PMT (scanning electron microscope) with a target number are acquired and serve as data to be reconstructed; and processing the data to be reconstructed by adopting an inversion reconstruction algorithm to reconstruct an imaging target image of the non-visual field. Thereby, high-efficiency high-quality non-visual field imaging can be realized.

Additional aspects and advantages of the present application will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the present application.

Drawings

The foregoing and/or additional aspects and advantages of the present application will become apparent and readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a schematic structural diagram of a non-visual-field imaging device based on a photomultiplier tube array according to an embodiment of the present disclosure;

FIG. 2 is a schematic flowchart of a non-visual-field imaging method based on a photomultiplier tube array according to a second embodiment of the present application;

FIG. 3 is an exemplary diagram of a PMT array detector according to an embodiment of the present application;

fig. 4 is a diagram of a detection system composed of a PMT and a time-dependent counting device according to an embodiment of the present invention.

Detailed Description

Reference will now be made in detail to embodiments of the present application, examples of which are illustrated in the accompanying drawings, wherein like or similar reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the drawings are exemplary and intended to be used for explaining the present application and should not be construed as limiting the present application.

The non-visual field imaging apparatus and method based on photomultiplier tube arrays according to the embodiments of the present application are described below with reference to the drawings.

Fig. 1 is a schematic structural diagram of a non-visual-field imaging device based on a photomultiplier tube array according to a first embodiment of the present application.

In particular, transient imaging is the key to realize non-visual field imaging, and conventional imaging is a point-to-point imaging mode, assuming that the speed of light is infinite, but in the field of transient imaging, the assumption that the speed of light is infinite does not hold, and corresponds to a point-to-point imaging mode, but a point-to-hyperbolic mode in an imaging system. In addition, since the signal of light in non-visual field imaging after several reflections to the detector end is very weak, the detector must have the capability of detecting single photon. The photomultiplier not only has extremely high sensitivity (single photon) but also has ultrahigh time resolution (picosecond magnitude), and the photomultiplier can collect the arrival time and the intensity of different photons by combining a photon counting technology related to time. By using an array of photomultiplier tubes (e.g., 32 x 32) and a counting device on each of the photomultiplier tubes, a three-dimensional structure of the non-field-of-view imaged object can be obtained using an optimization algorithm for inverse cone-of-view and a deep learning reconstruction algorithm. The scheme provided by the application is simple in implementation scheme, and a high-efficiency and high-quality non-visual field imaging technology can be realized.

As shown in FIG. 1, the non-visual-field imaging device based on the photomultiplier tube array comprises:

the device comprises a photomultiplier PMT array, a laser, a time correlation counting device and a calculation reconstruction device; the PMT array and the time correlation counting device are used as a collection camera for non-visual field imaging to shoot a target object; each PMT is connected with a time-to-digital converter TDC; the laser is used as an excitation source, and an electric synchronous signal output by the laser synchronizes with the time-related counting device; and the computing and reconstructing device acquires the time and photon number recorded by the PMT array as data to be reconstructed, and the data to be reconstructed are processed by adopting an inversion reconstruction algorithm to reconstruct an imaging target image of a non-visual field.

In the embodiment of the present application, the PMT array is a 32 × 32 PMT array, each PMT corresponds to one pixel, and each pixel is connected to one TDC.

In the embodiment of the application, an electric synchronization signal of a laser is used as a starting instruction of a TDC, and after the PMT receives photons, the PMT is subjected to time sampling through the TDC to obtain the number of the photons received by each PMT; and controlling to shoot a laser spot, and acquiring data to be reconstructed, wherein the data are acquired by the PMT array, and the number of 32 x 32 photons is changed along with time.

In the embodiment of the present application, 32 × 32 PMTs are embedded in a square structure, and each PMT receives information about the time variation of photons reflected from the diffuse reflection plate.

In the embodiment of the application, the electric pulse output by each PMT is used as the input end of a time-dependent counting device, the time-dependent counting device is composed of a carry chain and a delay line, and each PMT is provided with a counting module.

In the embodiment of the present application, the inverse reconstruction algorithm includes, but is not limited to, one or more of a forward model of elliptical tomography, an inverse cone of illumination, and a deep learning reconstruction algorithm.

Specifically, the time and intensity of different photons reaching the detector end can be obtained by adopting a high-precision TDC technology, the three-dimensional structure of a non-vision object is obtained through an inversion reconstruction algorithm, such as an inverse light cone decomposition algorithm, a deep learning algorithm and the like, a voltage signal output by the anode of the photomultiplier serves as the input end of the TDC module, and a synchronous electric signal of the laser serves as the START signal end of the TDC module. An array of 32 × 32 photomultiplier tubes is adopted, each photomultiplier tube is connected with a TDC counter, data of 32 × 32 photon numbers changing along with time can be obtained when one frame is collected, data support is provided for later algorithm reconstruction, and a forward model of elliptic chromatography, a reverse light cone solution and a deep learning reconstruction algorithm are adopted.

The application aims to obtain three-dimensional imaging of a non-visual field by adopting single-point laser excitation and a mode of array photomultiplier acquisition, and because laser scanning is not needed for each acquisition, the scheme can obtain three-dimensional dynamic imaging. The photomultiplier tube can collect extremely weak and ultra-fast light signals, but depth information, namely arrival time of different photons and extremely high resolution requirements, must be predicted for reconstructing a three-dimensional object in a non-visual field. The single photon counting technology is configured on the basis of the traditional PMT, the photon time information of 32 x 32 detectors can be obtained by adopting the PMT of an array (32 x 32) and the TDC of the corresponding array, and the non-visual field three-dimensional reconstruction can be obtained by adopting a forward model and a reverse light-resolving cone of the elliptical tomography and a deep learning reconstruction algorithm.

The method adopts a 32 × 32 PMT array, each detector is equivalent to one pixel, a TDC system of 32 × 32 channels is designed by adopting a carry chain and a delay line of an FPGA, and each pixel is connected with one TDC. The electric synchronization signal of the laser is used as a starting instruction of the TDC, and after the PMT receives the photons, the photons are subjected to time sampling through the TDC, so that the photon number of the photons received by the detector along with the time can be obtained. By shooting a laser spot and collecting by a PMT array, high-dimensional data of 32 x 32 photon numbers changing along with time can be obtained, the data comprises photon information of different ellipsoids, and the three-dimensional reconstruction of a non-visual field object can be obtained by using a reconstruction algorithm of deconvolution (light cone) or deep learning.

In order to realize the above embodiments, the present application also proposes a non-visual field imaging method based on a photomultiplier tube array.

Fig. 2 is a schematic flowchart of a non-visual field imaging method based on a photomultiplier tube array according to an embodiment of the present application.

As shown in fig. 2, the non-visual-field imaging method based on the photomultiplier tube array comprises the following steps:

step 101, a time-dependent counting device of PMT arrays and target number channels is set up, each PMT is provided with a counting channel, and pulse signals output by each PMT are connected to an input end of a TDC in the counting device.

And 102, configuring a laser with a corresponding waveband and software for recording the photon number, wherein the software is used for recording the photon number received by the PMT with the target number, and storing the recorded photon number and corresponding time information into a data format.

Step 103, the laser emits laser pulses, and the data of the time and the photon number recorded on the target number PMT are acquired as data to be reconstructed.

And 104, processing the data to be reconstructed by adopting an inversion reconstruction algorithm, and reconstructing an imaging target image of the non-visual field.

Specifically, a PMT array and a 1024-channel time-dependent counting device are set up, each PMT is provided with a counting channel, and a pulse signal output by each PMT is connected to an input end of a TDC in the counting device; configuring a laser with a corresponding waveband and software capable of recording photon number, wherein the software can record the photon number received by 1024 PMTs and store the recorded photon number and corresponding time information into a data format; the laser emits laser pulses to obtain data of time and photon number recorded on 1024 detectors as reconstructed original data; and reconstructing the imaging target of the non-visual field by adopting an optimization algorithm of inverse light cone or inverse time projection or a deep learning method.

As shown in fig. 3, using 32 × 32 PMTs to form an array type large detector, information of photons in the corresponding field of view over time can be obtained.

As shown in fig. 4, the electric pulse signal output from the anode of the PMT serves as the hit terminal of the time-dependent counter, and picosecond time-resolved time information can be obtained by using the carry chain and the delay line.

And (3) adopting a laser as excitation, synchronizing a time-related counting device by using an electric synchronizing signal output by the laser, finally obtaining 32 × 32 time-related counting values, and reconstructing a three-dimensional structure of the non-visual field by adopting an optimization or deep learning algorithm.

As shown in fig. 3, 32 × 32 PMTs embedded in a square structure, each detector can receive the time-varying information of the photons reflected from the diffusely reflecting plate.

As shown in fig. 4, the electric pulse output by the PMT is used as the input end of the time-dependent counting, and a counting device composed of a carry chain and a delay line is used, and a counting module is configured on each PMT.

Therefore, the PMT array and the time-dependent counting device are used as a collection camera for non-visual field imaging, the laser is used as an excitation source, high-dimensional data can be obtained by adopting array collection, and finally, the non-visual field imaging is realized by adopting a calculation imaging algorithm.

It should be noted that the foregoing explanation of the embodiment of the non-visual-area imaging apparatus based on the photomultiplier tube array also applies to the non-visual-area imaging method based on the photomultiplier tube array of this embodiment, and details are not repeated here.

In the description herein, reference to the description of the term "one embodiment," "some embodiments," "an example," "a specific example," or "some examples," etc., means that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. In this specification, the schematic representations of the terms used above are not necessarily intended to refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, various embodiments or examples and features of different embodiments or examples described in this specification can be combined and combined by one skilled in the art without contradiction.

Furthermore, the terms "first", "second" and "first" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present application, "plurality" means at least two, e.g., two, three, etc., unless specifically limited otherwise.

Any process or method descriptions in flow charts or otherwise described herein may be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing steps of a custom logic function or process, and alternate implementations are included within the scope of the preferred embodiment of the present application in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art of the present application.

The logic and/or steps represented in the flowcharts or otherwise described herein, e.g., an ordered listing of executable instructions that can be considered to implement logical functions, can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. For the purposes of this description, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic device) having one or more wires, a portable computer diskette (magnetic device), a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium could even be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, via for instance optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.

It should be understood that portions of the present application may be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, the various steps or methods may be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. If implemented in hardware, as in another embodiment, any one or combination of the following techniques, which are known in the art, may be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application specific integrated circuit having an appropriate combinational logic gate circuit, a Programmable Gate Array (PGA), a Field Programmable Gate Array (FPGA), or the like.

It will be understood by those skilled in the art that all or part of the steps carried by the method for implementing the above embodiments may be implemented by hardware related to instructions of a program, which may be stored in a computer readable storage medium, and when the program is executed, the program includes one or a combination of the steps of the method embodiments.

In addition, functional units in the embodiments of the present application may be integrated into one processing module, or each unit may exist alone physically, or two or more units are integrated into one module. The integrated module can be realized in a hardware mode, and can also be realized in a software functional module mode. The integrated module, if implemented in the form of a software functional module and sold or used as a stand-alone product, may also be stored in a computer readable storage medium.

The storage medium mentioned above may be a read-only memory, a magnetic or optical disk, etc. Although embodiments of the present application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present application, and that variations, modifications, substitutions and alterations may be made to the above embodiments by those of ordinary skill in the art within the scope of the present application.

Claims

1. A non-visual field imaging device based on a photomultiplier tube array, characterized in that, comprising: a photomultiplier tube PMT array, a laser, a time correlation counting device and a calculation reconstruction device;

The PMT array and the time-correlated counting device are used as a non-visual field imaging acquisition camera to photograph the target object; wherein, each of the PMTs is connected to a time-to-digital converter TDC;

The laser is used as an excitation source, and the electrical synchronization signal output by the laser synchronizes the time-dependent counting device;

The calculation and reconstruction device obtains the time and the number of photons recorded by the PMT array as data to be reconstructed, uses an inversion reconstruction algorithm to process the data to be reconstructed, and reconstructs a non-field of view imaging target image.

2. The apparatus of claim 1, wherein

The PMT array is a 32*32 PMT array, each PMT is equivalent to a pixel, and each pixel is connected to one of the TDCs.

3. The apparatus of claim 2, wherein

The electrical synchronization signal of the laser is used as the start command of the TDC. After the PMT receives the photons, the PMT is time-sampled by the TDC to obtain the number of photons received by each PMT;

A laser spot is controlled and collected through the PMT array to obtain the data to be reconstructed with the number of 32*32 photons changing with time.

4. The apparatus of claim 2, wherein

32*32 PMTs are embedded in a square structure, and each PMT receives the time-varying information of photons reflected from the diffuse reflector.

5. The apparatus of claim 1, wherein

The electrical pulse output by each of the PMTs is used as the input terminal of the time-dependent counting device, and a time-dependent counting device composed of a carry chain and a delay line is used, and each of the PMTs is configured with a counting module.

6. The apparatus of claim 1, wherein,

The inversion reconstruction algorithm includes, but is not limited to, one or more of the forward model of elliptical tomography, the inverse solution light cone and the deep learning reconstruction algorithm.

7. A non-field of view imaging method based on a photomultiplier tube array, characterized in that, comprising:

Build a PMT array and a time-dependent counting device with a target number of channels, each PMT is configured with a counting channel, and the pulse signal output by each PMT is connected to the input end of the TDC in the counting device;

Configure the laser of the corresponding band and the software for recording the number of photons, which are used to record the number of photons received by the PMT of the target number, and save the number of recorded photons and the corresponding time information as a data format;

The laser emits laser pulses to obtain the time and photon number data recorded on the target quantity PMT as data to be reconstructed;

An inversion reconstruction algorithm is used to process the data to be reconstructed to reconstruct an imaging target image that is not in the field of view.