HK40067194A - Slide-free histological imaging method and system - Google Patents

Description

Slide-free histological imaging methods and systems

Related applications

This application claims priority to U.S. Provisional Patent Application No. 62/973,101, filed September 19, 2019, the contents of which are incorporated herein by reference in their entirety.

Technical Field

This invention relates to slide-free histological imaging methods and systems.

Background Technology

Histological examination remains the gold standard for assessing surgical margins in malignant tumors. However, routine histological analysis involves a lengthy and costly sample preparation process that generates toxic reagent waste, depletes small samples, and extends the time to generate histopathology reports by hours to days. This lengthy and costly process includes formalin fixation and paraffin embedding (FFPE), followed by high-quality sectioning, staining, and then mounting the sample onto a slide. These unavoidable steps can take several days to complete, resulting in delays of hours to days in generating accurate diagnostic reports. While intraoperative frozen sections offer a faster alternative to FFPE histology by freezing fresh tissue before physical sectioning, intraoperative frozen sections still require 20 to 30 minutes of preparation and turnaround time. Furthermore, frozen sections have inherent problems with freezing artifacts, especially when dealing with lipid-rich tissues, which can lead to intraoperative misinterpretations and diagnostic traps.

The immense demand in histopathology has spurred numerous efforts to achieve rapid and non-invasive diagnosis of unstained fresh tissue. Certain microsurgical techniques for imaging non-sectioned tissues, including ultraviolet (UV) surface excitation microscopy, confocal laser scanning microscopy, and light slide microscopy, have reduced the arduous task and processing costs associated with preparing large numbers of slides in routine FFPE histology. However, these methods require specific fluorescent labeling to enhance molecular specificity. While fluorescence imaging is undoubtedly highly efficient for providing information on the morphology and dynamics of different biomolecules within cells, it necessitates the use of exogenous labelers or gene transfections, which can interfere with cellular metabolism and adversely affect subsequent clinical implementation. Furthermore, long-term monitoring of cells with fluorescent labels can lead to phototoxicity of cells and photobleaching of the fluorophores themselves.

Stimulated Raman scattering (SRS) and coherent anti-Stokes Raman scattering (CARS), which characterize image structures through intrinsic molecular vibrations of specific chemical bonds, offer label-free alternatives for examining C-H stretching in lipid-rich structures. Furthermore, nonlinear processes originating from non-centrosymmetric interfaces, including second-harmonic generation (SHG), third-harmonic generation (THG), and their combined modes, have shown great potential for intrinsic characterization of collagen and microtubule structures. However, these methods require high-power ultrafast lasers to maintain detection sensitivity and molecular contrast, which may not be readily available in most cases. Spectroscopic confocal reflectance microscopy allows for label-free, high-resolution in vivo imaging of myelinated axons, but due to low molecular specificity, confocal microscopy with tunable wavelengths remains necessary. Quantitative phase imaging techniques also offer significant possibilities for rapid refractive index mapping by measuring phase changes in unstained samples. However, they are mostly integrated into transport systems and are strictly limited by sample thickness. Furthermore, reflection-based imaging techniques, such as optical coherence tomography, have been adapted into intraoperative diagnostic tools for label-free imaging of human breast tissue; however, they are not designed to achieve subcellular resolution and are not suitable for probing the molecular targets expected in standard care clinicopathology.

Summary of the Invention

The purpose of this invention is to address one or more of the disadvantages described above or herein, or at least to provide a useful alternative.

In a first aspect, a computer-implemented method for generating pseudo-hematoxylin and eosin (H&E) staining images is provided, wherein the method includes: receiving an input image, which is an unlabeled sample-based ultraviolet-based autofluorescence microscopy (UV-AutoM) image or an ultraviolet-based photoacoustic microscopy (UV-PAM) image, wherein the input image is a grayscale image; transforming the input image into a pseudo-H&E staining image of the input image using a generative adversarial network; and outputting the pseudo-H&E staining image.

In some implementations, generative adversarial networks are generative adversarial networks with cycle consistency.

In some implementations, the method includes training a generative adversarial network using unpaired input and H&E-stained images.

In some implementations, the generative adversarial network includes four deep convolutional neural networks: a first generator deep convolutional neural network configured to transform an input image into a generated H&E image; a second generator deep convolutional neural network configured to transform the H&E image into a generated UV-AutoM or UV-PAM image; a first discriminator deep convolutional neural network configured to distinguish between H&E images in the training set and generated H&E images generated by the first generator deep convolutional neural network; and a second discriminator deep convolutional neural network configured to distinguish between UV-AutoM or UV-PAM images in the training set and generated UV-AutoM or UV-PAM images generated by the second generator deep convolutional neural network.

In some implementations, the first and second generator deep convolutional neural networks are ResNet-based or U-Net-based generator networks.

In some implementations, the first and second discriminator deep convolutional neural networks are PatchGAN discriminator networks.

In some implementations, the input image received in the form of a UV-PAM image is generated by: controlling a galvanometer scanner of a focusing component to focus ultraviolet light onto the sample according to a scanning trajectory; receiving photoacoustic waves emitted by the sample in response to ultraviolet light through at least one transducer; and generating a UV-PAM image based on the photoacoustic waves.

In some implementations, the input image received in the form of a UV-AutoM image is an estimated UV-AutoM image generated from a sequence of speckle-lit images captured according to a scan trajectory, wherein the estimated UV-AutoM image has a higher resolution compared to each speckle-lit image in the sequence.

In some implementations, the estimated UV-AutoM image is generated by: a) initializing a high-resolution image object based on interpolation of the average value of a sequence of speckle-illuminated images; b) for each speckle-illuminated image in the sequence: i) generating the estimated speckle-illuminated image by computationally shifting the high-resolution image object to a specific position in the scan trajectory; ii) determining the filtered object-pattern composite in the frequency domain based on the estimated speckle-illuminated image in the frequency domain and the optical transfer function; iii) based on the estimated speckle-illuminated image in the frequency domain, the corresponding captured speckle-illuminated image in the frequency domain, and the filtered object-pattern composite in the frequency domain. The optical transfer function is used to determine the updated estimated speckle-illuminated image in the frequency domain; iv) the high-resolution object is updated based on the updated estimated speckle-illuminated image, the estimated speckle-illuminated image in the spatial domain, and the speckle pattern; v) the speckle pattern is updated based on the updated estimated speckle-illuminated image, the estimated speckle-illuminated image, and the high-resolution image object; vi) Nesterov momentum acceleration is applied to the high-resolution image object and the speckle pattern; and c) step b) is performed iteratively until convergence of the reconstructed high-resolution image object is detected, which is the estimated UV-AutoM image with enhanced subcellular resolution in the centimeter-scale imaging region.

In a second aspect, a computer system configured to generate pseudo-hematoxylin and eosin (H&E) staining images is provided, wherein the computer system includes one or more memories storing executable instructions, and one or more processors, wherein the processors execute the executable instructions to cause the processors to: receive an input image, which is an unlabeled sample-based ultraviolet-based autofluorescence microscopy (UV-AutoM) image or an ultraviolet-based photoacoustic microscopy (UV-PAM) image, wherein the input image is a grayscale image; transform the input image into a pseudo-H&E staining image of the input image using a generative adversarial network; and output the pseudo-H&E staining image.

In some implementations, generative adversarial networks are generative adversarial networks with cycle consistency.

In some implementations, one or more processors are configured to train a generative adversarial network using unpaired input grayscale images and H&E-stained images.

In some implementations, the generative adversarial network includes four deep convolutional neural networks: a first generator deep convolutional neural network configured to transform an input image into a generated H&E image; a second generator deep convolutional neural network configured to transform the H&E image into a generated UV-AutoM or UV-PAM image; a first discriminator deep convolutional neural network configured to distinguish between H&E images in the training set and generated H&E images generated by the first generator deep convolutional neural network; and a second discriminator deep convolutional neural network configured to distinguish between UV-AutoM or UV-PAM images in the training set and generated UV-AutoM or UV-PAM images generated by the second generator deep convolutional neural network.

In some implementations, the first and second generator deep convolutional neural networks are ResNet-based or U-Net-based generator networks.

In some implementations, the first and second discriminator deep convolutional neural networks are PatchGAN discriminator networks.

In some implementations, the input image received in the form of a UV-PAM image is generated by: controlling a galvanometer scanner of a focusing component to focus ultraviolet light onto the sample according to a scanning trajectory; receiving photoacoustic waves emitted by the sample in response to ultraviolet light through at least one transducer; and generating a UV-PAM image based on the photoacoustic waves.

In some implementations, the input image received in the form of an estimated UV-AutoM image is generated from a sequence of speckle-lit images captured according to a scan trajectory, wherein the estimated UV-AutoM has a higher resolution compared to each speckle-lit image in the sequence.

In some implementations, the estimated UV-AutoM image is generated by: a) initializing a high-resolution image object based on interpolation of the average value of a sequence of speckle-illuminated images; b) for each speckle-illuminated image in the sequence: i) generating the estimated speckle-illuminated image by computationally shifting the high-resolution image to a specific position in the scan trajectory; ii) determining the filtered object-pattern composite in the frequency domain based on the estimated speckle-illuminated image in the frequency domain and the optical transfer function; iii) based on the estimated speckle-illuminated image in the frequency domain, the corresponding captured speckle-illuminated image in the frequency domain, the filtered object-pattern composite in the frequency domain, and the optical transfer function. The transfer function is used to determine the updated estimated speckle-illuminated image in the frequency domain; iv) the high-resolution object is updated based on the updated estimated speckle-illuminated image, the estimated speckle-illuminated image in the spatial domain, and the speckle pattern; v) the speckle pattern is updated based on the updated estimated speckle-illuminated image, the estimated speckle-illuminated image, and the high-resolution image object; vi) Nesterov momentum acceleration is applied to the high-resolution image object and the speckle pattern; and c) step b) is performed iteratively until convergence of the reconstructed high-resolution image object is detected, which is the estimated UV-AutoM image with enhanced subcellular resolution in the centimeter-scale imaging region.

In a third aspect, one or more non-transitory computer-readable media are provided, comprising executable instructions for configuring a computer system to generate pseudo-hematoxylin and eosin (H&E) staining images, wherein the computer system has one or more processors, wherein the one or more processors execute the executable instructions to configure the computer system to: receive an input image, which is an unlabeled sample-based ultraviolet-based autofluorescence microscopy (UV-AutoM) image or an ultraviolet-based photoacoustic microscopy (UV-PAM) image, wherein the input image is a grayscale image; transform the input image into a pseudo-H&E staining image of the input image using a generative adversarial network; and output the pseudo-H&E staining image.

In some implementations, generative adversarial networks are generative adversarial networks with cycle consistency.

In some implementations, one or more processors execute executable instructions to configure a computer system to train a generative adversarial network using unpaired input grayscale images and H&E-stained images.

In some implementations, the generative adversarial network includes four deep convolutional neural networks: a first generator deep convolutional neural network configured to transform an input image into a generated H&E image; a second generator deep convolutional neural network configured to transform the H&E image into a generated UV-AutoM or UV-PAM image; a first discriminator deep convolutional neural network configured to distinguish between H&E images in the training set and generated H&E images generated by the first generator deep convolutional neural network; and a second discriminator deep convolutional neural network configured to distinguish between UV-AutoM or UV-PAM images in the training set and generated UV-AutoM or UV-PAM images generated by the second generator deep convolutional neural network.

In some implementations, the first and second generator deep convolutional neural networks are ResNet-based or U-Net-based generator networks.

In some implementations, the first and second discriminator deep convolutional neural networks are PatchGAN discriminator networks.

In some implementations, the input image received in the form of a UV-PAM image is generated by: controlling a galvanometer scanner of a focusing component to focus ultraviolet light onto the sample according to a scanning trajectory; receiving photoacoustic waves emitted by the sample in response to ultraviolet light through at least one transducer; and generating a UV-PAM image based on the photoacoustic waves.

In some implementations, the input image received in the form of an estimated UV-AutoM image is generated from a sequence of speckle-lit images captured according to a scan trajectory, wherein the estimated UV-AutoM has a higher resolution compared to each speckle-lit image in the sequence.

In some implementations, the estimated UV-AutoM image is generated by: a) initializing a high-resolution image object based on interpolation of the average value of a sequence of speckle-illuminated images; b) for each speckle-illuminated image in the sequence: i) generating the estimated speckle-illuminated image by computationally shifting the high-resolution image to a specific position in the scan trajectory; ii) determining the filtered object-pattern composite in the frequency domain based on the estimated speckle-illuminated image in the frequency domain and the optical transfer function; iii) based on the estimated speckle-illuminated image in the frequency domain, the corresponding captured speckle-illuminated image in the frequency domain, the filtered object-pattern composite in the frequency domain, and the optical transfer function. The transfer function is used to determine the updated estimated speckle-illuminated image in the frequency domain; iv) the high-resolution object is updated based on the updated estimated speckle-illuminated image, the estimated speckle-illuminated image in the spatial domain, and the speckle pattern; v) the speckle pattern is updated based on the updated estimated speckle-illuminated image, the estimated speckle-illuminated image, and the high-resolution image object; vi) Nesterov momentum acceleration is applied to the high-resolution image object and the speckle pattern; and c) step b) is performed iteratively until convergence of the reconstructed high-resolution image object is detected, which is the estimated UV-AutoM image with enhanced subcellular resolution in the centimeter-scale imaging region.

Other aspects and embodiments will be understood through the description of the embodiments.

Attached Figure Description

Preferred embodiments of the invention will now be described by way of example only with reference to the accompanying drawings.

Figures 1A and 1B are schematic diagrams of examples of general-purpose computer systems on which the various arrangements described herein are implemented.

Figures 2A and 2B are schematic diagrams of examples of embedded systems on which the various arrangements described herein are implemented.

Figure 3 is a schematic diagram of an example UV-PAM system.

Figure 4A is an example of a UV-PAM image of gold nanoparticles with a diameter of 200 nm, in which the contour extracted along the white dashed line is averaged.

Figure 4B is an example of the average line profile of four gold nanoparticles, where the FWHM (full width at half maximum) of the Gaussian fit (solid line) is approximately 613 nm, representing the lateral resolution of the UV-PAM system in Figure 3.

Figure 4C is an example of the A-line signal at the center of the gold nanoparticle in Figure 4A, where the envelope of the A-line signal has an FWHM of 38 ns, which corresponds to 58 μm, representing the axial resolution of the UV-PAM system in Figure 3.

Figure 5A is a schematic diagram of an example UV-AutoM system.

Figures 5B and 5C are flowcharts illustrating an example computer-implemented method for reconstructing a UV-AutoM image from a sequence of low-resolution images illuminated by speckle lighting.

Figure 5D is an example of pseudocode illustrating a computer-implemented method for reconstructing UV-AutoM images from a sequence of low-resolution images illuminated by speckle lighting.

Figure 6 shows a graphical representation of an example of an SI reconstruction method and system for reconstructing UV-AutoM images.

Figure 7A is an example of a low-resolution UV-AutoM image captured by a 4X/0.1NA objective lens.

Figure 7B is an example of a high-resolution UV-AutoM image reconstructed using the methods in Figures 5B and 5C.

Figure 7C shows the line outlines of lines 700 and 710, which are marked in Figures 7A and 7B, respectively.

Figures 8A and 8B show examples of UV-AutoM images of two leaf samples with rough surfaces captured by a 4X/0.1NA objective lens.

Figures 8C and 8D show examples of high-resolution UV-AutoM images reconstructed from low-resolution images with multiple speckle illuminations using the methods of Figures 5B and 5C.

Figures 8E and 8F show high-resolution reference images of the two leaf samples captured in Figures 8A and 8B using a 10X/0.3NA objective lens.

Figure 9A is a UV-AutoM image of the entire mouse brain (FFPE slice after dewaxing, 4 μm thick), with a scale bar of 500 μm.

Figures 9B and 9C are enlarged views of boxes 910 and 920 in Figure 9A, each with a scale bar of 50 μm.

Figures 9D and 9E are bright-field H&E staining images corresponding to Figures 9B and 9C, each with a scale bar of 50 μm.

Figure 10A is an example image of a top view of a mouse brain.

Figures 10B and 10C are examples of reconstructed UV-AutoM images, each depicting a cross-section of the mouse brain along lines A-A and B-B cut through Figure 10A.

Figures 10D to 10H show high-pass quantum views of five functional regions of the mouse brain depicted in the reconstructed UV-AutoM image of Figure 10C.

Figure 11A is a functional block diagram of an example computer implementation of a system for generating pseudo-stained histological images (i.e., virtual stained histological images).

Figure 11B is a flowchart illustrating an example computer-implemented method 1150 for generating pseudohematoxylin and eosin (H&E) stained images.

Figure 12 is a functional block diagram showing the detailed workflow of the forward and backward loops of Cycle-GAN.

Figure 13 is a functional block diagram representing the example generator of Cycle-GAN in Figure 11.

Figure 14A is an example of a grayscale UV-PAM image of a mouse brain slice generated using the system of Figure 3, which is fed as input to a trained Cycle-GAN configured according to Figures 11 to 13.

Figure 14B is an example of a virtually stained histological image generated by Cycle-GAN using a grayscale UV-PAM image as the input image and serving as the output image.

Figure 14C is an example of a histological image of the same sample imaged in Figure 14A, obtained using bright-field microscopy after H&E staining.

Figure 15A is an example of a UV-AutoM image of a dewaxed FFPE mouse brain slice with a thickness of 7 μm.

Figure 15B is a virtual tinting version of the UV-AutoM image of Figure 15A using a GAN-based network.

Figure 15C is a bright-field H&E image of a mouse brain slice compared to Figure 15A.

Figures 16A and 16B are high-resolution UV-AutoM images reconstructed by SI, generated using the methods in Figures 5B and 5C, showing mouse brain samples with thicknesses of 100 μm and 200 μm, respectively.

Figures 16C and 16D are examples of virtually stained H&E images generated using the computer-implemented methods and systems shown in Figures 11 to 13.

Figure 17A shows an example of label-free UV-PAM images of the hippocampus from a dewaxed FFPE mouse brain sample with a thickness of 7 μm, where the UV-PAM images were generated using the system in Figure 3.

Figure 17B shows an example of a UV-AutoM image of the hippocampus region imaged in Figure 17A.

Figure 17C shows an example of a bright-field H&E stained image of a mouse brain sample corresponding to the images in Figures 17A and 17B.

Detailed Implementation

In any one or more accompanying drawings, where steps and/or features with the same reference numerals are referenced, these steps and/or features have the same function or operation for the purposes of this description, unless the contrary intention is present.

It should be noted that the discussions included in the "Background Art" section and the above discussions relating to the prior art arrangement relate to discussions of documents or devices that form public knowledge through their respective disclosures and/or uses. This should not be construed as an indication that such documents or devices of the inventor or patent applicant constitute part of common general knowledge in the art in any way.

Referring to Figures 1A and 1B, schematic diagrams of an example of a general-purpose computer system 100 are shown, on which various arrangements described herein are implemented.

As shown in Figure 1A, computer system 100 includes: a computer module 101; input devices, such as a keyboard 102, a mouse pointer device 103, a scanner 126, a camera 127, and a microphone 180; and output devices, including a printer 115, a display device 114, and a speaker 117. Computer module 101 can communicate with communication network 120 via connection 121 using an external modem transceiver device 116. Communication network 120 can be a wide area network (WAN), such as the Internet, a cellular telecommunications network, or a dedicated WAN. If connection 121 is a telephone line, modem 116 can be a conventional "dial-up" modem. Alternatively, if connection 121 is a high-capacity (e.g., cable) connection, modem 116 can be a broadband modem. Wireless modems can also be used for wireless connection to communication network 120.

Computer module 101 typically includes at least one processor unit 105 and a memory unit 106. For example, memory unit 106 may have semiconductor random access memory (RAM) and semiconductor read-only memory (ROM). Computer module 101 also includes multiple input/output (I/O) interfaces, including: an audio-video interface 107 for connecting to a video display 114, speakers 117, and microphone 180; an I/O interface 113 for connecting to a keyboard 102, mouse 103, scanner 126, camera 127, and optional joysticks or other human-machine interface devices (not shown) or projectors; and an interface 108 for external modems 116 and printers 115. In some embodiments, modem 116 may be incorporated into computer module 101, for example, within interface 108. Computer module 101 also has a local area network interface 111, which allows computer system 100 to be connected to a local area communication network 122, referred to as a local area network (LAN), via connection 123. As shown in Figure 1A, the local area communication network 122 can also be connected to the wide area network 120 via connection 124, which will typically include a so-called "firewall" device or a device with similar functionality. The local area network interface 111 may include an Ethernet circuit card, a wireless device, or an IEEE 802.11 wireless device; however, many other types of interfaces can be implemented for interface 111.

I/O interfaces 108 and 113 can provide either or both of serial and parallel connections, the former typically implemented according to the Universal Serial Bus (USB) standard and having a corresponding USB connector (not shown). Storage device 109 is provided, and it typically includes a hard disk drive (HDD) 110. Other storage devices, such as floppy disk drives and tape drives (not shown), may also be used. Optical disk drive 112 is typically provided to serve as a non-volatile data source. Portable storage devices, such as optical discs (e.g., CD-ROM, DVD, Blu-ray Disc ^™ ), USB-RAM, portable external hard disk drives, and floppy disks, may be used as suitable data sources for system 100.

Components 105 to 113 of computer module 101 typically communicate via interconnect bus 104 and in a manner consistent with the conventional operating mode of computer system 100 known to those skilled in the art. For example, processor 105 is connected to system bus 104 via connection 118. Similarly, memory 106 and optical disc drive 112 are connected to system bus 104 via connection 119. Examples of computers on which the described arrangement may be implemented include IBM-PC and compatible machines, SunSparcstation, Apple Mac ^™ , or similar computer systems.

The method described herein can be implemented using a computer system 100, wherein the process described herein can be implemented as one or more software applications 133 executable within the computer system 100. Specifically, the steps of the method described herein are implemented by instructions 131 (see Figure 1B) in the software 133 executing within the computer system 100. The software instructions 131 can be formed as one or more code modules, each code module being used to perform one or more specific tasks.

Software can be stored in a computer-readable medium, such as a storage device as described below. The software is loaded from the computer-readable medium into computer system 100 and then executed by computer system 100. A computer-readable medium on which such software or a computer program is recorded is a computer program product. The use of a computer program product in computer system 100 preferably implements advantageous means for detecting and/or sharing write operations.

Software 133 is typically stored in HDD 110 or memory 106. The software is loaded from a computer-readable medium into computer system 100 and executed by computer system 100. Therefore, for example, software 133 may be stored on an optically readable disk storage medium (e.g., CD-ROM) 125 read by optical disk drive 112. A computer-readable medium on which such software or computer programs are recorded is a computer program product.

In some cases, application 133 may be provided to the user, encoded on one or more CD-ROMs 125 and read through a corresponding drive 112, or alternatively, may be read by the user from network 120 or 122. Furthermore, the software may also be loaded into computer system 100 from other computer-readable media. Computer-readable storage media are any non-transitory tangible storage media that provides recorded instructions and/or data to computer system 100 for execution and/or processing. Examples of such storage media include floppy disks, magnetic tapes, CD-ROMs, DVDs, Blu-ray ^™ discs, hard disk drives, ROMs or integrated circuits, USB storage devices, magneto-optical discs, or computer-readable cards, such as PCMCIA cards, whether such devices are internal or external to computer module 101. Examples of temporary or intangible computer-readable transmission media that may also participate in providing software, applications, instructions, and/or data to computer module 101 include radio or infrared transmission channels and network connections to another computer or networked device, as well as the Internet or intranets, including email transmissions and information recorded on websites, etc.

The second part of application 133 and the corresponding code modules described above can be executed to implement one or more graphical user interfaces (GUIs) for presentation on display 114 or otherwise represented. Typically, by manipulating keyboard 102 and mouse 103, users of computer system 100 and applications can manipulate the interface in a functionally adaptable manner to provide control commands and/or input to the application associated with the GUI. Other forms of functionally adaptable user interfaces can also be implemented, such as an audio interface utilizing voice prompts output through speaker 117 and user voice commands input through microphone 180.

Figure 1B is a detailed schematic block diagram of processor 105 and "memory" 134. Memory 134 represents the logical aggregation of all memory modules (including HDD 109 and semiconductor memory 106) that can be accessed by computer module 101 in Figure 1A.

When computer module 101 is initially powered on, a Power-On Self-Test (POST) program 150 is executed. POST program 150 is typically stored in ROM 149 of semiconductor memory 106 in Figure 1A. Hardware devices such as ROM 149 storing software are sometimes referred to as firmware. POST program 150 checks the hardware within computer module 101 to ensure proper functioning and typically checks processor 105, memories 134 (109, 106), and the Basic Input/Output System Software (BIOS) module 151, which is also typically stored in ROM 149, to ensure correct operation. Once POST program 150 has successfully run, BIOS 151 activates hard disk drive 110 in Figure 1A. Activation of hard disk drive 110 causes bootloader 152 residing on hard disk drive 110 to be executed by processor 105. This loads operating system 153 into RAM memory 106, where operating system 153 begins operation. Operating system 153 is a system-level application that can be executed by processor 105 to implement various advanced functions, including processor management, memory management, device management, storage management, software application interface, and general user interface.

Operating system 153 manages memory 134 (109, 106) to ensure that each process or application running on computer module 101 has sufficient memory to execute without conflicting with memory allocated to another process. Furthermore, the different types of memory available in system 100 of Figure 1A must be used correctly so that each process can run efficiently. Therefore, the aggregated memory 134 is not intended to illustrate how specific memory segments are allocated (unless otherwise stated), but rather to provide a general view of the memory accessible to computer system 100 and how that memory is used.

As shown in Figure 1B, processor 105 includes multiple functional modules, including a control unit 139, an arithmetic logic unit (ALU) 140, and local or internal memory 148, sometimes referred to as cache memory. Cache memory 148 typically includes multiple storage registers 144-146 in its register section. One or more internal buses 141 functionally interconnect these functional modules. Processor 105 also typically has one or more interfaces 142 for communicating with external devices via system bus 104 using connection 118. Memory 134 is connected to bus 104 using connection 119.

Application program 133 includes a sequence of instructions 131, which may include conditional branching and loop instructions. Program 133 may also include data 132 used during its execution. Instructions 131 and data 132 are stored in memory locations 128, 129, 130 and 135, 136, 137, respectively. Depending on the relative size of instructions 131 and memory locations 128-130, a particular instruction may be stored in a single memory location, as depicted in the instruction representation shown in memory location 130. Alternatively, instructions may be segmented into multiple parts, each stored in a separate memory location, as depicted in the instruction segments shown in memory locations 128 and 129.

Typically, processor 105 is given a set of instructions to execute therein. Processor 105 waits for subsequent input, responding to it by executing another set of instructions. Each input can be provided from one or more of a plurality of sources, including data generated by one or more input devices 102, 103, data received from an external source on one of networks 120, 102, data retrieved from one of storage devices 106, 109, or data retrieved from storage medium 125 inserted into a corresponding reader 112, all shown in FIG1A. In some cases, the execution of a set of instructions can result in the output of data. Execution may also involve storing data or variables into memory 134.

The disclosed write detection and shared arrangement uses input variable 154, which is stored in corresponding memory locations 155, 156, and 157 within memory 134. The write detection and shared arrangement produces output variable 161, which is stored in corresponding memory locations 162, 163, and 164 within memory 134. Intermediate variable 158 can be stored in memory locations 159, 160, 166, and 167.

Referring to the processor 105 of Figure 1B, registers 144, 145, 146, arithmetic logic unit (ALU) 140, and control unit 139 work together to execute the sequence of micro-operations required to perform a "fetch, decode, and execute" cycle for each instruction in the instruction set constituting program 133. Each fetch, decode, and execute cycle includes: a fetch operation, which fetches or reads instruction 131 from memory locations 128, 129, 130; a decode operation, in which control unit 139 determines which instruction has been fetched; and an execute operation, in which control unit 139 and/or ALU 140 execute the instruction.

After this, a further fetch, decode, and execution loop can be executed for the next instruction. Similarly, a storage loop can be executed, through which the control unit 139 stores or writes values to storage location 162.

Each step or sub-process described herein is associated with one or more segments of program 133 and is executed by register portions 144, 145, 147, ALU 140, and control unit 139 in processor 105, which work together to perform instruction fetch, decode, and execution loops for each instruction in the instruction set of the segment of program 133.

The methods described herein can alternatively be implemented in dedicated hardware, such as one or more integrated circuits that perform the write detection and sharing methods, or their sub-functions. Such dedicated hardware may include a graphics processor, a digital signal processor, or one or more microprocessors and associated memory.

Figures 2A and 2B together form a schematic block diagram of a general-purpose electronic device 201 including embedded components, on which the write detection and/or sharing method to be described is deliberately practiced. The electronic device 201 may be, for example, a mobile phone, a portable media player, virtual reality glasses, or a digital camera, where processing resources are limited. However, the method to be described can also be implemented on higher-level devices, such as desktop computers, server computers, and other such devices with significantly more processing resources.

As shown in Figure 2A, the electronic device 201 includes an embedded controller 202. Therefore, the electronic device 201 can be referred to as an "embedded device". In this example, the controller 202 has a processing unit (or processor) 205 bidirectionally connected to an internal storage module 209. The storage module 209 may be formed of a non-volatile semiconductor read-only memory (ROM) 260 and a semiconductor random access memory (RAM) 270, as shown in Figure 2B. The RAM 270 may be volatile, non-volatile, or a combination of volatile and non-volatile memory.

Electronic device 201 includes a display controller 207 connected to a display 214, such as a liquid crystal display (LCD) panel. The display controller 207 is configured to display graphic images on the display 214 according to instructions received from an embedded controller 202 to which the display controller 207 is connected.

Electronic device 201 also includes a user input device 213, which is typically formed by keys, a keypad, or similar controls. In some embodiments, the user input device 213 may include a touch-sensitive panel physically associated with display 214 to collectively form a touchscreen. Such a touchscreen can therefore operate as a form of graphical user interface (GUI) rather than a prompt- or menu-driven GUI typically used with a keypad-display combination. Other forms of user input devices may also be used, such as a microphone (not shown) for voice commands or a joystick/thumb wheel (not shown) for easy navigation of menus.

As shown in Figure 2A, the electronic device 201 also includes a portable storage interface 206, which is connected to the processor 205 via a connection 219. The portable storage interface 206 allows complementary portable storage devices 225 to be connected to the electronic device 201 to act as a source or destination of data or to supplement the internal storage module 209. Examples of such interfaces allow connection to portable storage devices such as Universal Serial Bus (USB) storage devices, Secure Digital (SD) cards, PCMIA cards, optical discs, and hard disks.

Electronic device 201 also has a communication interface 208 to allow device 201 to be connected to a computer or communication network 220 via connection 221. Connection 221 can be wired or wireless. For example, connection 221 can be radio frequency or optical. Examples of wired connections include Ethernet. Furthermore, examples of wireless connections include Bluetooth ^™ type local interconnect, Wi-Fi (including protocols based on the IEEE 802.11 family of standards), Infrared Data Association (IrDa), etc.

Typically, electronic device 201 is configured to perform certain special functions. An embedded controller 202, possibly in conjunction with additional special function components 210, is provided to perform these special functions. For example, in the case of device 201 being a digital camera, component 210 could represent the camera lens, focus control, and image sensor. Special function components 210 are connected to the embedded controller 202. As another example, device 201 could be a mobile phone. In this case, component 210 could represent those components required for communication in a cellular phone environment. In the case of device 201 being a portable device, special function components 210 could represent multiple encoders and decoders of types including Joint Group of Picture Experts (JPEG), Moving Picture Experts Group (Moving Picture Experts Group) MPEG, MPEG-1 Audio Layer 3 (MP3), etc.

The methods described below can be implemented using an embedded controller 202, wherein the processes described herein can be implemented as one or more software applications 233 executable within the embedded controller 202. The electronic device 201 of FIG. 2A implements the described methods. Specifically, referring to FIG. 2B, the steps of the described methods are implemented by instructions in the software 233 executing within the controller 202. The software instructions can be formed as one or more code modules, each code module performing one or more specific tasks. The software can also be divided into two separate parts, wherein a first part and its corresponding code modules execute the described methods, and a second part and its corresponding code modules manage the user interface between the first part and the user.

The software 233 of the embedded controller 202 is typically stored in the non-volatile ROM 260 of the internal storage module 209. The software 233 stored in ROM 260 can be updated from a computer-readable medium as needed. The software 233 can be loaded into and executed by the processor 205. In some cases, the processor 205 can execute software instructions located in RAM 270. Software instructions can be loaded into RAM 270 by adding copies of one or more code modules from ROM 260 to RAM 270 via the processor 205. Alternatively, the software instructions for one or more code modules can be pre-installed by the manufacturer in a non-volatile area of RAM 270. After one or more code modules are located in RAM 270, the processor 205 can execute the software instructions of those one or more code modules.

Prior to distribution of electronic device 201, application 233 is typically pre-installed by the manufacturer and stored in ROM 260. However, in some cases, application 233 may be provided to the user, encoded on one or more CD-ROMs (not shown) and read via portable storage interface 206 of FIG2A before being stored in internal storage module 209 or portable storage device 225. Alternatively, software application 233 may be read by processor 205 from network 220 or loaded from other computer-readable media into controller 202 or portable storage medium 225. Computer-readable storage medium refers to any non-transitory tangible storage medium that participates in providing instructions and/or data to controller 202 for execution and/or processing. Examples of such storage media include floppy disks, magnetic tapes, CD-ROMs, hard disk drives, ROMs or integrated circuits, USB storage devices, magneto-optical disks, flash memory, or computer-readable cards, such as PCMCIA cards, whether such devices are internal or external to device 201. Examples of temporary or intangible computer-readable transmission media that may also be used to provide software, applications, instructions, and/or data to device 201 include radio or infrared transmission channels and network connections to another computer or networked device, as well as the Internet or intranet, including email transmissions and information recorded on websites. A computer-readable medium on which such software or computer programs are recorded is a computer program product.

The second part of application 233 and the corresponding code modules described above can be executed to implement one or more graphical user interfaces (GUIs) for presentation or other representation on display 214 of FIG. 2A. By manipulating user input device 213 (e.g., keypad), the user of device 201 and application 233 can manipulate the interface in a functionally adaptable manner to provide control commands and/or input to the application associated with the GUI. Other forms of functionally adaptable user interfaces can also be implemented, such as an audio interface utilizing voice prompts output through a speaker (not shown) and user voice commands input through a microphone (not shown).

Figure 2B illustrates in detail an embedded controller 202 having a processor 205 for executing application program 233 and internal storage 209. Internal storage 209 includes read-only memory (ROM) 260 and random access memory (RAM) 270. The processor 205 is capable of executing application program 233 stored in one or both of the connected memories 260 and 270. When the electronic device 201 is initially powered on, the system program residing in ROM 260 is executed. The application program 233, permanently stored in ROM 260, is sometimes referred to as "firmware." The processor 205 executing the firmware can implement various functions, including processor management, memory management, device management, storage management, and user interface.

Processor 205 typically includes multiple functional modules, including a control unit (CU) 251, an arithmetic logic unit (ALU) 252, a digital signal processor (DSP) 253, and local or internal memory 254 including a set of registers typically containing atomic data elements 256, 257, as well as internal buffers or cache memory 255. One or more internal buses 259 interconnect these functional modules. Processor 205 also typically has one or more interfaces 258 for communicating with external devices via system bus 281 using connection 261.

Application program 233 includes a sequence of instructions 262 to 263, which may include conditional branching and looping instructions. Application program 233 may also include data used during its execution. This data may be stored as part of the instructions or in a separate location 264 within ROM 260 or RAM 270.

Typically, processor 205 is given a set of instructions that are executed within the processor. These instructions can be organized into blocks that perform a specific task or handle a specific event occurring in electronic device 201. Generally, application 233 waits for an event and then executes the code block associated with that event. An event may be triggered in response to input from a user via user input device 213 of FIG. 2A, which is detected by processor 205. Events may also be triggered in response to other sensors and interfaces in electronic device 201.

The execution of a set of instructions may require reading and modifying numerical variables. Such numerical variables are stored in RAM 270. The disclosed method uses input variable 271 stored at known locations 272 and 273 in memory 270. Input variable 271 is processed to produce output variables 277 and 279 stored at known locations 278 in memory 270. Intermediate variable 274 may be stored in additional memory locations 275 and 276 in memory 270. Alternatively, some intermediate variables may exist only in register 254 of processor 205.

The execution of the instruction sequence is achieved in processor 205 through the repeated application of a fetch-execute loop. The control unit 251 of processor 205 maintains a register called the program counter, which contains the address of the next instruction to be executed in ROM 260 or RAM 270. At the beginning of the fetch-execute loop, the contents of the memory address indexed by the program counter are loaded into control unit 251. The loaded instruction controls subsequent operations of processor 205, such as loading data from ROM memory 260 into processor register 254, performing arithmetic combinations of register contents with the contents of another register, and writing register contents to a location stored in another register. At the end of the fetch-execute loop, the program counter is updated to point to the next instruction in the system program code. Depending on the instruction just executed, this may involve incrementing the address contained in the program counter or loading the program counter with a new address to implement a branch operation.

Each step or sub-procedure in the process of the method described below is associated with one or more segments of application 233 and is executed by repeatedly executing an instruction fetch-execution loop in processor 205 or similar program operations of other independent processor blocks in electronic device 201.

Various methods and systems are provided for high-throughput, label-free, and slide-free imaging based on the inherent light absorption contrast under ultraviolet (UV) illumination to directly probe histologically stained biomolecules. Two methods are disclosed: (i) photoacoustic microscopy (UV-PAM) and (ii) autofluorescence microscopy (UV-AutoM). To achieve high throughput in UV-PAM, a high-speed optical scanning configuration can be used. Related to UV-AutoM, speckle illumination (SI) is utilized, which allows for estimation of high-resolution images using low-magnification objectives, providing subcellular resolution images over centimeter-scale imaging areas, while allowing high tolerance to image blurring caused by tissue surface morphology, slide placement errors, and thickness.

For both UV-PAM and UV-AutoM images, a deep learning-based virtual staining method and system are disclosed, which can be used to generate large histological images of untreated fresh/fixed tissue at subcellular resolution. The virtual staining method and system utilize a generative adversarial network (GAN) configured to transform UV-PAM or UV-AutoM images of unlabeled tissue into histologically stained images using paired/unpaired training examples. The disclosed method and system can streamline the workflow of standard care histopathology from days to less than ten minutes, enabling intraoperative surgical margin assessment and reducing or eliminating the need for a second surgery due to positive margins.

UV-based photoacoustic microscopy (UV-PAM)

Unlike conventional optical microscopy, PAM takes advantage of highly specific optical absorption contrast. By using a UV pulsed laser (wavelength range of ~240–280 nm) as the excitation beam, cell nuclei can be highlighted, thus providing label-free histological images.

Referring to Figure 3, an example of a UV-PAM system 300 is shown. Specifically, a nanosecond pulsed UV laser 302 (e.g., WEDGE-HF 266nm, available from Bright Solution LLC) is focused by a focusing assembly 303. Specifically, the light emitted by the UV laser 302 is broadened by a pair of lenses 304, 308 (e.g., LA4647-UV and LA4663-UV, available from Thorlabs). The quality of the UV beam can be enhanced by a pinhole 306 located between these lenses 304, 308. The diameter of the pinhole 306 can be from 10 μm to 100 μm (e.g., P25C, available from Thorlabs). The beam is then reflected by a 1D galvanometer scanner 310 and then focused onto the bottom of a sample 312 by an objective lens 310 (e.g., Thorlabs' MicroSpot ^™ focusing objective lens (LMU-20X-UVB)). Sample 312 is placed at the bottom of water tank 314, which is held by sample holder 315 attached to XYZ translation stage 318. Water tank 314 is filled with water to allow photoacoustic waves to propagate upward and be detected by immersion ultrasonic transducer 316 (e.g., V324-SU, available from Olympus NDT).

The received sound pressure is converted into an electrical signal, which is then amplified by amplifier 320 (e.g., two ZFL-500LN-BNC+ amplifiers, available from Mini-Circuits), and finally received by computer system 100 or 201 via data acquisition system 322 (e.g., an ATS9350, available from Alazar Technologies). To generate a two-dimensional image, the maximum amplitude projection (MAP) of each A-line signal is first identified. The MAPs are then rearranged according to the order during the scanning process to generate a grayscale image.

In operation, the galvanometer scanner 310, with its focusing component, can be controlled to focus ultraviolet light onto the sample 312 according to a scanning trajectory. Control of the galvanometer scanner 310 can be performed by a part of the computer system 100 or a computerized embedded device 201. The transducer 316 is configured to receive photoacoustic waves emitted by the sample 312 in response to ultraviolet light. The computer system 100 or the embedded device 201 generates a UV-PAM image based on the photoacoustic waves.

To measure the lateral and axial resolution of the UV-PAM system, gold nanoparticles (200 nm in diameter) were imaged along the x and y axes with a step size of 0.15 μm (Fig. 4(a)). Data points from four gold nanoparticles were selected and averaged to measure the lateral resolution by Gaussian curve fitting, as shown in Fig. 4(b). The full width at half maximum (FWHM) of the Gaussian fitted curve was ~0.6 μm. To evaluate the axial resolution, the envelope of the A-line signal at the center position was extracted. Fig. 4(c) shows the A-line signal at the center position of the gold nanoparticle in (a). The FWHM of the A-line signal envelope was 38 ns, which corresponds to 58 μm, representing the axial resolution of the UV-PAM system.

UV-based autofluorescence microscopy (UV-AutoM)

In histopathological examination, objectives with a magnification factor of 20X-40X are typically required to achieve subcellular resolution for observing cell morphology and metabolic activity. However, such a magnification factor limits the field of view (FOV) to less than 1 ^mm² . Furthermore, high-magnification objectives are more susceptible to spatial variation aberrations and have a shallow depth of field (DOF), resulting in lower tolerance for errors in microscope slide placement and sample roughness. For these reasons, using high-magnification objectives for image stitching to capture large tissue surfaces is suboptimal.

A speckle illumination (SI) method is disclosed to alleviate the inherent trade-off between large FOV and high resolution (HR) in digital microscopy, thereby enabling high-throughput visualization of different regions of interest with subcellular resolution.

Figure 5A illustrates an example UV-AutoM system 500, in which unstained fresh tissue 312 is placed on a top-open sample holder connected to an XYZ motorized stage 318 (e.g., three L-509s, available from PI miCos). A UV laser 302 (e.g., WEDGE-HF 266nm, available from Bright Solution LLC) is collimated and projected onto a fused silica diffuser (e.g., DGUV10-600, available from Thorlabs) to produce a speckle pattern. The resulting wave is focused through an aspherical UV condenser lens 512 (e.g., #33-957, available from Edmund Optics) with a numerical aperture (NA) of 0.69, which is part of a focusing assembly 303. The sample 312 is obliquely illuminated through a UV transparency window 514 (e.g., with a transmittance greater than 90% from 200nm to 1500nm). The excited autofluorescence signal (primarily from NADH and FAD, with peak emission at 450 nm) is then collected by an inverted microscope 517 equipped with a 4X objective 518 (e.g., NA=0.1, a planar achromatic objective, available from Olympus NDT) and an infinity correction tube lens 520 (TTL180-A, available from Thorlabs), and finally imaged by a monochrome scientific complementary metal-oxide-semiconductor (sCMOS) camera 522 (e.g., PCOedge 4.2, 6.5 μm pixel pitch, available from PCO).

Low-magnification objectives are less affected by spatially varying aberrations at large fields of view (FOV) and offer greater distances of focus (DOF) and working distances, allowing for high tolerance to slide placement errors and flexible stage handling. However, their spatial resolution is largely limited by their low numerical aperture (NA) value, which is a determinant of the achievable resolution of the imaging system according to the Rayleigh criterion (i.e., the minimum distance at which the imaging system can be resolved is 0.61λ/NA, where λ is the fluorescence emission wavelength and NA is the numerical aperture of the objective). Therefore, spatial resolution (SI) reconstruction is used to circumvent this resolution limitation imposed by low-NA objectives in this configuration.

A computational imaging method for autofluorescence microscopy based on speckle illumination is disclosed. In a preferred embodiment, the method achieves high-throughput microscopy. Specifically, "high-throughput microscopy" involves using automated microscopy and image analysis to visualize and quantitatively capture cellular features at large scales. More specifically, due to the application of speckle illumination, the spatial bandwidth product (i.e., field of view/ ^resolution² ) of the high-throughput output is approximately or equal to 10 times that of conventional fluorescence microscopy, which is typically limited to the megapixel level. Low-magnification objectives are more advantageous for imaging large tissue surfaces because they are less affected by spatially varying aberrations. Furthermore, since low-magnification lenses have a large depth of field, defocused image blur caused by surface irregularities, tissue thickness, or slide placement errors can be minimized by implementing low-magnification lenses. However, the low numerical aperture (NA) value of such lenses significantly limits the achievable resolution, thus hindering their application to subcellular imaging targets. The proposed method performs iterative reconstruction from a sequence of low-resolution autofluorescence images illuminated by speckle illumination to bypass the resolution limitations set by the NA objective, thereby facilitating rapid high-resolution imaging over large imaging areas with arbitrary surface morphologies.

Referring to Figures 5B and 5C, flowcharts are shown of an example computer-implemented method 530 representing the reconstruction of UV-excited autofluorescence images (UV-AutoM) from a sequence of speckle-illuminated images. The computer-implemented method 530 can be performed by the computer system 100 described with respect to Figures 1A and 1B or by the embedded device 201 described with respect to Figures 2A and 2B. Figure 5D provides pseudocode further illustrating a more specific implementation of the reconstruction of high-throughput UV-excited autofluorescence images (UV-AutoM) from a sequence of speckle-illuminated images. The flowcharts of Figures 5B and 5C will be described here with reference to the pseudocode of Figure 5D.

Specifically, in step 530-1, method 530 includes recording a sequence _Ij (j = 1, 2, ..., N) of speckle-illuminated images of samples translated into a plane along a scanning trajectory at corresponding positions. In the sense that the output of method 530 is a high-resolution image, the sequence I of speckle-illuminated images is a low-resolution image with higher resolution compared to each _Ij in the low-resolution speckle-illuminated image. In one example, the sequence of speckle-illuminated images can be captured using a 4X/0.1NA objective lens.

In step 530-2, method 530 includes initializing an image object o(x, y) and a speckle pattern, where the image object o(x, y) is referred to herein as a high-resolution image object. As shown in line 3 of the pseudocode, the sequence of speckle-illuminated images is averaged and the averaged speckle-illuminated image is interpolated, wherein the high-resolution image object o(x, y) is set to the interpolated result of the averaged speckle-illuminated image. The speckle pattern is initialized to a matrix of 1s.

In step 530-3, the current position is set as the first position in the position sequence. The current position in the pseudocode of Figure 5D is represented by ( _xj , _yj ).

Steps 530-4 to 530-11 are performed on each speckle-illuminated image in the captured sequence, in the form of an inner loop of the flowcharts shown in Figures 5B and 5C. Referring to the pseudocode in Figure 5D, the inner loop is represented by lines 6 to 14, while the outer loop is represented by lines 5 to 17. For the inner loop, the current position variable is effectively incremented to the next position in the position sequence, and the corresponding speckle-illuminated image at the corresponding current position is used for the image processing steps to modify the high-resolution image object and the speckle pattern.

More specifically, in steps 530-4, method 530 includes generating an estimated image of speckle illumination by computationally shifting the high-resolution object to the current position o(xx _j , yy _j ) and then multiplying it by the speckle pattern p(x, y), as shown in line 7 of the pseudocode in Figure 5D.

In steps 530-5, method 530 includes determining the filtered object-pattern composite _ψj ( _kx , _ky ) in the frequency domain based on the estimated speckle-illuminated image multiplied by the optical transfer function OTF( _kx , _ky ), where _kx and _ky are spatial coordinates in the frequency domain. The optical transfer function is a known optical transfer function of the means for capturing a sequence of speckle-illuminated images of the sample.

It should be noted that the shift operations in steps 530-4 and 530-5 are collectively referred to as the application of the angular spectrum.

Steps 530-6 to 530-8 described below are the reconstruction process based on a phase retrieval algorithm called the Ptychographiciterative engine (PIE).

In steps 530-6, method 530 includes determining an updated estimated speckle illumination image in the frequency domain based on the estimated speckle illumination image in the frequency domain, the captured speckle illumination image at the current location in the frequency domain, the filtered object-pattern composite in the frequency domain, the optical transfer function, and the adaptive learning rate parameter α. More specifically, line 9 of the pseudocode in Figure 5D illustrates the specific calculation of the updated estimated speckle illumination image in the frequency domain, as shown in Equation 1 below:

Specifically, the updated estimated speckle illumination image in the frequency domain is calculated as: the estimated speckle illumination image in the frequency domain plus the adaptive learning rate parameter multiplied by the conjugate of the optical transfer function multiplied by the difference between the speckle illumination image captured at the current location in the frequency domain and the filtered object-pattern composite in the frequency domain divided by the square of the absolute maximum value of the optical transfer function.

In steps 530-7, method 530 includes updating the high-resolution object based on the updated estimated speckle illumination image and speckle pattern in the spatial domain. This is shown in line 10 of the pseudocode in Figure 5D and is represented by Equation 2 below:

Specifically, the high-resolution object is set to be equal to: the high-resolution object plus the conjugate multiplier of the speckle pattern, divided by the difference between the updated estimated speckle-illuminated image in the spatial domain and the estimated speckle-illuminated image in the spatial domain, and the square of the absolute value of the maximum value of the speckle pattern.

In steps 530-8, method 530 includes updating the speckle pattern based on the updated estimated speckle illumination image, the estimated speckle illumination image, and the high-resolution object. This is shown in line 11 of the pseudocode in Figure 5D and is represented by Equation 3 below:

Specifically, the speckle pattern is set to be equal to: the speckle pattern plus the conjugate of the high-resolution object multiplied by the difference between the updated estimated speckle-illuminated image in the spatial domain and the estimated speckle-illuminated image in the spatial domain divided by the square of the absolute value of the maximum value of the high-resolution object.

In steps 530-9, the total loss parameter _lossj for the current loop is calculated based on the absolute value of the difference between the captured speckle-illuminated image _Ij and the inverse Fourier transform of the filtered object-pattern composite _ψj in the frequency domain. This is shown in line 12 of the pseudocode in Figure 5D and is represented by Equation 4 below:

loss _j = ∑ _j |I _j -F ^-1 (ψ _j )| Equation 4

In steps 530-10, method 530 includes applying Nesterov momentum acceleration to the high-resolution object and speckle pattern. This step is performed to accelerate the gradient descent of the reconstruction process to speed up convergence.

In steps 530-11, method 530 includes determining whether the current position is the last position in the position sequence. If the response is affirmative (i.e., "yes"), the method proceeds to step 530-13. If the response is negative (i.e., "no"), the method then proceeds to step 530-12.

In steps 530-12, method 530 includes setting the current position as the next position in the position sequence. Method 530 then proceeds back to step 530-4 to perform one or more additional iterations of the inner loop represented by steps 530-4 to 530-11 until the last speckle-illuminated image is processed.

In steps 530-13, method 530 includes determining whether convergence has been detected based on the total loss parameter. This is determined by determining a loss ratio, which is calculated based on the difference between the total loss parameter calculated for the previous and current iterations of the inner loop (i.e., steps 530-4 to 530-11) divided by the total loss parameter of the previous iteration of the inner loop. The loss ratio is then compared to a loss threshold, which is set to 0.01 in the example pseudocode of Figure 5D. If the loss ratio is less than or equal to the loss threshold, the adaptive learning rate parameter α is reduced, which is halved in this example. The adaptive learning rate parameter α is adjusted to suppress oscillations in the loss function near the convergence point to minimize artifacts in the reconstructed image. Once the learning rate α is reduced to zero, the reconstruction process ends, such that a high-resolution object is determined and output in steps 530-14. The output high-resolution image object is a high-resolution UV-AutoM image with enhanced subcellular resolution over a centimeter-scale imaging region, which has improved resolution compared to the low-resolution image provided as input to reconstruction method 530 for each speckle illumination. The initially unknown speckle pattern at the beginning of method 530 is also determined in steps 530-14.

This computational imaging method, represented by Method 530, synthesizes a sequence of speckle-illuminated low-resolution images to reconstruct a high-resolution autofluorescence image (UV-AutoM). This method is implemented through a series of update processes in the spatial and frequency domains. The method begins with an initial guess of the high-resolution object. The object is first multiplied by the speckle pattern, Fourier transformed to the frequency domain, and low-pass filtered using an optical transfer function. The filtered spectrum is then inversely transformed to the spatial domain, its intensity replaced by the corresponding speckle-illuminated low-resolution image. Finally, this updated autofluorescence image is transformed to the frequency domain and further updated. One iteration is completed until all captured low-resolution images are involved, and Nesterov momentum acceleration is implemented for faster gradient descent. After multiple iterations, a high-resolution UV-AutoM image with enhanced subcellular resolution over centimeter-scale image regions is output. No prior knowledge of the speckle pattern is required; only the relative shifts between each low-resolution image are known.

After several iterations, a high-resolution UV-AutoM image is output, exhibiting enhanced subcellular resolution in the centimeter-scale image region. No prior knowledge of the speckle pattern is required; only the relative shifts (xj, yj) between each captured image are needed. This indicates that sufficient scanning range (greater than ~2 low NA diffraction-limited spot sizes) and finer scanning steps (less than the target resolution) can reduce distortion in reconstruction, and the ultimately achievable NA is the sum of the objective NA and the speckle NA.

Figure 6 illustrates a graphical representation of the SI reconstruction method and system previously described in Figures 5A to 5C. Figure 6 at 610 shows the physical constraints set in Fourier space by a 4X/0.1NA objective and the corresponding low-resolution raw image of a mouse brain sample (100 μm thick). The SI dataset 620 was acquired using a 4X/0.1NA objective, and as shown in Figure 6, in this example, the dataset includes low-resolution measurements of 49 speckle illuminations, captured by translating the sample to 49 different locations in the X-Y plane with a scan step size of 500 nm. The high-throughput UV-AutoM image 630 with an extended passband up to NA = 0.3 can then be reconstructed using the SI dataset, which corresponds to a 3x resolution enhancement compared to the image acquired using a 4X/0.1NA objective.

Fluorescent nanoparticles with a diameter of 500 nm (excitation/emission: 365 nm/445 nm, B500, available from Thermo Fisher) can be used to quantify the resolution performance of the aforementioned SI reconstruction method and system. Referring to Figure 7A, an example of a low-resolution fluorescence image captured using a 4X/0.1 NA objective under uniform illumination is shown, while Figure 7B is a high-resolution fluorescence image reconstructed from a low-resolution image illuminated by 49 speckle patterns scanned with a 500 nm step grating. Both Figures 7A and 7B were captured using a 266 nm UV laser. Figure 7C is an intensity map along the solid lines 700 and 710 indicated in Figures 7A and 7B, from which we can quantify the resolution enhancement achieved through the disclosed SI reconstruction method and system.

Advantageously, the SI reconstruction method and system are highly tolerant of rough surfaces. To demonstrate the high tolerance of our system to rough surfaces, Figures 8A-8C show UV-AutoM images of two untreated leaf samples. Figures 8A and 8B show low-resolution autofluorescence images captured using a 4X/0.1NA objective, while Figures 8C and 8D show high-resolution SI-reconstructed UV-AutoM images, reconstructed from low-resolution images illuminated by 49 speckles scanned at 500 nm steps. Figures 8E and 8F show corresponding high-resolution reference images captured using a 10X/0.3NA objective, exhibiting significant defocus areas compared to Figures 8C and 8D due to the shallow DOF of the high-NA objective, demonstrating that high-resolution UV-AutoM images reconstructed via the SI method using a low-NA objective can be far superior to those using a high-NA objective, especially when dealing with rough surfaces.

Referring to Figures 9A to 9F, the structural matching between UV-AutoM and bright-field H&E stained images is shown. Figure 9A is a UV-AutoM image of a whole mouse brain section (dewaxed FFPE section, 4 μm thick), Figures 9B and 9C are magnified images of boxes 910 and 920 in Figure 9A, respectively, while Figures 9D and 9E are the corresponding bright-field H&E histological images captured by a digital slide scanner (NanoZoomer SQ, Hamamatsu). It can be seen that cells concentrated in the cut margin (box 910) and hippocampal (box 920) regions are clearly resolved in the negative-contrast UV-AutoM image, i.e., appear as black on the image, and exhibit a near-perfect structural match (cell morphology and distribution) with the H&E stained image.

Referring to Figures 10A to 10G, high-throughput label-free visualization of a thick mouse brain (100 μm thick, cut by a Leica vibratory slicer) is shown. Due to the shallow penetration depth of UV light, the excited autofluorescent surface lies within a few micrometers below the surface, allowing label-free and slide-free imaging of thick samples. Furthermore, the aforementioned SI reconstruction method and system enable rapid, high-resolution visualization of the entire mouse brain using a 4X objective, resulting in fewer imaging aberrations compared to a 10X objective and allowing for greater tolerance to rough tissue surfaces. Figure 10A shows an example image of a top view of the mouse brain. Figures 10B and 10C are examples of reconstructed UV-AutoM images, each depicting a cross-section (lines AA and BB) cut through the corresponding location of the mouse brain in Figure 10A. Each reconstructed UV-AutoM image was generated using an SI dataset comprising 441 low-resolution images with speckle illumination. The exposure time for each low-resolution raw image was 200 ms, enabling rapid imaging, with a total acquisition time within 3 minutes. Figures 10D to 10H show high-pass quantum views of five functional regions in Figure 10C, including (a) corpus callosum 1010, (b) hippocampus 1020, (c) globus pallidus 1030, (d) caudate putamen 1040 and (e) parietal cortex 1050.

Generation of pseudo-stained histological images

Pathologists are typically trained to examine histologically stained tissue samples to make diagnostic decisions. However, both UV-PAM and UV-AutoM images are grayscale images. To address or mitigate this issue, a deep learning-based virtual staining method utilizing generative adversarial networks (GANs) is disclosed to transform UV-PAM or UV-AutoM images of unlabeled tissue into pseudohematoxylin and eosin (H&E) stained images.

GANs allow for virtual staining of unpaired UV-PAM/UV-AutoM and H&E images, greatly simplifying the image preprocessing process, which is challenging because tissue rotation or deformation may be necessary during staining. Paired training methods can also be used for UV-PAM/UV-AutoM images and their corresponding H&E stained images to perform pseudo-staining.

In some embodiments, UV-PAM images generated using the methods and systems described with respect to FIG3 and SI-reconstructed UV-AutoM images generated using the systems and methods of FIG5A, 5B, and 5C can be provided as input to a trained cycle-GAN to generate pseudo-stained histological images. Additionally or alternatively, UV-PAM images generated using the systems and methods disclosed in U.S. Patent Application No. 2014/0356897 (the contents of which are incorporated herein by reference in their entirety) can be used as input to a GAN to generate pseudo-stained histological images.

Referring to Figure 11A, a block diagram is shown illustrating an example computer-implemented system for generating pseudo-stained histological images (i.e., virtual stained histological images). This computer-implemented system can be implemented using computer system 100 or embedded system 201.

The computer-implemented system utilizes a generative adversarial network 1100 (GAN), which can be provided in the form of a cycle-consistent generative adversarial network (Cycle-GAN). Cycle-GAN 1100 comprises four deep convolutional neural networks: a first generator module G, a second generator module F, a first discriminator module X, and a second discriminator module Y.

As shown in Figure 11A, generator G is configured to learn to transform a UV-AutoM/UV-PAM input image 1110 (exemplified as a UV-PAM image) into a bright-field H&E image 1120 (exemplified as a BR-HE image), while generator F is configured to learn to transform the bright-field H&E image 1120 into a UV-AutoM/UV-PAM image 1110. Discriminator X is configured to distinguish between genuine UV-PAM images and fake UV-PAM images generated by generator F, as shown in output 1130. Simultaneously, discriminator Y is configured to distinguish between genuine bright-field H&E images and fake bright-field H&E images generated by generator G, as shown in output 1140. Once generator G can generate H&E images that discriminator Y cannot distinguish from the input genuine H&E image, generator G has learned this transformation from UV-AutoM/UV-PAM images to H&E images. Similarly, once generator F can generate UV-AutoM/UV-PAM images that discriminator X cannot distinguish from real H&E images, generator F has learned this transformation from H&E images to UV-AutoM/UV-PAM images.

Referring to Figure 11B, a flowchart illustrating an example computer implementation method 1150 for generating pseudohematoxylin and eosin (H&E) stained images is shown. The computer implementation method can be executed by computer system 100 or embedded system 201.

In step 1160, method 1150 includes receiving an input image. The input image is either a UV-based autofluorescence microscopy (UV-AutoM) image of an unlabeled sample or a UV-based photoacoustic microscopy (UV-PAM) image. The input image is a grayscale image.

In step 1170, method 1150 includes using a generative adversarial network to transform the input image into a pseudo-H&E staining image of the input image.

In step 1180, method 1150 includes outputting a pseudo-H&E staining image.

Preferably, the generative adversarial network is a generative adversarial network with cycle consistency.

In some implementations, method 1150 includes training a generative adversarial network using unpaired input and H&E-stained images.

Referring to Figure 12, a functional block diagram of the detailed workflow 1200 of the Cycle-GAN 1100 of Figure 11, representing the forward loop 1202 and backward loop 1204, is shown. This recurrent generative adversarial network comprises four deep convolutional neural networks: a first generator G configured to transform an input image into a generated H&E image; a second generator F configured to transform the H&E image into a generated UV-AutoM or UV-PAM image; a first discriminator Y configured to distinguish between H&E images in the training set and generated H&E images generated by the first generator's deep convolutional neural network; and a second discriminator X configured to distinguish between UV-AutoM or UV-PAM images in the training set and generated UV-AutoM or UV-PAM images generated by the second generator's deep convolutional neural network.

More specifically, after a UV-AutoM/UV-PAM image 1210 (here referred to as the input image) is input to the generator G, a forward loop 1202 is shown in the top row of the schematic diagram in Figure 12, where the generator G outputs a generated H&E image 1220. A discriminator Y is configured to determine whether the generated H&E image 1220 is real or fake (i.e., the discriminator Y is configured to identify whether the H&E image received from the generator G originates from the input real H&E image or from the generator G). Conversely, the generated H&E image 1220 from the generator G is provided as input to the generator F to transform it back into a UV-AutoM/UV-PAM image 1230, which is referred to as the recurrent image 1230. The loss between the input image 1210 and the recurrent image 1230 is called the cycle consistency loss in Cycle-GAN. As shown in the bottom row 1204 of the schematic diagram in Figure 12, the backward loop is symmetrical to the forward loop. The backward loop begins with the input H&E image 1240, where the backward loop learns to transform the BR-H&E image 1240 into a UV-AutoM/UV-PAM image 1250. Similarly, the discriminator X is configured to determine whether the generated UV-AutoM/UV-PAM image is real or fake by comparing the generated UV-AutoM/UV-PAM image with the input image.

Referring to Figure 13, a functional block diagram of an example generator 1300 representing the Cycle-GAN of Figure 11A is shown. In one form, the first and second generators, generators G and F, can be configured as ResNet-based generator networks. In an alternative embodiment, generators G and F can be configured as U-Net-based generator networks. Each ResNet-based generator can consist of several downsampling layers 1320A, 1330A, 1330B, 13330C, residual blocks 1310A-1310I, and upsampling layers 1330D, 1330E, and 1320B. In the example shown in Figure 13, nine residual blocks 1310A-1310I are used to train UV-PAM and HE images, each with 256x256 pixels. Spatial reflection padding (3x3) is added at the beginning of the neural network to ensure that the input and output of the corresponding generator neural network have the same size. Following the padding layer 1320A, each ResNet-based generator 1300 includes a downsampling path comprising three Convolution-InstanceNorm-ReLU layers 1330A, 1330B, and 1330C. Specifically, the first Convolution-InstanceNorm-ReLU downsampling layer 1330A has a larger receptive field with a kernel size of 7x7, while the other two layers 1330B and 1330C have smaller receptive fields with a kernel size of 3x3. The image size is restored to the original image size, and the number of channels increases to 64 after the first layer 1330A. As the image passes through the other two layers 1330B and 1330C, the image size is reduced by a factor of 2, while the number of channels increases by a factor of 2. Next are the downsampling layers 1320A, 1330A, 1330B, and 1330C, which is a long residual neural network comprising nine residual blocks 1310A-1310I. As the image passes through each residual block, the image size and number of channels remain unchanged (256x64x64). After nine residual blocks 1310A-1310I, generator 1300 includes an upsampling path comprising two Convolution-InstanceNorm-ReLU layers 1330D and 1330E with a kernel size of 3x3, and a reflection fill layer 1320B (3x3) with a kernel size of 7x7, connected to Convolution-InstanceNorm-ReLU layer 1340. After each upsampling layer, the image size is doubled, while the number of channels is halved. After two upsampling layers, the image size is restored to the original image size, and the number of channels is reduced to 64. The final connected layer 1340 is configured to maintain the same image size while reducing the number of channels to 3.

In a specific configuration, the discriminator networks Dx and Dy can be provided by a 70x70 PatchGAN discriminator, which consists of four 4x4 Convolution-InstanceNorm-LeakyReLU layers. PatchGAN will produce a one-dimensional output (true or false) after the last layer.

The described GAN was implemented using Python version 3.7.3 with PyTorch version 1.0.1. The software was implemented on a desktop computer with a 3.7GHz Core i7-8700K CPU and 32GB of RAM, running Ubuntu 18.04.2LTS. Training and testing of the Cycle-GAN neural network were performed using a GeForce GTX1080Ti GPU with 11GB of RAM. However, it should be understood that other computer systems or embedded systems can be utilized.

This article will discuss examples of virtual stained histological images generated using the Cycle-GAN described above.

Referring to Figures 14A to 14C, virtual stained images of UV-PAM images of mouse brain slices were generated using the UV-PAM method and system described above, and transformed using the Cycle-GAN 1100 described above. The UV-PAM images and the virtual stained images of the UV-PAM images were compared with histological images of the same samples after H&E staining. Specifically, 7 μm thick sample slices excised from FFPE mouse brains were imaged, wherein the disclosed UV-PAM method and system were operated using raster scanning with a step size of 0.63 μm. Since the UV-PAM method and system were configured to generate grayscale images, the grayscale UV-PAM images (as shown in Figure 14A) were then transformed into virtual stained images using the Cycle-GAN 1100 described with respect to Figures 11 to 13, as shown in Figure 14B. To evaluate whether the generated virtual stained images could provide information similar to conventional histological images, histological images of the same samples were obtained using bright-field microscopy after H&E staining, as shown in Figure 14C. When examining the color of cell nuclei and other connective tissues in virtual stained images generated by the UV-PAM system, it was determined that the virtual stained images generated by the UV-PAM system were substantially similar to conventional histological images. Overall, the UV-PAM system, combined with the deep learning system provided in the publicly available form of Cycle-GAN 1100, can generate substantially accurate virtual stained histological images without the need for conventional staining techniques.

Figures 15A to 15C illustrate examples of virtual staining of UV-AutoM images using a paired training method. Figure 15A is a UV-AutoM image of a 7 μm thick dewaxed FFPE mouse brain slice, and Figure 15B is a virtual staining version of the UV-AutoM image using a GAN model based on a paired pix2pix network. Figure 15C is the corresponding bright-field H&E image used for comparison purposes. Color transformation using paired datasets can accurately and reliably generate histological sample images. However, any paired training method requires rigorous data preprocessing procedures for image alignment and registration and is difficult to apply to virtual staining of thick samples. It was found that a Cycle-GAN-based network allows for color mapping without paired training examples and shows great potential on biological tissues of any thickness. Unpaired UV-AutoM and H&E images from dewaxed FFPE mouse brain slices can be fed into the Cycle-GAN network 1100. A well-trained Cycle-GAN1100 network can transform UV-AutoM images of unlabeled tissue into virtual H&E stained versions of unlabeled tissue, thus allowing pathologists to easily interpret UV-AutoM images.

Figures 16A and 16B illustrate the test results of the trained Cycle-GAN network 1100 on mouse brain samples of varying thicknesses. Figures 16A and 16B are SI-reconstructed high-resolution UV-AutoM images of mouse brain samples with thicknesses of 100 μm and 200 μm, respectively, while Figures 16C and 16D are the corresponding virtual H&E staining images. The successful color mapping from UV-AutoM contrast to the H&E staining version has greatly facilitated the development of the UV-AutoM imaging modality into a practical intraoperative diagnostic tool for use by surgeons and pathologists in the operating room.

It should be understood that complementary contrast exists between UV-PAM and UV-AutoM images, thus enabling methods and systems for generating UV-PAM and UV-AutoM images. Photons (or fluorescence) are generated through radiative relaxation, while heat is generated through non-radiative relaxation, where PA waves are released through thermally induced pressure/temperature increases in the sample. Therefore, PA and autofluorescence images are expected to exhibit complementary contrast according to energy conservation. This contrast is experimentally demonstrated in Figures 17A and 17B. Using UV laser excitation (266 nm), Figure 17A shows the label-free PA contrast (UV-PAM), while Figure 17B shows the label-free autofluorescence contrast (UV-AutoM) of the hippocampus from a 7 μm thick dewaxed FFPE mouse brain sample. Figure 17C is the corresponding bright-field H&E staining image, showing structural similarity to the UV-PAM and UV-AutoM images. Due to the strong absorption of cell nuclei in the ultraviolet range, cell nuclei concentrated in the hippocampus appear bright in the UV-PAM image and dark in the UV-AutoM image. This complementary imaging contrast mechanism enables dual-modal label-free imaging facilities to provide more structural and functional information about unprocessed fresh tissue.

Throughout this description, brain samples were extracted from Swiss Webster mice and subsequently fixed in 10% neutral buffered formalin for 24 hours at room temperature. For thin sections (2–8 μm), samples were processed using an FFPE workflow and sectioned using a microtome. FFPE tissue sections were dewaxed using xylene and mounted on quartz slides for imaging using the described UV-PAM and UV-AutoM systems, followed by H&E staining. For thick sections (20–200 μm), samples were directly cut using a vibratory microtome at different target thicknesses.

Although the invention has been described with reference to preferred embodiments, those skilled in the art will understand that the invention may be practiced in other forms.

The advantageous embodiments and/or further developments disclosed above—except in cases of clearly relevant or inconsistent alternatives—can be applied individually or in any combination of each other.

Claims (21)

1. A computer-implemented method for generating pseudohematoxylin and eosin H&E staining images, wherein the method comprises: Receive an input image, wherein the input image is an unlabeled sample image obtained by UV-autofluorescence microscopy (UV-AutoM) or UV-photoacoustic microscopy (UV-PAM), and the input image is a grayscale image. The input image is transformed into a pseudo-H&E-stained image using a generative adversarial network; and Output the pseudo-H&E staining image. The input image received in the form of a UV-AutoM image is an estimated UV-AutoM image generated from a sequence of speckle-lit images captured according to a scan trajectory, wherein the estimated UV-AutoM image has a higher resolution compared to each speckle-lit image in the sequence. The estimated UV-AutoM image is generated in the following manner: a) Initialize a high-resolution image object by interpolating the average value of the sequence of images illuminated by the speckle illumination; b) For each speckle-illuminated image in the sequence: i) The estimated speckle illumination image is generated by computationally shifting the high-resolution image object to a specific position in the scan trajectory; ii) Based on the estimated speckle illumination image and optical transfer function in the frequency domain, determine the filtered object-pattern composite in the frequency domain; iii) Based on the estimated speckle illumination image in the frequency domain, the corresponding captured speckle illumination image in the frequency domain, the filtered object-pattern composite in the frequency domain, and the optical transfer function, determine the updated estimated speckle illumination image in the frequency domain; iv) Update the high-resolution image object based on the updated estimated speckle illumination image, the estimated speckle illumination image in the spatial domain, and the speckle pattern; v) Update the speckle pattern based on the updated estimated speckle illumination image, the estimated speckle illumination image, and the high-resolution image object; and vi) Apply Nesterov momentum acceleration to the high-resolution image object and the speckle pattern; and c) Iteratively execute step b) until convergence of the reconstructed high-resolution image object is detected, the high-resolution image object being an estimated UV-AutoM image with enhanced subcellular resolution in a centimeter-scale imaging region. 2. The computer-implemented method according to claim 1, wherein the generative adversarial network is a generative adversarial network with cycle consistency. 3. The computer-implemented method of claim 1 or 2, wherein the method includes training the generative adversarial network using unpaired inputs and H&E-stained images. 4. The computer-implemented method according to claim 1 or 2, wherein the generative adversarial network comprises four deep convolutional neural networks, including: A first generator deep convolutional neural network is configured to transform the input image into a generated H&E image; The second generator is a deep convolutional neural network, which is configured to transform H&E images into generated UV-AutoM or UV-PAM images. A first discriminator deep convolutional neural network is configured to distinguish between H&E images in the training set and generated H&E images generated by the first generator deep convolutional neural network; and The second discriminator deep convolutional neural network is configured to distinguish between UV-AutoM or UV-PAM images in the training set and generated UV-AutoM or UV-PAM images generated by the second generator deep convolutional neural network. 5. The computer-implemented method according to claim 4, wherein the first generator deep convolutional neural network and the second generator deep convolutional neural network are generator networks based on ResNet or U-Net. 6. The computer-implemented method according to claim 4, wherein the first discriminator deep convolutional neural network and the second discriminator deep convolutional neural network are PatchGAN discriminator networks. 7. The computer-implemented method according to claim 1 or 2, wherein the input image received in the form of a UV-PAM image is generated in the following manner: A galvanometer scanner that controls the focusing component focuses ultraviolet light onto the sample according to the scanning trajectory; The photoacoustic waves emitted by the sample in response to ultraviolet light are received by at least one transducer; and The UV-PAM image is generated based on the photoacoustic waves. 8. A computer system configured to generate pseudohematoxylin and eosin H&E staining images, wherein the computer system includes one or more memories storing executable instructions, and one or more processors, wherein the processors execute the executable instructions to cause the one or more processors to: Receive an input image, wherein the input image is an unlabeled sample image obtained by UV-autofluorescence microscopy (UV-AutoM) or UV-photoacoustic microscopy (UV-PAM), and the input image is a grayscale image. The input image is transformed into a pseudo-H&E-stained image using a generative adversarial network; and Output the pseudo-H&E staining image. The input image, received in the form of an estimated UV-AutoM image, is generated from a sequence of speckle-lit images captured according to a scan trajectory, wherein the estimated UV-AutoM has a higher resolution compared to each speckle-lit image in the sequence. The UV-AutoM image is generated by the one or more processors in the following manner: a) Initialize a high-resolution image object by interpolating the average value of the sequence of images illuminated by the speckle illumination; b) For each speckle-illuminated image in the sequence: i) The estimated speckle illumination image is generated by computationally shifting the high-resolution image to a specific position in the scan trajectory; ii) Based on the estimated speckle illumination image and optical transfer function in the frequency domain, determine the filtered object-pattern composite in the frequency domain; iii) Based on the estimated speckle illumination image in the frequency domain, the corresponding captured speckle illumination image in the frequency domain, the filtered object-pattern composite in the frequency domain, and the optical transfer function, determine the updated estimated speckle illumination image in the frequency domain; iv) Update the high-resolution image object based on the updated estimated speckle illumination image, the estimated speckle illumination image in the spatial domain, and the speckle pattern; v) Update the speckle pattern based on the updated estimated speckle illumination image, the estimated speckle illumination image, and the high-resolution image object; and vi) Apply Nesterov momentum acceleration to the high-resolution image object and the speckle pattern; and c) Iteratively execute step b) until convergence of the reconstructed high-resolution image object is detected, the high-resolution image object being the estimated UV-AutoM image with enhanced subcellular resolution over a centimeter-scale imaging region. 9. The computer system according to claim 8, wherein the generative adversarial network is a generative adversarial network with cycle consistency. 10. The computer system of claim 8 or 9, wherein the one or more processors are configured to train the generative adversarial network using unpaired input grayscale images and H&E-stained images. 11. The computer system according to claim 8 or 9, wherein the generative adversarial network comprises four deep convolutional neural networks, including: A first generator deep convolutional neural network is configured to transform the input image into a generated H&E image; The second generator is a deep convolutional neural network, which is configured to transform H&E images into generated UV-AutoM or UV-PAM images. A first discriminator deep convolutional neural network is configured to distinguish between H&E images in the training set and generated H&E images generated by the first generator deep convolutional neural network; and The second discriminator deep convolutional neural network is configured to distinguish between UV-AutoM or UV-PAM images in the training set and generated UV-AutoM or UV-PAM images generated by the second generator deep convolutional neural network. 12. The computer system of claim 11, wherein the first generator deep convolutional neural network and the second generator deep convolutional neural network are generator networks based on ResNet or U-Net. 13. The computer system of claim 11, wherein the first discriminator deep convolutional neural network and the second discriminator deep convolutional neural network are PatchGAN discriminator networks. 14. The computer system according to claim 8 or 9, wherein the input image received in the form of a UV-PAM image is generated in the following manner: A galvanometer scanner that controls the focusing component focuses ultraviolet light onto the sample according to the scanning trajectory; The photoacoustic waves emitted by the sample in response to ultraviolet light are received by at least one transducer; and The UV-PAM image is generated based on the photoacoustic waves. 15. One or more non-transitory computer-readable media comprising executable instructions for configuring a computer system to generate pseudo-hematoxylin and eosin H&E staining images, wherein the computer system has one or more processors, wherein the one or more processors execute the executable instructions to configure the computer system to: Receive an input image, wherein the input image is an unlabeled sample image obtained by UV-autofluorescence microscopy (UV-AutoM) or UV-photoacoustic microscopy (UV-PAM), and the input image is a grayscale image. The input image is transformed into a pseudo-H&E-stained image using a generative adversarial network; and Output the pseudo-H&E staining image. The input image, received in the form of an estimated UV-AutoM image, is generated from a sequence of speckle-lit images captured according to a scan trajectory, wherein the estimated UV-AutoM has a higher resolution compared to each speckle-lit image in the sequence. The UV-AutoM image is generated by the one or more processors in the following manner: a) Initialize a high-resolution image object by interpolating the average value of the sequence of images illuminated by the speckle illumination; b) For each speckle-illuminated image in the sequence: i) The estimated speckle illumination image is generated by computationally shifting the high-resolution image to a specific position in the scan trajectory; ii) Based on the estimated speckle illumination image and optical transfer function in the frequency domain, determine the filtered object-pattern composite in the frequency domain; iii) Based on the estimated speckle illumination image in the frequency domain, the corresponding captured speckle illumination image in the frequency domain, the filtered object-pattern composite in the frequency domain, and the optical transfer function, determine the updated estimated speckle illumination image in the frequency domain; iv) Update the high-resolution image object based on the updated estimated speckle illumination image, the estimated speckle illumination image in the spatial domain, and the speckle pattern; v) Update the speckle pattern based on the updated estimated speckle illumination image, the estimated speckle illumination image, and the high-resolution image object; and vi) Apply Nesterov momentum acceleration to the high-resolution image object and the speckle pattern; and c) Iteratively execute step b) until convergence of the reconstructed high-resolution image object is detected, the high-resolution image object being the estimated UV-AutoM image with enhanced subcellular resolution over a centimeter-scale imaging region. 16. One or more non-transitory computer-readable media according to claim 15, wherein the generative adversarial network is a generative adversarial network with cycle consistency. 17. One or more non-transitory computer-readable media according to claim 15, wherein the one or more processors execute the executable instructions to configure the computer system to train the generative adversarial network using unpaired input grayscale images and H&E-stained images. 18. One or more non-transitory computer-readable media according to any one of claims 15 to 17, wherein the generative adversarial network comprises four deep convolutional neural networks, including: A first generator deep convolutional neural network is configured to transform the input image into a generated H&E image; The second generator is a deep convolutional neural network, which is configured to transform H&E images into generated UV-AutoM or UV-PAM images. A first discriminator deep convolutional neural network is configured to distinguish between H&E images in the training set and generated H&E images generated by the first generator deep convolutional neural network; and The second discriminator deep convolutional neural network is configured to distinguish between UV-AutoM or UV-PAM images in the training set and generated UV-AutoM or UV-PAM images generated by the second generator deep convolutional neural network. 19. One or more non-transitory computer-readable media according to any one of claims 15 to 17, wherein the first generator deep convolutional neural network and the second generator deep convolutional neural network are generator networks based on ResNet or U-Net. 20. One or more non-transitory computer-readable media according to any one of claims 15 to 17, wherein the first discriminator deep convolutional neural network and the second discriminator deep convolutional neural network are PatchGAN discriminator networks. 21. One or more non-transitory computer-readable media according to any one of claims 15 to 17, wherein the input image received in the form of a UV-PAM image is generated by: A galvanometer scanner that controls the focusing component focuses ultraviolet light onto the sample according to the scanning trajectory; The photoacoustic waves emitted by the sample in response to ultraviolet light are received by at least one transducer; and The UV-PAM image is generated based on the photoacoustic waves.