WO1999010771A1 - Focusing and autofocusing in scanning laser imaging - Google Patents

Abstract

The present invention encompasses methods and apparatus for focusing and autofocusing light imaging systems so as to improve image clarity and resolution. The method utilizes programming logic responsive to the transmission, absorption, or reflectance of light (40) from a target (20) onto a detector (32). The programming logic compares real-time detection data with previously stored data to determine the degree of focus of the light source on the target. The programming logic then adjusts the focal length of the light source (10) and the target (20) based on the degree of focus of the light source on the target. Preferably, the invention is used in conjunction with a laser imaging system.

Description

FOCUSING AND AUTOFOCUSING IN SCANNING LASER IMAGING

FIELD OF THE INVENTION This invention relates to a method and apparatus for focusing and autofocusing in carrying out scanning imaging, particularly scanning laser imaging of translucent objects. BACKGROUND OF THE INVENTION

To overcome inherent limitations in optical microscopy, scanning laser imaging systems have been devised that provide more rapid, quantitative capture of digital images.

These scanning laser imaging systems provide several advantages. First, they eliminate the need to move the sample glass slide underneath an objective lens, as is required in a typical optical system. These systems also allow rapid imaging of random spots in the field of interest. Other advantages are described in U.S. Patent 5,037,207 issued to Tomei et al. on August 6, 1991 entitled "LASER IMAGING SYSTEM."

Tomei et al. shows a conventional laser imaging system. The schematic diagram of

Figure 1 depicts the system. A primary laser provides a beam to a beam expander composed of an objective lens and a spatial filter. The beam exits the beam expander as an input collimated beam. A three-dimensional beam position controller receives the input collimated beam. The beam controller includes an imaging lens and galvanometrically- driven mirrors to provide control of the spot focus of a focused laser beam on a sample target. Forward light (i.e., light that is either scattered by the sample and transmitted through the target or primary light that is transmitted through the sample without scatter by the sample) is captured by a detector assembly. The detector assembly comprises an optical fiber faceplate, diffusion elements and a photomultiplier tube ("PMT"). The image signal produced by the PMT is subsequently sent to a support computer system which further processes the image signal for display on a high resolution monitor or for storage in an image storage unit. Further details concerning the overall construction and operation of the laser imaging system are provided in the Tomei et al. patent.

To achieve superior results with such a laser scanning system, the laser light must be accurately focused on the target. Focusing can be done manually, but that often requires special focusing equipment and requires human intervention with the potential for introducing error. In most cases, automatic focusing, or "autofocusing," makes the scanning process faster and more efficient than manual focusing. Unless focusing can be done with a minimum of extra equipment or steps, however, many of the advantages of scanning laser imaging are lost.

While the scanning prior art does include some autofocusing methods, most require additional hardware components that are dedicated to the autofocusing process. For example, U.S. Patent 5,287,272 issued to Mark R. Rutenberg et al. on February 15, 1994 and entitled "AUTOMATED CYTOLOGICAL SPECIMEN CLASSIFICATION SYSTEM AND METHOD" describes an autofocusing device for optical scanning. In addition to the apparatus required for the scanning process itself, the claimed device also includes several components dedicated to autofocusing. These include an additional light source, a bi-cell (a device with two photocells in a housing with a mask formed by a pair of openings for guiding light to the respective photocells), a differential amplifier, a piezoelectric device, and a mechanical coupling to a focusing control.

U.S. Patent 4,000,417 issued to William M. Adkisson et al. on December 28, 1976 and entitled "SCANNING MICROSCOPE SYSTEM WITH AUTOMATIC CELL FIND

AND AUTOFOCUS" also describes an autofocus system. Here again, the claimed system requires dedicated hardware for autofocusing. Another drawback of this system is that it requires a separate autofocus step each time a cell is located. Scanning and autofocusing cannot be done simultaneously. No methods or devices in the prior art allow for rapid focusing to be done during scanning without the use of dedicated hardware components. Accordingly, a method which can use existing photodetection components required for the scanning process itself to also carry out focusing during scanning would give significant advantages over the prior art. SUMMARY OF THE INVENTION

The present invention encompasses methods and apparatus for focusing and autofocusing light imaging systems so as to improve image clarity and resolution. The method utilizes programming logic responsive to absorption, scattering or emission of light from a target onto a light detector. The programming logic compares real-time detection data with previously stored data to determine the degree of focus of the light source on the target. The programming logic then adjusts the distance between the focusing objective and the target based on the degree of focusing of the light source on the target. Preferably, the invention is used in conjunction with a laser imaging system.

BRIEF DESCRIPTION OF THE DRAWINGS The objects, features and advantages of the present invention will be apparent to one skilled in the art in light of the following detailed description in which:

Figure 1 is a high level block diagram of a conventional laser imaging system. Figures 2A and 2B are photographs depicting, respectively, the intensity attenuation (absorption) measurements made by the photodetector when the light source is focused on the target and when it is not focused on the target. Figures 3 A and 3B are photographs depicting, respectively, the fluorescence or light scattering (emission) measurements made by the photodetector when the light source is focused on the target and when it is not focused on the target.

Figure 4 is a flowchart of the algorithm used to determine the optimum focal length offset for focusing the light beam on the target based on the photodetector output. DETAILED DESCRIPTION OF THE INVENTION

Before the present methods and apparatuses are described, it is to be understood that this invention is not limited to the particular apparatuses or methods described as such, which those of skill in the art can, of course, vary. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only, and is not intended to be limiting as to the scope of the present invention, which will be limited only by the appended claims.

It should be noted that, as used in this specification and the appended claims, the singular forms "a", "an" and "the" include the plural referents unless the context clearly dictates otherwise. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can be useful in the practice or testing of the present invention, preferred methods and materials are described below. All publications and patents mentioned herein are incorporated herein by reference. SCANNING LASER IMAGING During imaging operations, a laser or other light source must be focused on the target being scanned in order to obtain adequate resolution. The present invention allows this focusing to be done either manually, or preferably, automatically, during scanning. In addition, the focusing process of the present invention requires no additional hardware.

Instead, the present invention uses a computer and a beam controller or other focusing means that are already part of the imaging system, and requires only additional software.

The apparatus and methods described herein can be used to image a wide variety of objects. The invention is useful, for instance, in material sciences for material inspection. An example of this is inspection of chips in the semiconductor industry. The invention also finds use in the biological sciences. Examples of this are tissue samples, cell lines or populations, chromosomal spreads, nucleic acids, and proteins. Typically, the target being scanned is a microscope slide or Petri dish containing a biological specimen which has been stained histologically, immunochemically or fluorescently. When a laser beam with a wavelength within the absorption band of the stain falls on the cell, the resulting fluorescence, scattering or absorption can be monitored using a photodetector, such as a photomultiplier tube ("PMT") or other sensitive solid state detector. In the case of a non-absorbent target, light scattering may be measured as the indicator using any laser wavelength. As the laser beam is scanned across a line on the sample slide, the output of the photodetector will increase, decrease or remain unchanged, depending on whether the laser beam is intercepting a fluorophore stained cell, which emits light, or a color stained cell, which absorbs light, or an unstained cell, or no cell at all.

The numbers of the peaks or valleys in the signal produced by the photodetector as the laser beam scans, or the amplitude and shape changes in the signal, will vary depending on how well the laser beam is focused on the target. Focusing involves adjusting the focal length offset between the laser beam controller and the target. Focal length offset refers to the static modification to the focal length, and is independent of the dynamic modification performed automatically by a three-dimensional (3D) scanner during the scanning process. Once set, static modification or "offset" will remain constant for all pixel positions (x, y) in the scanning field before, during and after scanning. On the other hand, the dynamic focal length adjustment varies across the scanning field depending on the pixel position and occurs only during scanning. The dynamic focal length adjustment are calculated in advance for any scanning field. The static modification or offset is used to adjust the nominal focal length of a 3D scanner and to compensate for the differences in the target's thickness.

In the present invention, the output signal of the photodetector is monitored in real- time to assist in the focal length offset adjustment, which is accomplished through varying the position of the z-axis scanner, or retroreflector. Alternatively, the z-axis can be driven by a stepping motor and may be implemented on the sample stage. By accurately focusing the laser beam on the target, the accuracy and quality of imaging can be greatly improved. The present invention will now be explained more specifically in reference to the Figures. In Figure 1, the PMT 32 collects the light from the focused laser beam 16 transmitted through, or reflected by, a sample target 20. Alternatively, the absorbed or scattered light generated by the beam 16 striking the target 20 may be measured, or the fluorescent light at a set of predetermined wavelengths may be measured. The target 20 may comprise either a single light-transmissive substrate (such as glass, Plexiglas, fused silica and optically clear polymers) or a multiplicity of such substrates. On the top surface of the target 20, cells, particles or a thin layer of tissue are placed for image analysis. As the focused laser beam 40 strikes a spot on the target 20, light is scattered through a variety of angles, transmitted, reflected, absorbed, or generates fluorescence, depending on the nature of the spot on the target 20. In most cases, most rays are transmitted through the target 20 and collected by the PMT 32. These rays may then be detected in the same fashion as noted in the above-incorporated Tomei et al. patent.

The PMT 32 then passes the data representing these collected light rays as a signal out to the support computer system 34. The support computer system 34 stores the data generated by several scan lines internally. As the scanning progresses, the support computer system 34 also passes the data to image storage 38, for storage of the complete image for later playback, if desired, and to a high resolution monitor 36, where it can be viewed in real time.

A computer program running on the support computer system 34 can be used to display on the monitor 36 the signal being produced by the photodetector. The human operator can then monitor the peaks and valleys of the signal (samples of valleys can be seen in Figs. 2A and 2B, and of peaks in Figs. 3 A and 3B), and in particular, the number of peaks and valleys and the steepness of the curves of the peaks and valleys. As explained below, these factors can be used to determine the degree of focus of the laser beam 40 on the target 20. The operator can then send a signal through the support computer system 34 to the beam controller 18 to adjust the position of the z-axis scanner of the 3D beam controller, or retroreflector. By adjusting this position, the laser beam 40 can be focused on the target 20.

In addition to manual focusing, as described above, the present invention also allows for autofocusing. In most cases, autofocusing will be preferable to manual focusing, since it allows scanning to occur without human monitoring or intervention.

When doing autofocusing, a computer program running on the support computer system 34 uses an algorithm (which will be explained in detail later) to determine the degree of focus of the laser beam 40 on the target 20. This algorithm then corrects the focus by sending a signal from the support computer system 34 to the beam controller 18. Both the scanning process and the autofocusing process use the same data collected by the PMT 32. Those skilled in the art will recognize that many of the elements of the device described above can be replaced by similar elements. For example, any suitable light detector, such as a sensitive solid state detector or the like, can be used in place of the PMT 32.

Focusing In the focusing process of the present invention, the basic routine is to optimize, on a line-by-line basis, the acuity of the scanning laser beam as it scans the target 20. The method of optimization depends on the mode of acquisition (fluorescence, scattering or absorption) of the light from the target 20. For fluorescence and scattering, optimum focus occurs when the signal at the peaks (e.g., the highest points shown in Figs. 3 A and 3B) is at a maximum over the entire range of focal length offset adjustment. For absorption, optimum focus occurs when the signal at the valleys (e.g., the lowest points shown in Figs. 2A and 2B) is at a minimum over the range of focal length offset.

As will be evident to those skilled in the art, in addition to maximizing the height of peaks and minimizing the depths of the valleys in the signal, other factors can also be used as a basis for improving focus. For example, the steepness of the slope of the signal on the sides of the peaks and valleys provides a factor that can be used to determine the degree of focus, and optimizing the steepness of that slope also improves focus. Based on these types of factors to determine degree of focus, the focal length offset between the beam controller 18 and the target 20 can be adjusted. This can be accomplished either interactively by a human operator, or automatically with control software running on the support computer system 34. During interactive operation, a human operator can move the focal length offset continuously by adjusting the focus scroll bar 48 on the focus dialog box displayed on the monitor 36 within a particular adjustment range (for example, between 0 mm and 6 mm) while watching the display of the output signal from the PMT 32 on the monitor 36. For example, the operator might see a photodetector output signal like those shown in Figs. 2A, 2B, 3 A, and 3B on the monitor screen 36. Figs. 2A and 2B are photographs of monitor screen 36 displaying the photodetector output signal of a target being scanned in absorption acquisition mode. These modes provide an output signal defined by "valleys" 42a, 42b at two different focal length offset settings (as identified by the Z OFFSET indicator on the monitor control panel 46) of 2.906 mm and 3.308 mm, respectively, during the focusing optimizing operation. Similarly, Figs. 3 A and 3B show photographs of monitor screens displaying the photodetector output of a scan of a different target when in either a fluorescence or scattering light acquisition mode, this time with focal length offsets of 3.201 mm and 3.659 mm, respectively (as identified by the Z OFFSET indicator on the monitor control panel 46) . After viewing the signal on the monitor 36 for one scan line, the operator can vary the focal length offset of the laser beam 40 and observe the results. If the signal peak heights increase (during, for example, a fluorescence mode of acquisition) or the signal valleys depth decreases (during an absorption mode of acquisition), then the operator knows that he or she has improved the focus. Such improvement is shown by comparing Fig. 2B to Fig. 2A where it can be seen that the depth of the valleys has decreased when focal length offset varies from 2.906 mm to 3.308 mm. The operator can continue to adjust the focal length offset until the depth no longer appears to decrease, an indication of optimum focus. Similarly, by comparing Fig. 3B to Fig. 3A, it can be seen that the height of the peaks has increased when focal length offset varies from 3.201 mm to 3.659 mm. The operator can continue to adjust the focal length offset until the height no longer appears to increase, also an indication of optimum focus. The autofocus routine mimics the manual operations, except that the maximum peak height and minimum valley depth are identified by the computer instead of a human operator. This routine is shown by the flowchart in Fig. 4. In this example as well, the focal length offset range is from 0 mm to 6 mm. In the automatic adjustment routine, a computer program running on the support computer system 34 performs most of the required steps. The computer program initially, at step 50, splits a 1 mm span of focal length offset range into four equal intervals (representing five focal offset values: center; lower; lower middle; upper; and upper middle). The 1 mm focal adjusting span is set empirically based on the expected variation in sample thickness from one slide to another of within 0.5 mm. In any particular case where the variation is larger than 1 mm, the focal length offset optimizing routine will eventually yield the correct value. -

The computer program first sets the Center value by taking the current location of the focal offset scroll bar, which may be left there during the previous auto- or manual focusing operation. The Center value can be also set interactively based on the operator's best guess by moving the focal scroll bar 48 on the focus dialog box displayed on the monitor 36.

The computer program then sets the upper value of the focal length offset adjustment range as either the center value plus 0.5 mm, or if that result is greater than the maximum focal offset value of 6 mm, at 6 mm. Similarly, the computer program sets the lower value of the adjustment range at the center value minus 0.5 mm, or at 0 mm if it would otherwise fall below zero. Next, the computer program calculates two intermediate values, the lower middle and upper middle values. These values are set midway between, respectively, the lower and center values, and the upper and center values. Next, at step 52, the computer program then sends the appropriate signals from the support computer system 34 to have the beam controller 18 direct the beam 40 to scan the same line on the target 20 at each of these five focal offset values (the lower value, the lower middle value, the center value, the upper middle value, and the upper value). Then, the computer program acquires from PMT 32 the data it collects from each scan line, identified respectively as Data (1), Data (2), Data (3), Data (4), and Data (5).

The determination of which focal offset value provides the "best focus" will depend on the criteria chosen. A straight-forward "maximum peak height detection" routine for fluorescence and scattering, or "minimum valley depth detection" for absorption, will give a fairly good determination of the focal offset which provides the best focus. In addition, as will be evident to those skilled in the art, this method produces a focus that compensates (in an averaging sense) for astigmatism by scanning along both the x and y axes and averaging their focal offset values.

Another possible criterion for the determination of the best focal offset value is the peak (or valley) linewidth. In this case, the width of the peak (or valley) should be minimized, with the linewidth being measured at FWHM (Full Width at Half Maximum). If peak linewidth is used as the criterion, a better focus will be produced if the lines along the two independent axes are scanned, using methods that will be evident to those skilled in the art. Focusing with a laser beam of finite spot size will optimize focusing only along the scanned direction. Any astigmatism will cause the orthogonal axis (y-axis) to be slightly off focus, so for any optical system that is not free of aberrations, an optimized focus point would be a compromise between the best focus points of the two axes. It should be noted that focusing based on peak linewidth will be greatly enhanced if

Fourier transform filtering is done prior to determining the peak linewidth. As will be evident to those skilled in the art, Fourier transform filtering can be done in a variety of ways, such as including a Fast Fourier Transform computer routine within the computer program running on the support computer system 34. As a laser beam having a finite spot size is scanned across a sample line, the resultant line profile signal of peaks or valleys will be a convolution of the laser beam intensity profile of the focus spot at the beam's interception with the target profile depending on the size of the target. If the size of a cell or other features on the target is larger than or comparable to the laser beam size, the resultant peak (or valley) linewidth can be used as the criterion for determining the focus point in a relatively straight-forward manner. However, it is the relative change of the peak (or valley) linewidth that provides the measure for adjusting the focus. Using Fourier Transform filtering to remove high frequency noise contained in the acquired data should improve results since it allows the intrinsic change to be distinguished from noise. The scan line data arrays (Data (1) - Data (5)) acquired at step 52 are then processed by the computer program at step 54 to determine the current best focal offset value by using one of the criteria described above. The most straight-forward one is to find which focal offset value i generates in Data (i) the maximized valley depth for absorption or the maximized peak height for fluorescence (or scattering). For each of the five focal offset values initially set at step 50, the focal value which provides the best focus, as determined by whatever criterion is being used, is made the next center value. Then the computer program re-assigns the values for the other four focal offsets and applies a narrower focal offset adjusting range for the next loop. Essentially, three different situations can occur.

First at step 58, the computer program determines whether the best focus is at the lower middle (i.e., i = 2) or upper middle (i.e., i = 4) value. Since two cases are mirror images of each other, only the case of the lower middle value will be described here. If the best focus corresponds to i as 2, the lower middle value then becomes the next center value, and the previous lower value remains unchanged as the next lower value. The previous center value is assigned the new upper value at step 60. Two new upper middle and lower middle values are chosen as the points midway between, respectively, the new upper and center values, and the new lower and center values as set at step 72.

Second, if the best focus occurs at neither lower- nor upper-middle offset value, steps 62 & 66 then determine whether the best focus is at the lower (i.e., i = 1) or upper (i.e., i = 5) value. Here again, these are mirror images, so only the lower value case will be described here. If the best focus corresponds to i as 1, the new lower value is set at 0.9 times the old lower value and the new upper value is the previous center value, shown at step 64. The new center value is at midway between the new lower and new upper values, shown at step 72. The new lower middle value is located midway between the new lower and new center values, and likewise, the new upper middle value is located midway between the new center and new upper values. These new values are determined, respectively, by the equations at step 72.

The third case is that the best focus is at the center value, which remains as the next center value. In this case, the new lower and upper values will be assigned, respectively, the previous lower- and upper-middle values at step 70. The new lower- and upper- middle values will be determined by taking the point midway between the new center value and lower or upper value, respectively, as indicated at step 72.

The line scan data will be acquired again at the five new focal length offset values, processed and compared to determine the best focus under the narrower focal length offset adjusting range. The entire sequence is repeated using an increasingly narrower focal length offset range until the focusing is achieved at the desired level of precision. Once the best focal length offset is determined, the desired portion of the target is scanned at that focal length offset, in conjunction of the dynamic focal length adjustment, which keeps the scanning beam focused at a flat target within the scanning field.

Although the foregoing invention has been described in some detail by way of illustration and example for purposes of clarity and understanding, it will be apparent to those skilled in the art that certain changes and modifications may be practiced. Therefore, the description and examples should not be construed as limiting the scope of the invention, which is delineated by the appended claims.

Claims

What is claimed is:

1. In a light imaging system comprising a light source and focusing means to focus the light source on a target being imaged, an apparatus for improving the focus of the light source on the target comprising: a light detector for measuring the absorption, scattering or emission of light respectively through or from the target; and programming logic, responsive to the transmission, scattering or emission of light, for determining the degree of focus of the light source on the target.

2. The apparatus of claim 1 wherein the light detector is a solid state detector.

3. The apparatus of claim 1 wherein the target is a translucent microscopic slide.

4. The apparatus of claim 1 wherein the apparatus further comprises a scattered light detector for the detection of light which is scattered by the target.

5. The apparatus of claim 1 wherein the apparatus further comprises a fluorescent light detector for the detection of fluorescent light at a set of predetermined wavelengths.

6. A method of improving the focus of a light source on a target in carrying out the scanning imaging of the target, the method comprising the steps of: scanning light from the light source over the target; detecting the transmission, scattering or emission of light to generate detection data; storing the detection data; processing the detection data; comparing the detection data with previously stored data to determine the degree of focus of the light source on the target; and adjusting the focal length between the focusing objective and the target based on the degree of focus of the light source on the target.

7. The method of claim 6 wherein the comparing step comprises comparing maxima in the detection data with corresponding maxima in the previously stored data.

8. The method of claim 6 wherein the comparing step comprises comparing minima in the detection data with corresponding minima in the previously stored data.

9. The method of claim 6 wherein the comparing step comprises comparing steepness of slopes in the detection data with corresponding steepness of slopes in the previously stored data.

10. The method of claim 6 wherein the comparing step comprises comparing peak linewidth of the detection data with corresponding peak linewidth in the previously stored data.

11. The method of claim 6 wherein the comparing step comprises comparing valley linewidth of the detection data with corresponding valley linewidth in the previously stored data.