KR100941062B1 - Image Forming System and Method Using Inverse Spatial Optical Design - Google Patents

Abstract

An image forming system 10 and method 600 are provided that facilitate performing optical image formation. System 10 includes a sensor 20 with one or more receivers and an image transfer medium 30 that scales the sensors and receivers to the object FOV 54. Display 864 coupled to computer 824, memory 864, and / or sensor 20 provides storage and / or display of information related to output from the receiver to generate and / or process images, A plurality of illumination sources 60 may be used in connection with the image delivery medium 30. The image transfer medium 30 may be configured as a k-space filter 110 that correlates the pitch 116 associated with the receiver to the diffraction limit spot 50 in the object FOV 54, the pitch 116 being the object FOV. Can be mapped to the size of diffraction limit spot 50 in 54.

Description

Imaging system and methodology employing reciprocal space optical design

TECHNICAL FIELD The present invention relates generally to images and optical systems, and more particularly to systems and methods that facilitate performing image formation through image transfer media that project the characteristics of a sensor to an object field of view.

Microscopes make it easy to produce large images of small objects. Larger magnifications can be achieved by allowing light from an object to pass through two or more lenses as compared to a simple microscope with one lens. Compound microscopes have two or more converging lenses, arranged in a line of lenses, which in turn refract light. The result is more magnified images than can be magnified with either lens alone. The light illuminating the object first passes through a short focal length lens or lens group called an objective lens, and then passes some distance before passing through a long focal length lens or lens group called an eyepiece. Often a group of lenses is simply called a singular expression. Usually these two lenses are kept in a paraxial relationship with each other, so that the axis of one lens is arranged to be in the same orientation as the axis of the second lens. Determining how a highly magnified image is produced in the observer's eye is the quality of the lenses, the characteristics of the lenses, the relationships of the lenses, and the objectives' relationship to the object.

The first lens or the objective lens is usually a small lens having a very small focal length. The specimen or object is placed in the path of a light source with sufficient intensity to illuminate as desired. The objective is then lowered until the sample is very close to this focal point, not the focal point of the lens. Light that leaves the sample and passes through the objective lens produces a true inverted, magnified image behind the lens, within the microscope, generally at a point called the intermediate image plane. The second objective or eyepiece has a long focal length and is placed in the microscope such that the image produced by the objective lens is closer to the eyepiece than one focal length (ie, inside the focal point of the lens). The image from the objective is now the object for the eyepiece. When the object is inside one focal length, the second lens refracts the light to produce a virtual inverted, enlarged second image. This is the final image seen by the observer's eye.

Alternatively, common infinity space or infinity corrected design microscopes do not focus the light leaving the objective, but converge even after passing through a tube lens that causes the projected image to be in focus of the eyepiece for magnification and observation. Use an objective lens with infinite conjugation that results in parallel luminous flux. Many microscopes, such as the compound microscopes described above, are designed to provide some quality images to the human eye through the eyepiece. There is a difficulty in connecting a machine vision sensor, such as a charge coupled device (CCD) sensor, to the microscope so that the image can be seen on a monitor. This is because the image quality provided by the sensor and seen by the human eye is reduced compared to the image seen by the human eye directly through the objective lens. As a result, conventional optical systems for enlarging, observing, inspecting and analyzing small objects mostly require the careful attention of professionals who monitor the process through the eyepiece. Artificial vision or computer-based image displays from the above-described image sensor displayed on a monitor or other output display device are not only the quality perceived by the human observer through the eyepiece, but also for other reasons.

Summary of the Invention

The following presents a simplified summary of the invention in order to provide a basic understanding of certain aspects of the invention. This summary is not an extensive overview of the invention. It is not intended to identify key or critical elements of the invention or to delineate the scope of the invention. Its sole purpose is to present some concepts of the invention in a simplified form as a prelude to the more detailed description that is presented later.

The present invention relates to a system and method that facilitates performing image formation of optical image forming systems. With respect to some optical and / or image forming system parameters, a significant degree of performance improvement can be realized over conventional systems (eg, greater effective resolution magnification, greater working distance, increased absolute space). Resolution, increased spatial FOV, increased depth of field, modulation transfer function of about 1, obsolete oil immersion objectives and eyepieces). This allows the receptors of the sensor to be effectively scaled (eg, "mapped", "scaled", "matched", "to occupy the object FOV at a scale or magnitude associated with a point or spot of diffraction limit within the object FOV). Reduced ”), by fabricating an image transfer medium (eg, one or more lenses, optical fiber media, or other media) in a sensor having one or more receivers. Accordingly, bandpass filtering of spatial frequencies in what is known as free space or "k-space" is achieved such that the projected size of the receiver (projection from the sensor towards the object space) is filled in the k-space.

That is, the image transfer medium is made, configured, and / or selected such that the conversion to k-space is achieved, with a priori design decision that k-space or significant bandpass frequencies are maintained throughout and above and below k-space frequencies. Are reduced. Note that frequencies above and below the k-space frequency cause blurring and contrast reduction and are generally associated with conventional optical system designs that define inherent constraints on the modulation transfer function and "optical noise". . This also shows that the systems and methods of the present invention are contrary to or contrary to conventional geometric paraxial beam designs. As a result, the limitations of many known optical designs associated with conventional systems are alleviated by the present invention.

In accordance with one aspect of the present invention, a system and method of a "k-space" design is provided that defines a "unit-mapping" relationship of a modulation transfer function (MTF) of an object plane to an image plane. The k-space design projects pixels or receptors in the image plane onto the object plane to encourage optimal theoretical relationships. This is defined as a one-to-one correspondence between the image sensor receiver and projected object plane units (e.g., the units are defined as the minimum resolvable points or spots in the object FOV) that match according to the receiver size. A k-space design is a "unit-mapping" or "unit-" as an effective "intrinsic space filter" that means that both spectral components of an object and image are substantially coincident or quantized in k-space (also called inverse-space). "Matching" works. With the advantages provided by the k-space design, the systems and methods are incidentally related, for example, using dry objective image formation and without the use of an immersion technique having inherent limitations to the aforementioned parameters. And much larger effective resolution magnification with much increased FOV, field of view, absolute spatial resolution, and working distance is possible.

One aspect of the present invention relates to an optical system comprising an optical sensor having a light receptor array having a pixel pitch. Lenses optically associated with the optical sensor consist of optical parameters that are functionally related to the pitch and desired resolution of the optical system. In turn, the lens operates to map the portion of the object with the desired resolution along the light path to the light receptor associated.

Another aspect of the invention is directed to a method of designing an optical system. The method includes selecting a sensor having a plurality of light receptors having a pixel pitch. The resolution of the preferred minimum spot size is selected for the system and a lens consisting of optical parameters based on the pixel pitch and the desired minimum spot size or the selected existing lens is used to map a plurality of light receptors to the portion of the image along the desired resolution. Is provided.

The following description and the annexed drawings set forth in detail certain illustrative aspects of the invention. These aspects are indicative of the various ways in which the principles of the invention may be employed, but only some of these methods, and the present invention covers all such aspects and their equivalents. Other advantages and novel features of the invention will become apparent from the following detailed description of the invention when considered in conjunction with the drawings.

1 is a schematic block diagram illustrating an image forming system according to an aspect of the present invention.

2 is a diagram illustrating a k-space system design in accordance with an aspect of the present invention.

3 is a system diagram illustrating sensor receptor matching in accordance with an aspect of the present invention.

4 is a graph illustrating a sensor matching consideration according to an aspect of the present invention.

5 is a graph illustrating a modulation transfer function in accordance with an aspect of the present invention.

6 is a graph illustrating a figure of merit related to the spatial field index according to an aspect of the present invention.

7 is a flowchart illustrating an image forming method according to an embodiment of the present invention.

8 is a flowchart illustrating a method of selecting an optical parameter in accordance with an aspect of the present invention.

9 is a schematic block diagram illustrating an image forming system according to an aspect of the present invention.

10 is a schematic block diagram illustrating a modular image forming system in accordance with an aspect of the present invention.

11-13 illustrate an alternative image forming system in accordance with an aspect of the present invention.

14-18 illustrate application examples in accordance with an aspect of the present invention.

The present invention relates to optical and / or image forming systems and methods. In accordance with one aspect of the present invention, a k-space filter is provided that can be constructed from an image transfer medium such as an optical medium that correlates an image sensor receiver to an object FOV. Various light sources may be employed to achieve one or more operational goals and for diversity of applications. The k-space design of the image forming system of the present invention facilitates the capture and analysis (eg, automatic and / or manual) of images with high FOV at a substantially higher effective resolution magnification compared to conventional systems. This may include employing a low numerical aperture (NA) associated with a low magnification objective to achieve a very high effective resolution magnification. As a result, substantially large field of view (DOF) is also realized at very high effective resolution magnifications. The k-space design facilitates the use of homogeneous illumination sources that are not sensitive to position changes and thereby improves inspection and analysis methods.

According to another aspect of the present invention, the objective-to-object distance (e.g., working distance) can be maintained at low high magnification effective resolution magnification image formation and comparable (e.g. Size) typical of about 0.1 mm or more and about 20 mm or less, as opposed to conventional microscope systems that may require significantly smaller (smaller than 0.01 mm) object-to-objective distances for effective resolution magnification values Spacing can be achieved. In another aspect, the working distance is at least about 0.5 mm and at most about 10 mm. It will be appreciated that the present invention is not limited to operation at the operating distances described above. In many cases the above-mentioned working distances are employed, but in some cases more or more of these binomial distances are employed. In addition, immersion or other refractive index matching media or fluid for the objective lens at one or more effective image magnification levels of the present invention is generally not needed (eg, substantially no improvement is obtained), but is still "infinity correction". Note that it exceeds the effective resolution magnification level that can be achieved in conventional microscopy optical design variants including systems employing objective lenses.

The k-space design of the present invention allows a small "blurred circle" or diffraction limit point / spot on the object plane to correspond substantially one-to-one by "unit-mapping" of the object and image spaces with respect to the associated object and image fields. It is defined as determined by the design parameters for matching the image sensor receivers or pixels. This enables the improved performance and capabilities of the present invention. One possible theory of k-space design is that, because the free-transformation of both objects and images is formed in k-space (also referred to as "inverse space"), the sensor uses k-space through optical design techniques and component substitution in accordance with the present invention. This is from a mathematical concept that will be mapped to the object plane at. It will be appreciated that a plurality of different transformations or models may be used to construct and / or select one or more components in accordance with the present invention. For example, wavelet transform, Laplace (s-transform), z-transform and other transforms may be similarly employed.

The k-space design method is geometrical, paraxial ray-tracking, because k-space optimization facilitates the spectral components of objects (e.g., tissue samples, particles, semiconductors) and images that are identical in k-space and thus quantized. And conventional optical systems designed according to optimization theory. Therefore, there is no inherent limitation imposed on the modulation transfer function (MTF) describing the contrast versus resolution and absolute spatial resolution in the present invention. For example, quantization in k-space provides a single modulation transfer function that is not realized by conventional systems. High MTF, spatial resolution, and effective resolution image magnification can be achieved by using a low magnification objective with a low numerical aperture (e.g. a "unit-mapping") of pixels projected in the "intrinsic spatial filter" provided by the k-space design. For example, generally less than about 50x with a numerical aperture less than about 0.7).

If desired, " infinity-corrected " objectives can be employed with spectral variable components, polarization variable components, and / or contrast or phase variable components, as well as associated optical components and illumination. These components can be included in the optical path-length between the objective lens and the image lens in “infinity space”. Accordingly, optical system accessories and variants can be arranged as interchangeable modules in this geometry. In contrast to conventional microscope imagers using "infinity corrected" objectives, the k-space design enables maximum optimization of the infinite space geometry by the concept of "unit-mapping". This means that there is no specific limitation on the number of additional components that can be inserted into the "infinity space" geometry, as in conventional microscope systems, which typically specify only two additional components without optical correction.

The present invention also enables a "base-module" design that can be configured and reconfigured to use transmissive or reflected illumination if desired in the operation of a plurality of different applications, if desired. This includes virtually all typical artificial visual illumination schemes (eg darkfield, brightfield, phase-contrast), and other microscopic transmission techniques (Köhler, Ave) at substantially any offset, and epi- Lighting, and variations thereof. The system of the present invention can be employed in a robust plurality of optical-mechanical designs because the k-space design is not sensitive to environmental and mechanical vibrations and therefore does not generally require separation from heavy structural mechanical designs and vibrations associated with conventional microscope imaging devices. Can be. Other features include storage (e.g., an on-device database, image data transmission to a remote computer for storage / analysis) and display of images generated in accordance with the present invention (e.g., computer display, printer, film, and Other output media), if desired, may include digital image processing. Remote signal processing of the image data may be provided, as well as communication and display of the image data, for example, via associated data packets communicated via a network or other medium.

First, FIG. 1 illustrates an image forming system 10 according to an aspect of the present invention. Image forming system 10 includes a sensor having one or more receptors, such as pixels or individual photo detectors (see eg FIG. 3), operatively coupled to image transfer medium 30 ( 20). The image transfer medium 30 is made or configured to scale the size of the sensor 20 to an object field of view (FOV), indicated at 34, in the image plane established by the position of the sensor 20. The two-dimensional reference 36 of the X and Y coordinates is provided to illustrate the scaling or reduction of the apparent or virtual size of the sensor 20 to the object FOV 34. Directional arrows 38 and 40 illustrate the direction in which the apparent size of sensor 20 is reduced toward object FOV 34.

The object FOV 34 established by the image transfer medium 30 is related to the position of the object plane 42 that contains one or more objects during microscopy. Sensor 20 may be substantially any size, including one or more receivers of various sizes and shapes, of similar size or proportionality to each sensor to respond to light received from objects under inspection in object FOV 34. , Shape and / or technology (eg, digital sensor, analog sensor, charge coupled device (CCD) sensor, CMOS sensor, charge injection device (CID) sensor, array sensor, linear scan sensor). When light is received from the object FOV 34, the sensor 20 is sent to a storage device, such as a memory (not shown), or to a remote location, for example, if desired, substantially any intervening digital processing. It provides an output 44 that can be displayed from the memory via a computer and a combined display (eg, direct bit mapping from the sensor memory to the display). Note that in-device or remote signal processing of image data received from the sensor 20 may also be performed. For example, output 44 may be converted into electronic data packets and transmitted to a remote system via a network and / or wireless transmission system and protocol for analysis and / or presentation. Similarly, output 44 may be stored on a computer within the device prior to being sent to the next computing system for analysis and / or display.                 

The scaling provided by the image transfer medium 30 is determined by the new k-space configuration or design in the medium that augments the critical k-space frequencies that are important and reduces the frequencies outside the predetermined frequencies. This has a band pass filter effect that passes spatial frequencies within the image transmission medium 30 and in particular defines the image forming system 10 in terms of resolution rather than magnification. As will be described in detail below, the resolution of the image forming system 10 as determined by the k-space design may include, for example, other features as well as high effective resolution magnification, absolute spatial high resolution, large field of view, greater working distance, And a plurality of features in the displayed or stored image, such as having a one-value modulation transfer function. To determine k-spatial frequencies, a "pitch" or spacing between adjacent receptors on sensor 20 is determined, which pitch is related to the distance between the centers of adjacent receptors and is approximately the size or diameter of a single receptor. The pitch of the sensor 20 defines the Nyquist "blocking" frequency band of the sensor. This frequency band is augmented by the k-space frequency design and the other frequencies are reduced. To illustrate how scaling is determined in the image forming system 10, a spot or point 50 of small or diffraction limit is shown on the object plane 42. The diffraction limit point 50 represents the minimum resolvable object determined by the optical properties in the image transmission medium 30 and is described in detail below. Scaled receptor 54, shown in front of FOV 34 and having a size determined according to the pitch of sensor 20, coincides or scales to approximately the same size as diffraction limit point 50 within object FOV 34. It became.                 

That is, the size of any given receiver of the sensor 20 is effectively reduced in size through the image transfer medium 30 such that the size of (or is consistent with) the size of the diffraction limit point 50. This also has the effect of filling the object FOV 34 with all of the respective receivers of the sensor 20 that are suitably scaled to be similar in size to the diffraction limit point 50. As described below, matching / mapping sensor characteristics to the minimum resolvable object or point in the object FOV 34 defines the image forming system 10 in terms of absolute spatial resolution, thus improving the operating performance of the system.

This illumination source is provided in the present invention so that photons from an illumination source 60 can be transmitted through and / or reflected from objects in the FOV 34 so that the receptors in the sensor 20 can be activated. Can be. The present invention provides illumination if potential self-emissive objects (e.g., fluorescent or phosphorescent biological or organic material samples, metals, minerals, and / or other inorganic materials, etc.) emit sufficient radiation to activate the sensor 60. Note that the invention could potentially be employed without the circle 60. However, light emitting diodes provide an effective illumination source 60 in accordance with the present invention. Indeed, any illumination source 60 may be applied, including coherent and non-coherent illumination sources, visible and non-visible wavelengths. However, for non-visible wavelength illumination sources, the sensor 20 will have to be adapted accordingly. For example, in the case of an infrared or ultraviolet light source, the infrared or ultraviolet sensor 20 will be employed respectively. Other illumination sources 60 include wavelength specific lighting, broadband lighting, continuous light, strobe light, Kohler lighting, Abe lighting, phase-contrast lighting, darkfield lighting, brightfield Illumination, and Epi illumination. Transmissive or reflective illumination techniques (eg, reflection and divergence) may also be applied.

In FIG. 2, system 100 illustrates an image transfer medium in accordance with an aspect of the present invention. The image transfer medium 30 shown in FIG. 1 is configured and / or in accordance with the k-space design concept described above, in particular to promote k-space frequencies 114 of a given band and to reduce frequencies outside of this band. Can be provided through the selected k-space filter 110. This determines the pitch "P" which is the distance between adjacent receptors 116 in the sensor (not shown) and the optical medium in the filter 110 such that the pitch "P" magnitude of the receiver 116 matches the diffraction limit spot 120. Is achieved by sizing. Diffraction limit spot 120 may be determined from the optical properties of the medium in filter 110. For example, the numerical aperture of an optical medium such as a lens defines the minimum object or spot that can be resolved by the lens. The filter 110 performs a k-space transformation such that the magnitude of the pitch is effectively “unit-mapped”, projected, corrected and / or reduced so as to effectively match the size or scale of the diffraction limit spot 120.

It will be appreciated that multiple optical configurations may be provided to achieve k-spatial filter 110. One such configuration can be provided by an aspherical lens adapted to perform k-space transformation and reduction of sensor space into object space. Another configuration can be provided by a plurality of lens configurations, the lens combination being selected to provide filtering and scaling. Another configuration may employ an optical fiber taper 132 or an image conduit, wherein the plurality of optical fibers or optical fiber arrays are configured in a funnel shape to perform mapping of the sensor to the object FOV. Note that the optical fiber taper 132 is generally in physical contact between the sensor and the object under inspection (eg, contact with a microscope slide). Another possible k-spatial filter 110 configuration employs a holography (or other diffraction or image structure) optical element 136, which provides a hologram (or other diffraction or image) to provide other mapping to the present invention. (phase structure) (e.g., computer generated, optically generated and / or other methods) to form a substantially flat optical surface.

The k-space optical design enabled by the k-space filter 110 is based on the "projected effective pixel-pitch" of the sensor, which means that the physical size of the sensor array elements follows the object plane through the optical system ( "Projection") is the derived value. Therefore, conjugate planes and optical transform spaces correspond to the Nyquist cutoff frequency of the effective receptor or pixel size. This maximizes effective resolution image magnification and FOV as well as field of view and absolute spatial resolution. A novel optical theory application is provided that does not rely on conventional geometric optical design parameters of paraxial ray tracing to determine a combination of conventional optics and image formation. This will be described later.

A free transform of objects and images is formed in k-space (also called "inverse-space") (by means of an optical system). For example, the optical media employed in the present invention is a relatively inexpensive standard " offset " having a configuration that defines " unit-mapping " or " unit-matching " for almost all image and object fields. off-the-shelf) "components. The small blurry circles or diffraction limit spots 120 of the object plane are defined by a design that matches the pixels in the image plane (e.g., of the optimally selected image sensor) in a nearly one-to-one correspondence, and thus The free transform can be matched. This means that by design optically the blurred circles are scaled to approximately the same size as the receptor or pixel pitch. The present invention is defined as configuring a true space filter, such as k-space filter 110. With this design definition and implementation, the spectral components of both the object and the image in k-space can be nearly identical or quantized. It also defines that the sensor's Modulation Transfer Function (MFT) (contrast versus spatial resolution) matches the MTF of the object plane.

3 illustrates an optical system 200 according to one aspect of the present invention. System 200 includes a sensor 212 having a plurality of receptors or sensor pixels 214. For example, sensor 212 may have an M by N sensor pixel array 214 having M rows and N columns (e.g., 640 x 480, 512 x 52, 1280 x 1024), where M and N are integers, respectively. to be. While generally a rectangular sensor 212 with square pixels is shown, it will be appreciated that the sensor can be substantially any shape (eg, circular, elliptical, hexagonal, square, etc.). Also, it will be appreciated that each pixel 214 in the array may be substantially any shape or size and that pixels in any given array 212 are similar in size and shape in accordance with one aspect of the present invention.

The sensor 212 can be any arbitrary technology (eg, digital sensor, analog sensor, charge coupled device (CCD) sensor, CMOS sensor, charge injection device (CID), including one or more receptors (or pixels 214). Sensor, array sensor, linear scan sensor). According to one aspect of the invention, each pixel 214 is of similar size or proportionality and responds to light (eg, visible light, non-visible light) received from objects under inspection as described herein.

Sensor 212 is coupled to lens network 216 that is configured based on the performance requirements of the optical system and the pitch size of sensor 212. The lens network 216 scales (or projects) the size of the sensor of the image plane (e.g., pixel 214) to the object FOV 220, according to one aspect of the invention, based on the position of the sensor 212. Act). Object FOV 220 relates to the position of object plane 222 that includes one or more objects (not shown) during inspection.

When sensor 212 receives light from object FOV 220, sensor 212 is sent to a storage device, such as a memory (not shown) or to a remote location, for example, if desired, substantially any intermediate. (intervening) provides an output 226 that can be displayed from memory via a computer and combined display, without digital processing (eg, direct bit mapping from sensor memory to the display). Note that in-device or remote signal processing of image data received from sensor 212 may also be performed. For example, output 226 may be converted into electronic data packets and sent to a remote system via a network for analysis and / or display. Similarly, output 226 may be stored on a computer within the device prior to being sent to the next computing system for analysis and / or display.                 

The scaling (or effective projection) of the pixel 214 provided by the lens network 216 is determined by the novel k-space configuration or design according to one aspect of the present invention. The k-space design of lens network 216 increases important certain k-space frequencies and reduces frequencies outside certain frequencies. This has a band pass filter effect that passes spatial frequencies within the lens network 216 and specifically defines the image forming system 200 in terms of resolution rather than magnification. As will be described in detail below, the resolution of the image forming system 200 determined by the k-space design, as well as other features, such as high "effective resolution magnification" (the figure of merit described below), relative absolute space Enhances multiple features of the displayed or stored image, such as having high resolution, large field of view, large working distance, and a one-value modulation transfer function.

To determine k-space frequencies, a “pitch” or spacing 228 between the centers of adjacent receptors 214 on the sensor 212 is determined. The pitch (eg pixel pitch) corresponds to the distance between the centers of adjacent receptors, indicated at 228, which is the size or diameter of a nearly single receptor when the sensor includes all equally sized pixels. Pitch 228 defines the Nyquist “blocking” frequency band of sensor 212. It is this frequency band that is augmented by the k-spatial frequency design and other frequencies are reduced. In order to illustrate how scaling is determined in the image forming system 10, the preferred minimum resolvable spot size point 230 is illustrated in the object plane 222. For example, point 230 may represent a minimum resolution object that is determined by the optical properties of lens network 216. That is, the lens network provides optical characteristics (e.g., magnification, aperture) that allow each pixel 214 to be matched or scaled to approximately the same size in the object FOV 220 as the preferred minimum resolution spot size of the point 230. Number). For illustrative purposes, scaled receptor 232 is shown in front of the object FOV as having a size that depends on the pitch 228 of sensor 212, which is approximately the same as point 230.

By way of example, the lens network 216 is each predetermined receiver (e.g., pixel 214) to be about the same size as point 230, which is typically the smallest spot size resolvable by system 210. The point 230 is sized to represent a minimum resolution object determined by the optical properties in the lens network determined by the diffraction law (eg, diffraction limit spot size). Thus, the lens network 216 may be designed to effectively scale each pixel 214 of the sensor 212 to any size that is at or above the diffraction limit magnitude, for example, a resolvable spot. The size can be chosen to provide any desired image resolution that meets these criteria.

After selecting the desired resolution (resolution spot size), the lens network 216 is designed to provide a magnification that scales the pixel 214 to the object FOV 220 size accordingly. This has the effect of filling the object FOV 220 with all of the respective receptors of the sensor 212 that are suitably scaled to be similar in size to the point 230 corresponding to the desired resolvable spot size. Matching / mapping sensor characteristics to the desired (eg minimal) minimum resolvable object or point 230 within object FOV 220 defines image forming system 200 in terms of absolute spatial resolution, and In one aspect, it improves the operating performance of the system.

As another example, assume that the sensor array 212 provides a pixel pitch 228 of about 10.0 microns in order to provide unit-mapping in accordance with this example. Lens network 216 includes an objective lens 234 and a secondary lens 236. For example, the objective lens 234 can be set to the secondary lens at infinity conjugate while allowing flexibility in the distance between the objective lens and the secondary lens. Note that substantially all of the pixels 214 are projected onto the objective FOV 220 defined by the objective lens 234. For example, each pixel 214 is scaled through the objective lens 234 to approximately the same size as the desired minimum resolution spot size. In this example, the preferred resolution at image plane 222 is 1 micron. Accordingly, a magnification of 10 times acts to project the 10 micron pixel back onto the object plane 222 to reduce it to the size of 1 micron.

The size reduction of the array 212 and associated pixels 214 selects the transfer lens 236 to have a focal length "D2" (from array 212 to transfer lens 236) of about 150 millimeters, and the objective lens. For example, it may be achieved by selecting to have a focal length "D1" (from objective 236 to object plane 222) of about 15 millimeters. Therefore, the pixel 214 is effectively reduced in size to about 1.0 micron per pixel, thereby matching the size of the preferred resolution spot 230 and filling the object FOV 220 with a "virtually reduced" pixel array. do. It will be appreciated that other arrangements of one or more lenses may be employed to provide the desired scaling.                 

In view of the foregoing, those skilled in the art have described optical media (eg, lens network 216) in accordance with an aspect of the present invention, wherein "objects and image spaces are" units "for substantially all images and object FOVs. It will be appreciated that one can design with relatively inexpensive standards of "off-the-shelf" components having a configuration defining "mapping" or "unit-matching". Lens network 216, and in particular objective 234, performs a free transformation of objects and images in k-space (called "inverse-space"). This transformation is taken for image optimization by the k-space design of the present invention.

Small blurry circles or airy disks on the object plane are defined by a design that matches the pixels in the image plane (e.g., of the optimally selected image sensor) to the airy disk in a nearly one-to-one correspondence and thus pixelation. The transform can be matched to the free of the array. This means, by design, that the airy disk is scaled through the lens network 216 to be approximately the same size as the receptor or pixel pitch. As mentioned above, lens network 216 is defined to construct an intrinsic spatial filter (eg, a k-space filter). With this design definition and implementation, the spectral components of both the object and the image in k-space can be nearly identical or quantized. It can also match the Modulation Transfer Function (MTF) of the sensor (contrast versus spatial resolution) to the MTF of the object plane according to one aspect of the present invention.

As shown in FIG. 3, k-space is defined as the area between the objective lens 234 and the secondary lens 236. Any optical medium, lens type and / or lens combination that scales and / or projects the object array 212 to the object FOV 220 according to unit or k-space mapping as described herein is the scope of the present invention. You will know that you are within.

To illustrate the novelty of the illustrated lens / sensor combination shown in FIG. 3, conventional objectives of size according to conventional geometric paraxial beam techniques are generally provided by magnification, numerical aperture, and focal length and other objective lenses. Note that the size is determined according to the parameter. Thus, the objective lens will be sized to have a longer focal length than the subsequent lens that approaches or approaches the sensor (eyepiece in a conventional microscope) to provide magnification of small objects. This allows the magnification of a small object on the object plane to be projected as an enlarged image of the objects over the "parts" of the sensor and, among other problems of the sensor, known detail blur (e.g. Ray-ray diffraction on optics) And other limitations), co-power problems, and Nyquist aliasing. The k-space design of the present invention serves as an alternative to conventional geometric paraxial beam design principles. That is, the objective lens 234 and the secondary lens 236 act to provide a reduction in the size of the sensor array 212 to the object FOV 220, as indicated by the relationship of the lenses.

The illumination source 240 is provided in the present invention so that photons from the illumination source 240 can be transmitted through and / or reflected from objects in the FOV 234 so that the receptors in the sensor 212 can be activated. Can be. The present invention is potentially useful without the illumination source 240 if the potential self-emissive objects (e.g., an object or sample having emission characteristics as described above) emit sufficient radiation to activate the sensor 12. Note that it may be employed. Indeed, any illumination source 240 may be applied, including coherent and non-coherent illumination sources, visible and non-visible wavelengths. However, for non-visible wavelength illumination sources, a suitable sensor 212 will be used. For example, in the case of an infrared or ultraviolet light source, an infrared or ultraviolet sensor 212 may be employed respectively. Other illumination sources 240 may include wavelength specific illumination, broadband illumination, continuous light sources, strobe light sources, Kohler illumination, ABE illumination, phase-contrast illumination, darkfield illumination, brightfield illumination, and epi illumination. Transmissive or reflected illumination techniques may also be applied.

4 shows a mapping graph and a comparison graph 300 between the pixel size projected on the X axis and the diffraction limit spot resolution size “R” on the Y axis. The vertex 310 of the graph 300 corresponds to the unit mapping between the projected pixel size and the diffraction limit spot size, representing an optimal relationship between the lens network and the sensor according to the present invention.

It will generally be appreciated that objective lens 234 (FIG. 3) should not be chosen such that the diffraction limit magnitude "R" of the minimum resolution objects is smaller than the projected pixel size. Doing so may result in "economic waste" resulting in more accurate information being lost (eg, choosing an objective lens that is more expensive than necessary, such as having a larger numerical aperture). This is shown to the right of the dividing line 320 with reference 330, which shows the projected pixel 340 larger than the two small diffraction spots 350. Conversely, when the objective lens is selected such that the diffraction limit performance is made larger than the projected pixel size, blurring and empty magnification may occur. This is shown to the left of line 320 at 360, where the projected pixel 370 is smaller than the diffraction limit object 380. However, even though substantially one-to-one correspondence is not achieved between the projected pixel size and the diffraction limit spot, the system is less than optimal (e.g., of line 320 from vertex 310 on graph 300). It will be appreciated that 0.1%, 1%, 2%, 5%, 20%, 95%) going down the left or right side may still provide suitable performance in accordance with one aspect of the present invention. Therefore, less than optimal matching is included within the spirit and scope of the present invention.

For example, as shown in FIG. 3, the diameter of the lens in the system is determined by the critical spatial frequencies (e.g., pixel size and It will be appreciated that the frequencies used to define the shape must be of a magnitude that does not substantially attenuate. In general this means that a larger diameter lens (eg 10 to 100 millimeters) should be chosen to ensure that important spatial frequencies are not attenuated.

5 shows a modulation transfer function 400 in accordance with the present invention. A modulation percentage of 0 to 100% is shown on the Y axis, which defines the percentage of contrast between black and white. On the X axis, absolute spatial resolution is shown in micron intervals. Line 410 indicates that the modulation percentage is constant at about 100% for varying spatial resolutions. Thus, the modulation transfer function is about 1 for the present invention up to the limit imposed by the sensor's signal-to-noise sensitivity. For illustrative purposes, the modulation transfer function of a conventional optical design is shown as a line 420, which can be an exponential curve that is generally characterized by an asymptotic range that generally decreases the modulation percentage (contrast) to a decrease in spatial resolution. .                 

FIG. 6 shows the quantizable figure of merit of the present invention dependent on two main factors: absolute spatial resolution (RA in microns) on the Y axis of the graph 500 and FOV (F in microns) on the X axis. FOM). A suitable FOM called "spatial field number" (S) can be expressed as the ratio of these two previous quantities, with larger S values being preferred for image formation as follows.

S = F / RA

Line 510 indicates that it is in a nearly constant state over the FOV and for different values of improved absolute spatial resolution over conventional systems.

8, 14, 15, and 16 illustrate methods for facilitating image forming according to the present invention. For the purpose of simplicity of explanation, the methods may be shown and described in a series of steps, the invention being not limited by the order of the steps and in accordance with the invention some steps in different orders and / or shown and It will be appreciated that the same may be done with other steps as described. For example, those skilled in the art will appreciate that alternatively the method may alternatively be represented as a series of interrelated states or events, such as in a state diagram. In addition, steps not shown in the entirety may be necessary to implement the method according to the invention.

Proceeding to 610 of FIG. 7, lenses having diffraction limit characteristics at approximately the same size of pixels are selected to provide unit-mapping and optimization of the k-space design. At 614, the lens characteristic is also selected so that spatial frequencies are not reduced in k-space. As mentioned above, this generally means that larger diameter optics are selected to mitigate the attenuation of the desired k-space frequencies, which is important. At 618, pixels with a pitch "P" in the image plane formed by the position of the sensor fit the object FOV to a size that is nearly diffraction limit spot (eg, unit-mapped) within the object FOV, depending on the pitch. The lens configuration to be scaled is selected. At 622, the image outputs data from the sensor for real-time monitoring and / or displays the data directly on a computer display and / or subsequently stores the data in memory for analysis in image processing and / or memory in the device or remotely. Is generated.

8 illustrates a method that may be employed to design an optical / image forming system in accordance with an aspect of the present invention. The method starts at 700 selecting a suitable sensor array for the system. The sensor array typically includes a receiver pixel matrix having a known pitch size as determined by the manufacturer. The sensor can be substantially any shape (eg, square, round, square, triangular, etc.). By way of example, assume that a sensor of 640 x 480 pixels with a pitch size of 10 μm is selected. It will be appreciated that the optical system may be designed for any type and / or size of sensor array in accordance with an aspect of the present invention.

Next, at 710, the image resolution is determined. Image resolution corresponds to the minimum desired resolution spot size in the image plane. Image resolution may be determined based on the application (s) the optical system is being designed, such as any resolution that is at least the magnitude of the minimum diffraction limit. Thus, it will be appreciated that there are selectable design parameters that can be tailored to provide the desired image resolution for virtually any type of application. In contrast, most conventional systems tend to limit the resolution in accordance with Rayleigh diffraction, which means that the lens's native spatial resolution cannot exceed the diffraction limit for a given wavelength.

After selecting the desired resolution (710), a suitable amount of magnification is determined at 720 to achieve this resolution. For example, the magnification is functionally related to the pixel pitch of the sensor array and the minimum resolvable spot size. The magnification M can be expressed as follows.

M = x / y equation (1)

x is the pixel pitch of the sensor array,

y is the preferred image resolution (minimum spot size).

Thus, in the above example assuming that the pixel pitch is 10 mu m and the preferred image resolution is 1 mu m, equation (1) provides an optical system of 10 magnification. That is, it is configured to reverse project each 10 μm pixel onto the object plane to reduce each pixel to a 1 micron resolution spot size.

The method of FIG. 8 also includes the determination of the numerical aperture at 730. The numerical aperture NA is determined according to the established diffraction law which relates the NA of the objective lens to the minimum resolvable spot size determined at 710 for the optical system. By way of example, the calculation of NA may be based on the following equation.

NA = 0.5 x λ / y equation (2)                 

λ is the pixel pitch of the sensor array,

y is the preferred image resolution (minimum spot size).

If the optical system continues the example with a resolved spot size of y = 1 micron, assuming a wavelength of about 500 nm (eg green light), NA = 0.25 satisfies equation (2). Note that a relatively inexpensive 10x objective lens on the market provides a numerical aperture of 0.25.

It will be appreciated that the relationship between NA, wavelength and resolution shown in equation (2) can be expressed in different ways depending on various factors that reveal the action of the objective lens and the condenser lens. Thus, in accordance with an aspect of the present invention, the crystal at 730 is not limited to any particular formula, but merely follows known general physical laws that the NA is functionally related to wavelength and resolution. After the lens parameters are designed according to the selected sensor 700, the corresponding optical components can be arranged to provide the optical system 740 according to one aspect of the present invention.

For illustrative purposes, it is assumed that an example optical system made according to the method of FIG. 8 is employed for microscopic digital image formation. In comparison, in conventional microscopy, hundreds of magnifications are usually required to image and resolve structures up to 1 micron (and less) in size. The basic reason for this is that optics are typically designed for situations when the selected sensor is the human eye. In contrast, the method of FIG. 8 designs an optical system with respect to the sensor, which allows for significant performance improvements at reduced cost.                 

In the k-space design method, according to one aspect of the invention, the optical system is designed for individual sensors having a known fixed size. As a result, the method can provide a much easier, robust and inexpensive optical system design method of “back projecting” the sensor size into the object plane and calculating the magnification. The second part of the method allows the optics providing the magnification to easily have a NA sufficient to optically resolve spots of similar size as the back projected pixels. Optical systems designed according to one aspect of the present invention have the advantage of using cumtom and / or ready-made components. Thus, in this example, inexpensive optics can be used in accordance with an aspect of the present invention in obtaining a suitable result, with the corrected microscope optics being relatively inexpensive. According to the present invention, when custom designed optics is used, the range of acceptable magnifications and numerical apertures becomes quite large, and certain performance gains can be realized over the use of ready-made optical components.

In view of the concepts described above with respect to FIGS. 1-8, a plurality of related image forming applications may be enabled and improved by the present invention. For example, these applications are not limited to the following, but include, but are not limited to, image forming, control, inspection, microscopy and / or other automated analysis, such as

(1) biomedical analysis (eg, cell colony counting, histology, freezer, cytology, hematology, pathology, oncology, fluorescence, interference, phase and many other clinical microscopy applications);

(2) particle size measurement applications (eg, drug manufacturers, paint manufacturers, cosmetic manufacturers, food process engineering, and others);                 

(3) air quality monitoring and suspended particulate measurement (eg, cleanroom proof, environmental proof, etc.);

(4) other requirements for optical defect analysis and high resolution microscopy of transmissive and opaque materials (such as in metallurgy, automated semiconductor inspection and analysis, automatic vision systems, 3-D image formation, etc.); And

(5) image forming technologies such as cameras, copiers, fax machines, and medical systems

And the like.

9, 10, 11, 12, and 13 illustrate possible example systems that may be configured employing the concepts described above with respect to FIGS. 1-8. 9 is an optical path flow diagram in an image forming system 800 constructed in accordance with the present invention.

System 800 employs a light source 804 that emits illumination light received by collector 808. The output from the condenser 808 can be sent to the microscope condenser lens 816 by a fold mirror 812, which projects the illumination light onto the slide stage 820, and the object (not shown). , Located on or in the slide stage) can be imaged in accordance with the present invention. Slide stage 820 is automatically positioned via computer 824 and associated slide feed 828 to image one or more objects within the FOV formed by objective lens 832. The objective lens 832 and / or other components shown in the system 800 may be manually adjusted to achieve different and / or desired image characteristics (e.g., magnification, focus, object to appear in the FOV, depth of field, etc.). And / or computer 824 and associated controls (not shown) (eg, servo motors, tube slides, linear and / or rotary position encoders, optical, magnetic, electronic, or other feedback mechanisms, control software, etc.) Note that it may be adjusted automatically via.

Light output to the objective lens 832 may be sent through an optional beam splitter 840, which alternatively includes an optical shaping optic 844 and an associated light source 848. 842 (for illuminating objects from above the slide stage 820). Light passing through the beam splitter 840 is received by the image forming lens 850. Output from the image forming lens 850 can be sent to the CCD or other image forming sensor or device 854 via the fold mirror 860. The CCD or other image forming sensor or device 854 converts light received from the object into digital information for transmission to the computer 824 and the object image may be displayed to the user in real time and / or in memory at 864. Can be stored in. As discussed above, digital information forming an image captured by a CCD or other image forming sensor or device 854 may be sent by the computer 824 to the display / memory 864 as bit-map information. If desired, image processing such as automatic comparison with a given sample or image may be performed to determine and / or analyze the identity of the object under inspection. This may include the adoption of virtually any type of image processing technology or software that can be applied to image data captured in memory 864.

10 is a system 900 showing a modular approach to an image forming design in accordance with an aspect of the present invention. The system 900 may be based on a sensor array 910 (eg, installed in a conventional camera) having a pixel pitch of approximately 8 microns (or other size), where the array sizes are from 640 x 480 to 1280 x 1024 (or any other size as above). System 900 includes a modular design in which each module is substantially separate from the other modules, with alignment tolerances being relaxed.

Modules are

. A camera / sensor module 914 including an imaging lens 916 and / or a fold mirror 918;

. epi-lighting module 920 for insertion into k-space region 922;

. Sample maintenance providing module 924;

. A light shaping module 930 including a light collector 924;

. Sub-stage lighting module 940.

Note that the system 900 may employ, for example, the following commercially available components.

. Condenser optics 934 for providing light (NA <= 1)

  . (E.g. Olympus U-SC-2)

. For a given magnification, the magnification and numerical aperture selected such that the pixel pitch projected on the object plane satisfies the desired characteristic, which is similar in size to the spot resolved by the diffraction limit of the optics, for example (4x, 0.10), (10x, 0.25), (20x, 0.40), (40x, 0.65) standard plan / achromatic objective 944.                 

The system 900 may, for example, have auxiliary and / or additional optical components, modules, filters, such as when the imaging lens 916 is an f = 150 mm achromatic triplet. An infinity-space (k-space) between the objective lens 944 and the image forming lens 916 is used to facilitate insertion of the back into the k-space region at 922. In addition, an infinity-space (k-space) between the objective lens 944 and the image forming lens 916 may be provided for easily injecting light (via the light forming path) into the optical path for epi-lighting. . For example, the light forming path for epi-lighting,

. A light source 950 such as an LED driven from a stable current supply;

  . (For example, HP HLMP-CW30)

. A transmission hologram for source homogenization and for imposing a spatial virtual circle 950;

  . (E.g., POC optical form diffuser polyester film 30 degrees FWHM)

. A variable aperture 960 for reducing the effect of scattered light entering the image forming optical path by limiting the NA of the source 950 to the NA of the image forming optics;

   . (E.g. Door Lapse Iris Diaphragm SM1D12 05-12.0mm Aperture)

. A collecting lens 960 that is employed to maximize the light collected from the virtual circle 950 and to match the k-space characteristics of the virtual circle to that of the image forming optics; And

  . (E.g. f = 50mm aspherical lens, f = 50mm achromatic bifocal lens)

. Partially reflective beam splitter 964 employed to form a coaxial optical path and image path. For example, optics 964 provides 50% reflectivity at the first side (45 degree inclination), and the second side has a broadband refractive index control coating.                 

The sub-stage lighting module 940 is provided by a configuration similar to, for example, the epi-lighting configuration described above.

. A light source 970 (LED driven from a stable current supply);

  (For example, HP HLMP-CW30)

. Transmission holograms (coupled to light source 970) for source homogenization and imposition of spatial virtual sources;

  (E.g., POC optical form diffuser polyester film 30 degrees FWHM)

. A collecting lens 974 that is employed to maximize the light collected from the virtual circle 950 and to match the k-space characteristics of the virtual circle to that of the image forming optics;

  . (E.g. f = 50mm aspherical lens, f = 50mm achromatic bifocal lens)

. A variable aperture 980 for reducing the effect of scattered light entering the image forming optical path by limiting the NA of the source 970 to the NA of the image forming optics;

   . (E.g. Door Lapse Iris Diaphragm SM1D12 05-12.0mm Aperture)

. A mirror 988 that provides 90 degrees of diverting the light path to provide fine control for accurately aligning the optical modules; And

. A relay lens (not shown) which is employed to accurately position the image of the variable aperture 980 in the object plane (on the slide 990), to properly position the holographic diffuser and to achieve Kohler illumination.

  . (E.g. f = 100 nm plano-convex lens).

As noted above, computer 994 and associated display / memory 998 are provided to display in real time and / or to store / process digital image data captured in accordance with the present invention.

11 illustrates a system 1000 in accordance with an aspect of the present invention. In this aspect, the sub-stage illumination module 1010 (e.g. Kohler, Ave) can project light through the transmissive slide 1020 (object under inspection not shown), and the achromatic objective lens ( 1030 receives light from the slide and sends the light to image capture module 1040.

12 illustrates a system 1100 in accordance with an aspect of the present invention. In this aspect, the top-stage or epi-illumination module 1110 can project light onto an opaque slide 1120 (object under inspection not shown), and may be an objective lens 1130 (compound lens device or other type). Can receive the light from the slide and send the light to the image capture module 1040. As noted above, the objective lens 1130 and / or the slide 1120 may be controlled manually and / or automatically to locate the object (s) being inspected and / or to position the objective lens. FIG. 13 illustrates a system 1200 similar to system 1000 in FIG. 11 except that composite objective 1210 is employed in place of a chromogenic lens.

The image forming systems and processes described above with respect to FIGS. 1-13 may be employed to capture / process an image of a sample, where the image forming systems read the image generated by the image forming systems and read this image into many current memory technologies. They are coupled to a processor or computer to compare the various images in the onboard data storage.                 

For example, the computer may include analytical components to perform the comparison. Some of the many algorithms used in image processing are not limited to these, but convolution (and many others based thereon), FFT, DCT, thinning (or skeletal), edge detection, and contrast enhancement Include. They are typically implemented in software but may use dedicated hardware for speed. Fast Fourier Transform (FFT) is an algorithm that calculates a Fourier transform of a set of discrete data values. For example, given a finite set of data points, a periodic sampling taken from an actual signal, the FFT represents the data in its component frequencies. It also deals with essentially the same inverse relationship to reconstruct a signal from frequency data. Discrete cosine transform (DCT) is a technique for representing a waveform as a sum of weighted cosines. For example, there are a variety of existing programming languages designed for image processing, including but not limited to IDL, Image Pro, Matlab, and many others. There is no particular limitation on particular custom image processing algorithms that can be written to perform functional image manipulation and analysis.

The k-space design of the present invention makes it possible to optically correlate pre-frequency information contained in an image directly to stored information in order to perform optically correlated image processed analysis of a given sample object in real time.

14 illustrates an application 1300 for measuring particle size that may be used in the systems and processes described above. Particle size measurements may include closed / open loop monitoring, manufacturing, and control of particles in real time for particle sizes that are automatically determined according to the k-space design concepts described above. This will include automated analysis and detection techniques for particle identification of particles having similar or different sizes (n different sizes, n is an integer) and m shaped / size particles (m is an integer). Can be. In one aspect of the invention, the desired particle size detection and analysis can be achieved through a direct measurement approach. This means that the absolute spatial resolution per pixel is directly related to the imaged particles, in linear units of measurement, without substantial consideration of the particle medium and associated particle distribution. In general, direct measurements do not create a model, but provide the metrology and morphology of the imaged particles in any given sample. This reduces the processing of modeling limits exhibited by modeling algorithms, statistical algorithms, and other current technologies. Thus, the problem is one of sample handling and form that improves the accuracy and precision of the measurement, if desired, since the particle data is directly imaged and measured rather than modeled.

Proceeding to 1310 of the particle size measurement application 1300, particle size image parameters are determined. For example, the basic device design may be configured to form an image at a desired absolute spatial resolution and effective resolution magnification per pixel as described above. These parameters determine, for example, FOV, field of view (DOF), and working distance (WD). Real time measurement can be achieved by asynchronous image formation of the medium at a selected timing interval, at a common video rate, and / or at a desired image capture rate in real time. Real-time image formation may be accomplished by capturing images at a selected time for subsequent image processing. Asynchronous image formation can be achieved by capturing images at the selected time by sending instrument illumination and duty cycle pulses at the selected time for subsequent image processing.

At 1320, a sample introduction process is selected for automated (or manual) analysis. Samples may be introduced into any of the following image forming processes in (but not limited to) the image forming apparatus fabricated in accordance with the present invention.

1) All the methods and permeable media described above.

2) Individual hand samples in cuvettes, slides, and / or permeable media.

3) for example, a continuous flow of particles in a gas or liquid stream.

4) If the image forming apparatus is configured as a reflective image forming apparatus, the samples may be opaque and may be on an opaque "carrier" (automatic and / or manual) regardless of the material being analyzed.

At 1330, configure a process control and / or monitoring system. Real-time, closed loop and / or open loop monitoring, manufacturing (eg, closed loop based on particle size), and particle characteristics (eg, size, shape, shape, cross section, distribution, density, specific mass deviation, And control of the process by direct measurement of other parameters can be determined automatically). Although direct specific techniques may be performed on a given particle sample, it will be appreciated that this automated algorithm and / or process may be applied to the imaged sample if desired. In addition, particle characterization apparatus based on direct measurement can be installed at any given point in the manufacturing process to monitor and communicate particle properties for process control, quality control, and the like by direct measurement.                 

At 1340, a plurality of different sample types can be selected for analysis. For example, particle samples in any of the above-described forms may be used as part of a closed-feedback-loop system of a process (open loop if desired) to control, record, and / or communicate particle properties of a given sample type. Techniques may be incorporated) into the device in continuous flow, periodic, and / or asynchronous processes for direct measurement. In asynchronous and / or synchronous, asynchronous defines an image formation by a triggering signal sent by an event or by a trigger signal initiated by an object generating a trigger signal to initiate an event or image formation, whereby synchronization is used to trigger illumination. Defines the image formation by the sent timing signal. Asynchronous and / or synchronous image formation can be accomplished by pulsed to the illumination source to substantially match the desired image field of any particle flow rate. This allows the image sensor to “flash” on and off solid state illumination at a predetermined duty cycle so as to capture, display and record an image for processing and analysis, for example by a computer and / or mechanical, optical and / or It can be controlled by an electronic "trigger" mechanism. This provides an easy illumination and image forming process if it can be effectively timed to stop the activity of the flowing particles in the medium or rather to stop the movement of the particles. This also allows the sample in the image field to capture particles in the field for subsequent image processing and analysis.

Real-time (or near real-time), closed-loop and / or open-loop monitoring, fabrication, and control of processes by k-space based direct measurement of particle characterization at 1340 are not limited to, but are not limited to, ceramics, metal powder, pharmaceuticals. Based on cement, minerals, ores, coatings, adhesives, pigments, dyes, carbon blacks, filter materials, explosives, food preparation, health and cosmetic emulsions, polymers, plastics, micelles, beverages, and many particle-required monitoring and control Applicable to a wide range of processes involving materials.

Other applications include, but are not limited to:

. Instrument calibration and standards;

. Industrial-sanitary research;

. Substance research;

. Energy and combustion studies;

. Diesel and gasoline engine exhaust measurement;

. Industrial emissions sampling;

. Basic aerosol studies;

. Environmental research;

. Bio-aerosol detection;

. Pharmaceutical research;

. Health and agricultural experiments;

. Inhalation poison; And / or

. Filter Testing.

At 1350, software and / or hardware based computerized image processing / analysis may be performed. Images from a device made in accordance with the present invention may be processed according to virtually any hardware and / or software process. Software-based image processing can be accomplished with commercially available software because dedicated software and / or image file formats are digital formats (eg, bitmaps of captured particles). Analysis, characterization, and the like may be provided by: For example, the analysis can be based on measurements (based on direct measurements) and / or comparisons (databases). Comparative analysis may include a comparison with a database of image data for known particles and / or their separate bodies. Advanced image processing can characterize and classify images in real time and / or with periodic sample measurements. The data can be discarded and / or recorded when desired, and data matching the known sample characteristics can, for example, initiate a suitable selected response. Moreover, devices constructed in accordance with the present invention can be linked for communication in any data transfer process. This may include wireless, broadband, telephone modem, standard telecom, Ethernet or other network protocols (eg, the Internet, TCP / IP, Bluetooth, cable TV transmission, and the like).

FIG. 15 illustrates a fluorescent application 1400 in accordance with an aspect of the present invention that may be employed in the systems and processes described above. The k-space system includes an optical system comprising a low intensity light source 1410, such as a light emitting diode (LED) that emits light having a wavelength of about 200 to 400 nm (eg, ultraviolet light). Are made according to LEDs may be employed to provide epi-lighting, transillumination, as described herein (or other types). Even with the use of LEDs (or other low-power UV light sources), the 1420 introduces a UV excitation wavelength to the flat surface that supports the object under test, so that the instantaneous wave coupling of the UV light can excite the fluorophore in the object. To enable waveguide illumination. For example, UV light can be provided at approximately a right angle to the substrate on which the object is placed. At 1430, the LED (or other light source or combination thereof) may emit light for a predetermined time and / or be controlled like a strobe that emits pulses at a desired rate. In 1440, the object is excited for the time determined at 1430. At 1450, automatic and / or manual analysis is performed on the object during the excitation period.

By way of example, the ultraviolet sensitive object fluoresces in response to excitation of the UV light from the light source. Fluorescence is the state of a substance (organic or inorganic) in which a substance continues to emit light while absorbing excitation light. Fluorescence may be an inherent property of the material, or may be induced by employing a fluorescent dye strain or dye, for example. The dye may have an affinity for a particular protein or other receptor to facilitate the discovery of other states associated with the object. In one particular example, by fluorescence spectroscopy and / or digital image formation, methods are provided for studying various materials exhibiting secondary fluorescence.

By another example, a UV LED (or other source) may produce a strong UV radiation flash for a short time, so that the image is constructed by the sensor immediately (eg, from a few milliseconds to several seconds). This mode can be used to investigate the temporal decay characteristics of the fluorescent component of an object under test. This may be important if two parts of the object (or different samples) may have different emission decay characteristics, but respond (eg, emit almost the same fluorescence under continuous illumination).

As a result of using a low power UV light source, the light from the light source can cause at least a portion of the object being tested to emit light, although not generally at ultraviolet wavelengths. Since at least part of the object fluoresces, it is possible to correlate the pre or post fluorescence images with respect to the images obtained during the fluorescence of the object to identify the different properties of the object.

In contrast, most conventional fluorescence systems are configured to irradiate a sample to separate the much weaker re-emission fluorescent light from the brighter excitation light, typically through filters. In order to enable detectable fluorescence, these conventional systems usually require a powerful light source. For example, the light source can be a mercury or xenon arc (burner) lamp, which can give a high intensity illumination that is powerful enough to take an image of the fluorescent sample. In addition to operating at high temperatures (eg, typically 100-250 watts of lamps), these types of light sources typically have a short operating life (eg, 10-100 hours). In addition, such conventional power sources for light sources include timers to help track the number of hours of use, since arc lamps become ineffective and susceptible to destruction when used beyond their rated life. In addition, mercury burners generally do not provide even intensity across the spectrum, ranging from ultraviolet to infrared, since most of the mercury burner's intensity is near ultraviolet. This often requires precise filtering to remove unwanted light wavelengths. Thus, in accordance with one aspect of the present invention, it will be appreciated that the use of UV LEDs provides nearly even intensity at the desired UV wavelength, thereby reducing the power consumption and heat generated through its use. In addition, the LED light source replacement cost is significantly lower than conventional lamps.

16 illustrates a thin film application 1500 according to one aspect of the present invention. Films and thin films are generally over the molecular thickness of a certain material, or a plurality of materials, deposited on thin layers (suitably for the respective materials on several selected substrates), such as metal coating (eg For example, reflective, opaque, transmissive, optical coatings (e.g., interference, transmission, refractive index control, bandpass, blocking, protection, multiple coatings, etc.), plating (including partial) For example, metal, oxidation, chemistry, anti-oxidation, heat, etc., electrical conduction (eg, macro and micro-circuit deposition and composition), optical conduction (eg, deposited optical materials of variable refractive index, micro- And macro-optical "circuits"), which may include, but are not limited to. This is characterized by deposition in a number of ways, such that the desired layer of any material (s) on the substrate has the desired thickness, density, continuity, uniformity, adhesion, and other parameters associated with any given deposited film. Other coatings and any layered film and film-like materials on any substrate that may be included. Related thin film analysis may include detection of microbubbles, voids, fine dust, deposition defects, and the like.

Proceeding to 1510, a k-space system is configured for thin film analysis in accordance with an aspect of the present invention. Applications of k-space image forming apparatus for thin film inspection and characterization can be used, for example, to identify and characterize defects in thin films or films. Such a system makes it easy to:

1) passive observation of the substrate with all types of deposited thin films;

2) automatic observation / analysis and characterization of substrates with all types of deposited thin films for pass-fail inspection;

3) Automatic observation and characterization of substrates with all types of deposited thin films for computer controlled relative deposition, which is selected for verification, verification, etc., for example, CD-ROM, DVD-ROM ) May include image data written in the.

The k-space apparatus can be configured to form an image at a preferred absolute spatial resolution (ASR) and a preferred effective resolution magnification (ERM) per pixel. These parameters can easily determine FOV, DOF, and WD, for example. This can be objective-based design and / or chromaticity-design configuration (eg for wide FOV and appropriate ERM, and ASR). The illumination can be selected based on inspection parameters, for example as trans-lighting and / or epi-lighting.

At 1520, the substrate is

1) shift of the optical image forming path-length by an optical scanning method; And / or

2) mounted to an imager to be scanned by indexing an object directly tested by a process of mechanical movement and control (e.g., automatically by a computer or manually by an operator). This facilitates inspection of the entire surface or portions of the desired surface.

As described above in the context of particle size measurement, the timing interval selected and / or asynchronous in real time for each scanned region of the substrate (e.g., determined by the FOV) at a common video rate and / or image capture rate. Image formation may be provided. Images of the index and / or scanned regions can be captured at a desired frequency for subsequent image processing. In addition, samples may be introduced into the device manually and / or automatically at a "feed" such as from a conveyor system.

At 1530, operating parameters for thin film applications are determined and applied. Typical operating parameters (not limited to the following),

1) imaging of various defects and characteristics including but not limited to surface (s) (or within the surface) particles and holes of the thin film;

2) modular design that can be changed as needed for both reflective and transparent substrates;

3) Automatic counting and classification of surface defects on successively indexed (and / or "scanned") image areas by size, location, and / or number (with index identification and totaling for each sample surface) ;

4) register location of the fault for subsequent manual inspection;

5) subsequent porting (eg, via Ethernet or other protocols), or manual and / or automatic image processing of archives and documents on computers, servers, and / or clients. Provide images in standard format (s); And / or

6) Nominal scan time per surface of several seconds to several minutes, depending on the total area. Scan and indexing rates are generally understood to depend on the sample area and subsequent processing.

At 1540, software and / or hardware based computerized image processing / analysis may be performed. Images from a device made in accordance with the present invention may be processed according to virtually any hardware and / or software process. Software-based image processing can be accomplished by dedicated software and / or commercially available software because the image file format is a digital format (eg, a bitmap of captured films). Analysis, characterization, etc. may be provided by: For example, the analysis can be based on measurements (based on direct measurements) and / or comparisons (databases). Comparative analysis may include a comparison with a database of image data for known membranes and / or variants thereof. Advanced image processing can characterize and classify images in real time and / or with periodic sample measurements. The data can be discarded and / or recorded when desired, and data matching the known sample characteristics can, for example, initiate a suitable selected response. Moreover, devices constructed in accordance with the present invention can be linked for communication in any data transfer process. This may include wireless, broadband, telephone modem, standard telecom, Ethernet or other network protocols (eg, the Internet, TCP / IP, Bluetooth, cable TV transmission, and the like).

In another aspect of the present invention, an image forming system constructed as described above may be combined to provide enhanced biological material image forming systems and methods with high effective resolution magnification and spatial high resolution among features of a biological material. Provide methods. The biological material imaging systems and methods of the present invention provide enhanced images (larger effective magnification, improved resolution, improved field of view) that enable not only the identification of biological material but also the classification (eg normal or abnormal) of biological materials. Depth, etc.).

Biological materials include microorganisms (organisms that are too small to be observed with the naked eye), such as bacteria, viruses, protozoa, fungi, and ciliary insects; For example, cellular material from cells (lysed, intracellular material, or whole cells), cellular material from an organism such as proteins, antibodies, lipids, and carbohydrates, tagged or untagged; And clumps of tissue (tissue samples), blood, pupils, irises, fingertips, teeth, parts of skin, hair, mucous membranes, bladder, breasts, reproductive system elements of muscle / cancer, muscle, vascular elements, central nervous system elements Field, liver, bone, colon, pancreas, and other parts of the organism. Since the biological material imaging system of the present invention can employ a relatively large working distance, parts of the body can be inspected directly without having to take a tissue sample.

Cells include human cells, non-human animal cells, plant cells, and shout / study cells. Cells include prokaryotic and eukaryotic cells. The cells may be healthy, cancerous, mutated, damaged or diseased.

Examples of non-human cells are anthrax, actinomycetes, azotobacters, anthrax, Bacillus cereus, bacteroides, pertussis, Borrelia burgdorferi, Vibrio, chlamydia, Clostridium bacteria, cyanobacteria, Di Inococcus radiodurans, Escherichia coli, Enterococci, Haemophilus influenzae, Helicobacter pylori, Pneumococcal bacterium, Lactobacillus, Rossonia intracellularis, Legionella, Listeria, Mycobacterium, Nacobacteria, Mycobacterium tuberculosis, Mycobacteria , Neisseria gonococcus, meningobacteria, prebotella spp., Pseudomonas, Salonella, Sella cheers, dysentery, Staphylococcus aureus, Streptococcus, Tiomargarita namibiensis, Treponema, Vibrio cholera, Yexinia enterocollitica , Estonia festis, and so on.

Other examples of biological substances include colds, infections, malaria, chlamydia, syphilis, gonorrhea, conjunctivitis, anthrax, meningitis, botulism, diarrhea, brucellosis, campylobacter, candidiasis, candidiasis, cholera, coccidiosis fungi, yeast fungus, Diphtheria, Pneumonia, Food Poisoning, Glanders (Bruckholdereria Malay), Influenza, Hansen's Disease, Histoplasmosis, Legionellisis Disease, Leptospirosis, Listeria Disease, Ubiger, Nocardiasis, Nontubulolytic Mycoxia It causes diseases such as bacterium, peptic ulcer, whooping cough, pneumonia, parrot disease, salmonella enteritis, bacterial dysentery, sforthcum, streptococcal infection, toxic shock syndrome, trachoma, typhoid, urinary tract infection and Lyme disease. As described below, the present invention also relates to a method of diagnosing any one of the aforementioned diseases.

Examples of human cells include fibroblasts, skeletal muscle cells, neutrophils, lymphocytes, erythrocytes, erythrocytes, osteoblasts, chondrocytes, basophils, eosinophilic leukocytes, adipocytes, invertebrate neurons (Helix aspera) , Mammalian neurons, adrenomeldulari cells, melanocytes, epithelial cells, endothelial cells, all kinds of tumor cells (especially melanoma, myeloid leukemia, lung, breast, ovary, colon, kidney, prostate, pancreatic and testicular carcinoma) , Leukocytes, including hematopoietic cells, nerve cells, lung cells, Stem cells such as kidneys, liver and myocyte stem cells, keel cells, chondrocytes and other tissue connective cells, keratinocytes, melanin cells, hepatocytes, kidney cells, and adipocytes. Examples of study cells include modified cells, Jurket T cells, NIH3T3 cells, CHO, COC, and the like.

Useful sources of cell lines and other biological materials include ATCC cell lines and hybridomas, bacteria and bacteriophages, yeast, fungus, and botany, and protozoa: Algae and Protozea, and others available from Americal Type Culture Co. (Rockville, Md.). These are non-limiting examples as cells and other biological materials may be listed in length.

Identification or classification of biological material may in some cases allow diagnosis of the disease. Accordingly, the present invention also provides improved diagnostic systems and methods. For example, the invention relates to atherosclerosis, angiogenesis, atherosclerosis, inflammation, atherosclerosis heart disease, myocardial infarction, traumatic to arterial or venous wall, neurodegenerative disorders, and cardiopulmonary disorders, such as cancer, skeletal muscle pathology. Methods of detecting and characterizing pathologies of such as the digestive system, the reproductive system, and the digestive tract are provided. The invention also provides methods for the detection and characterization of viral and bacterial sensitization. The invention also makes it possible to evaluate the effects of various pharmaceutical or physiological activities on biological materials, both in vitro and in vivo. For example, the present invention makes it possible to assess the effect of a physiological agent, such as a drug, on an individual of cells or tissue grown in culture.

The biological material imaging system of the present invention makes it possible to obtain data from biological material samples with computer driven control or automated process control. In this context, a computer or processor coupled to a biological material imaging system includes or is coupled to a memory or database containing images of biological material, such as cells with various types of disease. In this situation, the normal and abnormal of the biological material can be automatically assigned. The biological material imaging system obtains images from a given biological material sample so that the images are compared with images in memory, such as images of diseased cells in memory. In one aspect, the computer / processor performs a comparative analysis of the collected image data and the stored image data, and based on the results of the analysis, the identity of the given biological material, the classification of the given biological material (normal / abnormal, cancerous) Formulate a determination of cancer free, benign / malignant, infected / non-infected, etc.) and / or status (diagnosis).

If the computer / processor determines that there is sufficient similarity between the particular images from the biological material sample and the stored images (such as diseased cells or of the same biological material), the image is stored and stored in this image. Associated data can be generated. If the computer / processor determines that there is not enough similarity between the specific image from the biological material sample and the stored cells / stored images of the specific biological material, the biological material sample is repositioned and the additional images in the memory. Compare with them. It will be appreciated that statistical methods may be applied by the computer / processor to assist in determining that there are sufficient similarities between certain images from the biological material sample and stored images of the biological material. Any suitable correlation means, memory, operating system, interpretive components, and software / hardware may be employed by the computer / processor.

FIG. 17 illustrates a preferred aspect of an automated biological material image forming system 1600 in accordance with an aspect of the present invention that enables data acquisition from biological material samples by computer driven control or automated process control. Image forming system 1602 described / configured with reference to FIGS. 1-16 above may be employed to capture an image of biological material 1604. The image forming system 1602 is coupled to a processor 1606 and / or a computer that reads an image generated by the image forming system 1602 and compares the image to various images in the data storage 1608.

Processor 1606 includes analytical components for making comparisons. Some of the many algorithms used for image processing include convolution (many others are based thereon), FFT, DCT, thinning (or skeletal), edge detection, and contrast enhancement. They are typically implemented in software but may use dedicated hardware for speed. Fast Fourier Transform (FFT) is an algorithm that calculates a Fourier transform of a set of discrete data values. For example, given a finite set of data points, a periodic sampling taken from an actual signal, the FFT represents the data in its component frequencies. It also deals with essentially the same inverse relationship to reconstruct a signal from frequency data. Discrete cosine transform (DCT) is a technique for representing a waveform as a sum of weighted cosines. For example, there are several applications designed for image processing, such as cellular language for image processing (CELIP) and visual programming language (VLP).

Data storage 1608 includes one or more sets of predetermined images. The images may include normal images of various biological materials and / or abnormal images of various biological materials (sick, mutated, physically destroyed, etc.). Images stored in data storage 1608 provide the basis for determining whether the captured image is similar (similarity) to the images stored. In one aspect, an automated biological material imaging system 1600 may be employed to determine whether a biological material sample is normal or abnormal. For example, automated biological material imaging system 1600 may facilitate the diagnosis of a given disease or condition by identifying the presence of disease cells, such as cancer cells, in a biological material sample. In another aspect, automated biological material imaging system 1600 identifies the presence of a disease causing biological material (such as the disease causing bacteria described above) and / or results in an entity such as a bacteria being given a biological material. The diseases / diseases listed above can be diagnosed by determining that the disease is infectious or by determining that a given biological material is abnormal (cancer).

In another aspect, automated biological material imaging system 1600 can be employed to determine the identity of a biological material of unknown origin. For example, automated biological material imaging system 1600 can identify white powder including anthrax. The automated biological material imaging system 1600 also facilitates biological material processing, such as counting white blood cells or red blood cells in blood samples.

The computer / processor 1606 may be coupled to a controller that controls a servo motor or other means for moving the biological material sample within the object plane to facilitate remote / handsfree image formation. That is, a motor, regulator, and / or other mechanical means can be employed to move the biological material sample slide within the object FOV.

In addition, images of the biological material inspection process are optimized for viewing on computer screens and / or closed circuit monitors, so that remote and web-based viewing and control can be implemented. Real-time image formation facilitates at least one of rapid diagnosis, data collection / generation, and the like.

In another aspect, an automatic biological material imaging system sends images to parts of a person (such as arm disorders, corneal blur, etc.). The images can be sent to a computer / processor (such as over a network such as the Internet), which is instructed to identify possible presence of certain types of disease cells (images of which are stored in memory). When diseased cells are identified, the computer / processor instructs the system to remove / destroy diseased cells using, for example, lasers, liquid nitrogen, cutters, and / or the like.

18 illustrates a high level artificial vision system 1800 in accordance with the present invention. System 1800 includes an image forming system 10 (FIG. 1) in accordance with the present invention. Since the image forming system 10 has been described in detail above, details thereof will be omitted for simplicity. The image forming system 10 may be employed to collect data relating to a product or process 1810 and adjust the product or process 1810, for example, with respect to production, process control, quality control, testing, inspection, and the like. Provide image information to a controller 1820. The image forming system 10 described above provides granularity of image data collection that cannot be achieved by many conventional systems. In addition, highly reliable artificial visual inspection of the product or process 1810 may be provided with reliable image data provided by the present image forming system 10. For example, fine product defects that cannot normally be detected by many conventional artificial vision systems can be detected by the system 1800 as a result of image data collected by the image forming system 10. Controller 1810 may be any suitable controller or control system employed, for example, in connection with a manufacturing method. The controller 1810 is common to artificial vision-based control systems and collects images to remove defective products or processes, re-examine the products or processes, or accept products or processes. Data is available. System 1800 may be employed in any suitable artificial vision based environment, and all such applications of the invention are intended to be within the scope of the claims appended hereto.

For example, the system 1800 may be employed in conjunction with semiconductor fabrication where device and / or process tolerances are critical in producing consistent, reliable semiconductor-based products. Thus, product 1810 may be, for example, a semiconductor wafer, and image forming system 1800 may have data (eg, critical size, thickness, potential coupling, other physical aspects) relating to devices formed on the wafer. ... may be employed to collect. The controller 1820 can use the collected data to remove wafers due to various combinations, modify processes in connection with fabrication of devices on the wafer, accept wafers, and the like.

What has been described above are preferred aspects of the present invention. Of course, it is not possible to describe every conceivable combination of components or methods for the purpose of describing the invention, but one skilled in the art will recognize that many other combinations and substitutions of the invention are possible. Accordingly, the invention is intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the appended claims.

Claims (12)

A sensor with one or more receivers having a receiver size parameter; And An image delivery medium having a resolution parameter correlated with said receptor size parameter, said image delivery medium scaling an individual said receiver to a diffraction limited spot in an object field of view (FOV); An imaging system, characterized in that it comprises a. The image forming system of claim 1, wherein the image transfer medium provides a k-space filter that correlates a pitch associated with the receiver to a diffraction limit spot within the image transfer medium. 3. The image forming system of claim 2, wherein the pitch is unit mapped to the size of the diffraction limit spot within the image transfer medium. The image forming system of claim 1, wherein the image transfer medium further comprises at least one of an aspherical lens, a multiple lens configuration, an optical fiber taper, an image conduit, and a holographic optical element. The sensor of claim 1, wherein the sensor further comprises an M × N array of pixels associated with the receiver, where M and N each represent an integer row and column, wherein the sensor is a digital sensor, an analog sensor, a CCD sensor. And at least one of a CMOS sensor, a CID sensor, an array sensor, and a linear scan sensor. The system of claim 1, wherein the image forming system further comprises a computer and a memory receiving the output from the sensor, wherein the computer stores the output in the memory and performs an automatic analysis of the output in the memory. And map the memory to a display to enable manual analysis of the image. The system of claim 1, wherein the image forming system further comprises an illumination source that illuminates one or more non-emitting objects within the object FOV, wherein the illumination source is a light emitting diode, wavelength specific illumination, broadband illumination, continuous illumination, strobe illumination , Kohler lighting, Abbe lighting, phase contrast lighting, dark field lighting, bright field lighting, Epi lighting, coherent light, non-coherent light, visible light and invisible light Image forming system. 8. The system of claim 7, wherein the invisible light further comprises at least one of an infrared wavelength and an ultraviolet wavelength. The method of claim 1, wherein the image forming system further comprises an associated application, wherein the application comprises image forming, control, investigation, autopsy speculum analysis, biomedical analysis, cell colony counting, histology, freezer analysis, cytology, hematological cancer , Pathology, oncology, fluorescence, interference, phase analysis, biological material analysis, particle size application, thin film analysis, air quality monitoring, suspended particulate measurement, optical defect analysis, metallurgy, semiconductor inspection and analysis, automatic visual system, 3-D At least one of an image forming, camera, copier, fax machine, and medical system application. In the method for generating an image, Determining a pitch size between adjacent pixels on the sensor; Determining a resolvable object size in the optical medium; And Scaling the pitch size through the optical medium to correspond to the resolvable object size; Image generation method comprising a. (a) an image forming system for collecting image data from a product or process, comprising: A sensor with at least one receiver with a receiver size parameter, and At least one optical device having a resolution parameter that correlates to the receptor size parameter and directs light from an object FOV to the receiver, the optical device scaling individual receivers to a diffraction limit spot in the object FOV; And (b) a controller that receives the image data and uses the received image data in connection with the manufacture of the product or the control of the process; Artificial vision system (machine vision system) comprising a. 12. The artificial vision system of claim 11, wherein the artificial vision system is used in a semiconductor-based processing system.

Images