SE519942C2 - Focus deviation estimation method in optical system, involves modifying sharpness of image by processing predetermined degree so that two images are equally sharpened - Google Patents

Abstract

An image reproducing an object, has a focal plane located at a known distance (Z12) from the focal plane of previous image whose sharpness is modified by signal processing at a predetermined degree so that two images are equally sharpened. The focus deviation between object and the focal plane of previous image is estimated based on the modification degree and the distance. Independent claims are included for the following: (1) Optical system arrangement; (2) Digital memory medium storing focus deviation estimation program; (3) Microscope system; (4) Digital image processing method.

Description

10 15 20 25 30 35 2 Such a scan requires constant refocusing, since the test tube is seldom flat and because the mechanism that displaces the object transversely cannot be assumed to be perfect in the sense that no movement along the optical axis takes place. If the system is to be able to deliver new, well-focused images, for example with the frequency 5OHz, then focusing must take place both quickly and reliably.

The focus can be said to have two basic steps. In a first step, the focus deviation, ie the distance between the focus plane of the optical system, and the object is estimated. second step, the optical system can be adjusted, or the imaged object can be moved so that it ends up The small move- which the system images objects with optimal sharpness, In a closer focus plane of the optical system. The steps required to perform the second step can be achieved relatively reliably by means of shifted stepper motors or with piezoelectric crystals. A comparatively more difficult problem is then in the first step to quickly, easily and cheaply achieve a reliable estimate of the focus deviation.

The estimation of the focus deviation is based on a focus-measurement principle, measure, ie a procedure, which generates a focus- ie a measure of how well-focused the system is.

The focus measurement principle can be active or passive, as will be explained below. The focus measure can be relative, for example the sample variance of the image's included pixel intensities, or absolute, ie a direct measure of the focus deviation.

An example of an active focus measurement principle is to use infrared light emitted by the optical system and which is reflected by the object to be imaged.

Such focus measurement principles are used in many autofocus still cameras. The disadvantage of such a principle is, in addition to requiring an extra light source, extra optics, extra sensors and special calibration, that it only works satisfactorily if it is the object or sub-object of interest in, for example, a sample. which reflects the infrared light and not, for example, a coverslip applied to the sample.

From now on, only passive focus measurement principles will be treated. When passive focus measurement principles are used, the information from one or more receiving image sensors is used to obtain a focus measurement. A good relative focus measure should then have its maximum or minimum at optimal focus. An example of a focus measure for passive measurements is the pixel-wise sample variance in an image. A poorly focused and thus more blurred image then gives lower sample variance than a better focused and thus more contrast-rich image.

However, it is not certain that it is only possible to estimate the focus deviation on the basis of such an image. It may be the case that the imaged object per se is rather poor in contrast and thus gives a low sample variance, which indicates a large focus deviation, despite the fact that the image is comparatively well focused.

And even if this is not the case, it is not known from a single value of the sample variance in which direction the imaged object is to be moved in order to obtain optimal focus.

A system based on such a focus measurement principle will therefore be adjusted in the wrong direction at the first attempt in 50% of cases and will therefore be slow. A scanning microscopy also stops.

A possible method of making such a focus measurement principle faster is to add extra optics to the system that enables the calculation of focus dimensions in several image planes simultaneously.

An example of such a system is shown in US-A-5,912,699.

It uses three image sensors, with three separate focus planes.

The disadvantages of such systems are above all higher price and increased complexity. Such systems can also not make sufficiently accurate measurements on preparations that contain several layers, since these layers interfere with each other when focus measurements based on variance are calculated. Summary of the Invention An object of the present invention is to completely or partially remedy the above-mentioned problems.

This object is achieved with a method for estimating the distance between a part of an object and a focus plane belonging to a first image depicting the object according to claim 1, an optical system according to claim 12, a memory medium containing a computer program according to claim 15 and a microscope system according to claim 17.

More particularly, according to a first aspect, the invention relates to a method, in an optical system, for estimating the distance between an object and a focus plane belonging to a first image which depicts the object.

The method is characterized in that a second image is used, which has a focus plane located at a known distance from the focus plane of the first image, in that the sharpness of the first image is modified by signal processing by a degree so that the first and the second image are substantially equally sharp, and that the distance between the object and the focus plane of the first image is estimated on the basis of this degree of modification and said known distance.

Simply put, the system takes advantage of the fact that the one of the first and second images whose focus plane is closest to the depicted sub-object will look sharpest. How much the sharper image needs to be blurred (or the blurred image sharpened) for the two images to have equal sharpness is a measure of how much closer the sub-object of the sharper image's focus plane is. If the first and second images are equally sharp / blurry, it can be assumed that the sub-object is located between the focus plane of the two images. If the mutual distance between the focus plan is known, the distance between each of these and the sub-object can be estimated.

Such a procedure has been shown to perform reliable estimates very quickly. The estimates do not only indicate how large a focus deviation is, but also in which direction the system task, even if it has a sign, i 10 l 25 20 25 30 35 519 Qfizfíiiiiiïf, i i 5 mets optics or the object should be adjusted. No extra optics are normally required to realize the system, which provides a cheap measurement procedure.

Preferably, the process further includes the steps of: generating a first set of secondary images from the first image, by varying degrees of modification of the sharpness of the first image; estimating a first set of correlations between overlapping portions of on the one hand at least a subset of the first set of secondary images and on the other hand the second image; to identify the degree of modification of the sharpness that results in the largest of the correlations, after which the identified degree is used in estimating the distance between the object and the focus plane of the first image.

Correlation between overlapping parts of two images gives a very good measure of how equal the sharpness of the images is.

According to a preferred embodiment, the method further comprises the further steps of: generating a second set of secondary images from the second image, by varying degrees of modification of the sharpness of the second image; and estimating a second set of correlations between overlapping portions of on the one hand at least a subset of the second set of secondary images and on the other hand the first image. Then said identification also takes place on the basis of said second set of correlations; and consideration is also given, in the estimation, to which of the said first and second sets of correlations includes the greatest value.

This allows a comparatively accurate measurement of the distance between the object and the focus plane of the first image, and the modification of the sharpness only needs to be done by reducing the sharpness level, which is called blurring.

Preferably, the modification of the sharpness means at least a reduction of the sharpness, preferably by means of a Gaussian, or approximately Gaussian filter. l0 l5 20 25 30 35 519 942 6 This is a comparatively fast and simple method, which does not add noise to the image.

According to one embodiment, the modification of the sharpness means at least an increase of the sharpness, by means of an approximately inverse Gaussian filter.

Such an embodiment allows accurate measurement of the above-mentioned distance, as long as the sub-object lies between the respective focus plane of the first and the second image, even if only the sharpness of the first image is modified.

Preferably, the modification of the sharpness is carried out by means of a folding, which is a relatively fast procedure.

According to an embodiment of the present invention, the first and the second image are Fourier transformed, after which said modification of the sharpness and said estimation of correlations are performed in the frequency domain. If the Fourier transform can take place quickly, this allows for easier modification of the sharpness and estimation of correlations. Above all, it will be a faster procedure if many sharp modifications and many correlation estimates are to be made.

Preferably, the first image is generated by light within a first wavelength range and the second image is generated by light within a second wavelength range at least partially separate from the first wavelength range.

According to such a method, it is utilized that the optics have a lack of longitudinal color correction. No additional sensor needs to be used, but the first and second images can still be registered without readjustment of the optics.

According to an alternative embodiment, the first image is generated before an adjustment of the system optics known in terms of amount and character, and the second image is generated after said adjustment.

This provides a simple and reliable measurement method in systems where such adjustment takes place for another reason. 10 15 20 25 30 35 519 94 ") 7 According to a second alternative embodiment, the first image is generated before a movement of the object known in terms of amount and character in relation to a focus plane of the optical system and the second image is generated after said movement.

This provides a simple and reliable measurement method in systems where such movement takes place for another reason.

According to a third alternative embodiment, the first image is registered by means of a first image sensor and the second image is registered by means of a second image sensor.

This provides a quick and accurate estimate of the above-mentioned distances in systems where there is no or only a small lack of color correction.

According to a second aspect of the invention, this relates to an arrangement in an optical system for estimating the distance between an object and a focus plane belonging to a first image depicting the object. The arrangement utilizes a second image, which has a focus plane located at a known distance from the focus plane of the first image and has means for modifying the sharpness of the first image by a degree so that the first and second images are substantially equally sharp. and means for estimating the distance between the object and the focus plane of the first image on the basis of this degree of modification and said known distance.

This arrangement entails corresponding advantages as the method shown above and can similarly be modified in a number of corresponding ways. In general, the arrangement then comprises means for carrying out the steps shown in the procedures.

According to a third aspect of the invention, it relates to a digital memory medium comprising a computer program for estimating in an optical system the distance between an object and a focus plane belonging to a first image which images the object. image, from the focus plane of the first image, The computer program utilizes a second having a focus plane located at a known distance and includes instructions for modifying the sharpness of the first image 10 15 20 25 30 35 519 QD $> FJ 8 by a degree such that the first and second images are substantially equally sharp and instructions for estimating the distance between the object and the focus plane of the first image on the basis of said degree of modification and said known distances.

This computer program has similar advantages as the method shown above and can be modified in a number of corresponding ways similarly. In general, the computer program then includes instructions for performing the steps shown in the procedures.

According to a fourth aspect, the invention relates to a microscope system with a microscope and a computer system, which comprises an arrangement as above. Such a system entails corresponding advantages.

The invention will now be described in more detail with reference to the accompanying figures.

Brief description of the figures Fig. 1 schematically shows an optical system.

Fig. 2 shows a number of images of an object comprising non-stained red blood cells, which are imaged by phase contrast microscopy.

Fig. 3 shows a number of images of an object comprising MGG-stained red blood cells, which are imaged by light field microscopy.

Fig. 4 shows a system in which a method according to the present invention is applied.

Fig. 5a shows the result of an experiment based on the images in Fig. 2.

Fig. 5b shows the result of an experiment based on the images in Fig. 3.

Fig. 6 shows a flow chart for a method according to the invention.

Description of Preferred Embodiments Fig. 1 schematically shows an optical system. A lens 1 then images an image of an object 6, which is located in an object plane 5, on a sensor image plane 2, belonging to a two-dimensional sensor. The system is shown for the sake of clarity with only one lens. In a real microscope system, there are several lenses. The optics, ie in this case the system of lens and sensor have two shown associated focus planes. A first focus plane 3 is located at the distance (focus deviation) 21 above the object plane and is the plane in which the above-mentioned optics is focused for blue light. A second focus plane is located at the distance (focus deviation) z2 below the plane of the object and is the plane in which the optics are focused for green light. The first and second focus planes are substantially parallel and located at the known distance zu from each other. The optics can also be assumed to have a third separate focus plane for red light (not shown).

The fact that the first and second focus planes do not coincide is due to the fact that the focal length of the optics is wavelength dependent. It is then said that the optics have incomplete longitudinal color correction. This is usually considered a problem. In connection with the present invention, however, this incompleteness can be used to facilitate focusing of the system.

Fig. 2 shows a number of images, taken with a CCD image sensor, of an object comprising non-stained red blood cells, which are imaged with phase contrast microscopy. The object is in five different positions in relation to the microscope's optics. The distance between two adjacent positions is about 0.5 μm. For the different modes, pictures 30- 34 with a green color component are shown in the left column and the corresponding pictures 35-39 with a blue color component in the right column. Images 30- 39 are shown in grayscale. As can be seen from the image, the sharpest green and blue images appear at different distances from the lens.

Fig. 3 shows a number of images of an object comprising MGG- (May-Grünwald-Giemsa) -coloured red blood cells, which were imaged by light field microscopy. 10 15 20 25 30 35 519 942 10 The object is in five different positions in relation to the optics of the microscope. The distance between two adjacent positions is about 0.8 μm. pictures 40- and corresponding pictures 45- 49 with blue color component in For the different modes, 44 with green color component are shown in the left column and the right column. As can be seen from the image, the sharpest green and blue images appear at different distances from the lens.

Fig. 4 shows a microscope system comprising a microscope, a computer system and an arrangement according to a preferred form of the present invention. The arrangement performs a method according to a preferred embodiment of the present invention. A light source 100 then illuminates a sample 102, consisting of a medical sample in the form of a stain to be imaged. The sample can, for example, smear blood on a test tube. An interesting object can then consist of a white blood cell in this sample. A portion of the light transmitted through the sample 102 is captured by a lens, which images it on an image sensor 106.

The image sensor 106 may be a single-chip or a three-chip color sensor, but normally includes a matrix of small photosensitive elements, each of which generates a pixel in a recorded image. The sensor may also include pixel elements provided with filters so that they detect light of a certain color. The image sensor 106 is connected to an image memory 108, which can store images received by the image sensor 106.

The light source 100 can be said to emit a red, a green and a blue color component (ie light with different wavelengths), denoted ro, go and bo. These can be emitted at the same time as white light, but it is also possible to arrange one or more light sources which emit light with two or three of the green and blue ones in a certain sequence. This can enable the use of the primary colors red, each of which is powered by a simpler image sensor.

The light transmitted through the object can similarly be said to consist of three basic images, a red, a green and a blue. The lens 104 projects these images onto the image sensor 106, which These are denoted rl, gl and bl. results in three registered images denoted r, g and b. These images can also be used in image processing units 110 and image analysis 112. The images r, g and b will be transformations of rl, gl and bl, since the lens is not ideal. In addition, since the lens cannot be assumed to process incoming light completely independent of wavelength, these transformations will differ for the different colors.

From now on, only the functionality of the colors green and blue will be treated in detail, but the principle of red is the same.

The relationship between the images at the illuminated object and at the image sensor can be expressed: b = @ * m @ fl Å »g: g | * hÅÛ-z 22)) The sign “*” then denotes a two-dimensional fold.

The functions h1 and hg denote impulse responses, which each have wavelength parameters ol and og, respectively, and which are strongly dependent on respective parameters z1 and 22, respectively, which denote the distance between the focus plane of the current color and the object, as shown in Fig. 1.

The green and blue images captured by the image sensor 106 and stored in the image memory 108 are sent to an estimation module, to estimate the distances z1 and 22. It can be assumed that the distance between the green and the blue focus plane zu, cf. Fig. 1, is known. The distance zu may be measured in the manufacture of the system, but preferably it is measured in recurring calibration procedures.

The blue image b, is connected to a first convolution module 116. The green image is a second convolution module 120. which is a matrix with pixel values, (also a matrix with pixel values) is connected to The convolution modules are here 10 15 20 25 30 519 94-2 12 general means for modifying the sharpness of an image. In the folding modules, the images are modified electronically. In a preferred embodiment, blur is added to the images. This is called below that the image is blurred (blur). The degree of added blur is hereinafter referred to as the blur parameter, denoted oy The output signal from the first convolution module 116 may be called a blurred blue image and denoted b * (o¿), where: b * (Û'3) = b * h (Û-3 ) The designation “*” then means a two-dimensional discrete convolution, and h is a Gaussian impulse response: -w 1 -eh (x, y, 0) = 2 O '_0O In the same way a blurred green image is generated g * «n ) in the second estimation module 120, whereby: g * (0;) = g * h (03) The image can optionally be bandpass filtered before, after or integrated with the modification of the sharpness. This can be done to compensate for high frequency noise, low frequency light variations or color differences. Generally, bandpass filtering is used to highlight components that contain sharpness information. Preferably, a modified Gaussian filter is then used in the sharpening modification.

The blurred blue image b * (o3) and the original green image g are input to a first correlation module 122, two images are calculated. In general, this correlation is expressed in which the pixel-wise correlation between these between two images a and b: 10 15 20 25 30 519 9 ~ 122 where m and n express the size of the images in number of pixels. The sample values of the different images are denoted ä and b_.

However, negative correlation values are not likely in The correlation can assume values between 1 and -1. this context. Above, pixel-based correlation is used, which is preferred, however, other correlation calculations, such as for example between groups of pixels, are conceivable. The first correlation module 122 generates the output parameter p (o3), which expresses the correlation between g and b * (0 fi.

Similarly, a second correlation module 124 generates an output parameter p "(o3), which expresses the correlation between b and g * kn).

The modification of the images and correlation between the images do not necessarily have to be performed on the entire images. The procedure can also be performed on part of the image.

In the system there is a parameter calculator 126. This controls the blur parameter 03 until a globally largest value for p (o3) and p ~ (o3) is obtained. By globally largest value is meant the common maximum for p (o3) and p "(o fi.

The value of o3 that generates a maximum correlation value is then used for the estimation of z1 and z2. In making this estimate, a number of assumptions are made. It is assumed that hl and hg are Gaussian and identical to the 120, except for their respective wavelength and focus impulse responses used in the convolution modules 116, with deviation parameters. Furthermore, it is assumed that the details in the object that appear with blue light are also visible with green light. Under such conditions, the expressions for green and blue color component can be simplified to: b I bl * h (Û_1 (Z1 g I bl * h (o'z (zz)) 519 942 14 Three cases are possible: a) If ö2 (z2 )> ö1 (z1) is given p (03) = l then: gob * = [a * hbxzz))] ° [b. * h (= [b] * h <«fz> 1o [f». * h (Uaza fl f:)]: 1 »fw then Ü22 (Z2) = Û12 (Z1) + Û'32 5 b) Om o¿ ( z1)> o¿ (z2) is given in a similar way p ~ (o3) = l then: Û12 (Z1): Ü22 (Zz) + Ü32 10 c) If o¿ (z1) = o¿ (z2) is given in a similar way set fi> (03) = D ~ (03) = 1 then! o ', 2 (z]) = 0š (z2), ie when a3 = 0 15 For each of these cases a system of equations must be solved with respect to 21 and 2; a) {Û 'fi (Z2): Ü12 (Z1) + Ü32 q + z2 = 42 20 b) Z1 + Zz: 212 c) {z1 + z2 = 212 10 15 20 25 30 35 519 942 15 The parameter calculator 126 performs numeric calculations corresponding to those shown above and can therefore, on the basis of ol, og, 03 and 212, make estimates of 21 and 22.

These estimates can then be used by the optical system. In the example shown, the estimates are input to a focus controller 128, which controls a focus mechanism 129 to the desired position.

The parameter calculator 126 as well as the convolution 116, 120 and correlation modules 122, 124 can be implemented as ASIC (Application Specific Integrated Circuit) or FGPA (Field Programmable Gate Array) circuits. Alternatively, all or part of the calculation method included in the invention may be implemented in terms of software as a computer program. The computer program then comprises instructions which are intended to be executed in a digital signal processor (DSP), the meter calculator at least one means for identifying a PC processor or the like. In general, the parade is 63, which provides the greatest correlation and means for calculating estimates for the values of 21 and z2.

Simply put, it can be said that the system uses that the image whose focus plane is closest to the depicted sub-object will look sharpest. How much the sharper image needs to be blurred by signal processing (or the blurred image sharpened) for the two images to have equal sharpness is a measure of how much closer the sub-object is in relation to the sharper image's focus plane. If the first and second images are equally sharp / blurry, it can be assumed that the sub-object lies in the middle of the focus plane of the two images. If the mutual distance between the focus plan is known, the distance between each of these and the sub-object can be estimated.

It is possible to use only a folding module 116 and a correlation module 122. One can then speak of an asymmetric system, since the sharpness is modified in only one of the images in contrast to the system shown in Fig. 4. A prerequisite for this to work satisfactorily is that the sharpness can not only be reduced in the folding module, but also increased. When increasing the sharpness, an approximately inverse Gaussian filter is used. Such a filter enhances the contrasts in the figure, but can also be assumed to amplify and add noise to the image, since contrasts that may not exist at all in a focused image can be achieved.

The size of the measurement area within which a relevant focus deviation estimate can be made is determined by the distance zu between the focus plan. As long as the object of interest is between these planes, the procedure provides a directly useful estimate. A large lack of longitudinal color correction in an optical system, which in most cases is considered negative, results according to the invention in a large measuring range, which may be considered positive. There are at least three possibilities for creating images with different color content. Either an image sensor with color-filtered pixel elements can be used. Such a blue and a red image. Alternatively, different image sensors are used for the different colors. A surface sensor emits a green, a further alternative is to, for example, first illuminate the object with a green light, whereby a first image is registered, after which the object is illuminated with a blue light and a second image is registered.

However, alternative methods also exist for producing images with different focus planes without utilizing lack of longitudinal color correction. This possibility can be used if you want a larger measuring range. One possibility is then to register a first image before and a second image after an adjustment of the system optics known in terms of amount and character, for example a refocusing. Alternatively, the images can be registered before and after a movement of the object known along the optical axis of the system known in terms of amount and character. An additional possibility is to use several sensors with different optical distances to the imaged object. Fig. 5a shows the result of an experiment based on the images in Fig. 2. For each pair of images, at a given distance between the object and the lens, a two-part correlation curve is given, as a function of blur pair. - meter 03. Each correlation curve consists in its left and right part of p, (for example 123), as a function of o3 (ie different sharp modifications in the first image), o3 (ie different sharp modifications of the second image). The respective p ~, (for example 125), as a function of the resulting five correlation curves have their maxima for five different values of o3, which corresponds to the object being in five different positions in relation to the objective. The curve 130 is related to the image pair 30, 35 2), the curve 131 with 31,36 (fig. 2) maxima reaches the correlation 1 as in the assumption (fig. Etc. None of the five above. The reason is that when real image pairs are analyzed it is often several of the conditions in the description above in connection with Fig. 4 which are not fulfilled.

First, the object is not always thin in relation to the depth of field of the lens. The resulting correlation curve then becomes a weighting of a number of correlation curves - one for each thin “layer” in the object. the top, Then the position of the resulting correlation and thus corresponding estimates of the focus deviations may be disturbed more or less.

Secondly, it is seldom the case that both the object and the focus plane are completely flat. The result is then, because (and varies across the image. The effects hardly cancel each other out, that zl and 22 thus GL G2 and the optimal og Even zu can vary across the image.

Third, an impulse response for a microscope objective is composed of a diffraction-limited part and a defocus-dependent part. The former is only approximately Gaussian.

Fourth, all the details of the object seldom behave the same for, for example, green and blue light. 10 15 20 25 30 35 519 942 18 Although the assumptions made in connection with the description of Fig. 4 are only approximately correct, the model gives very good estimates of the system's focus deviations. It is also possible, if necessary, to adjust the method to compensate for some of these deviations from the assumptions. If, for example, a colored white blood cell is imaged, this gives significantly larger amplitudes in green than in blue. It is possible to compensate for this by excluding in the correlation calculation the pixels where the green component deviates by more than a certain amount from its mean value.

On the other hand, the system is not affected to any great extent by the motion blur that results from the test tube being in motion during scanning microscopy. This is because the images of the different colors are affected substantially equally by the movement (compare the beam path r, g, b in Fig. 4).

Fig. 5b shows the result of an experiment based on the images in Fig. 3. Curve 140 is related to the image pair 40,45 (Fig. 3), curve 141 by 41.46 (Fig. 3), it is clear that the object can no longer be considered as etc. . Here is thin. The driving relationship curves show clear double top pairs. A stained sample of red blood cells contains two main “layers” that can be imaged by light field microscopy, namely the top of the slide and the top of the red blood cells, respectively. When a non-stained sample of red blood cells is imaged with phase contrast microscopy, the object's variations in refractive index are mainly reproduced, which mainly occur in the “layer” where the red cells are alternated with microscope oil.

In a system according to the invention, therefore, two distances can be generated which are both correct. This is a crucial difference in relation to the systems that use focus measures based on variance. In such systems, the two "layers" interfere with each other so that an incorrect focus measure results. In the image series shown in Fig. 3, a relative focus measurement principle based on a variance measure therefore gives an error of approximately 0.2 μm when calculating the best focus position.

According to the present invention, on the other hand, a sample with two layers gives a certain advantage, since it is essentially sufficient that one of the two layers lies within the measuring range, ie between the focus plane of the two images, in order to be able to perform an interesting distance calculation.

As an alternative to the folding modules shown in Fig. 4, a system based on a Fourier transform can be used. The incoming green and blue images b and g are then transformed by means of a discrete Fourier transform to the frequency plane, after which modification of sharpness and correlation is performed with the transformed images.

The transform is calculated numerically in a known manner using a FFT (Fast Fourier Transform) algorithm. The transform itself is relatively computationally demanding and thus time-consuming. However, if many different modifications and correlation calculations are to be performed for each pair of images, this method will still be faster overall, since sharp modification and correlation can be performed much faster in the frequency plane. Thus, the FFT algorithm is suitable to use if many values of og are to be tested.

In general, the transformation, sharpening modification and correlation calculations proceed as follows when the FFT algorithm is used.

The areas in the green and blue images that are transformed should be square with a page, which has a size in number of pixels, which can be expressed in a 2-power, for example 256 * 256 pixels. What is to be transformed is thus a matrix with pixel intensity values.

In order to function satisfactorily, the Fourier transform requires that a periodic continuation be present in an image.

For this reason, the images are mirrored in two dimensions, so that larger images are created. For example, a matrix: 10 15 20 25 519 942 20 is mirrored to a larger matrix: 1 2 2 1 3 4 4 3 3 4 4 3 1 2 2 1 This means that the subsequent FFT algorithm gives more useful results. The result of the discrete Fourier transform is for the blue image the two-dimensional frequency spectrum B (u, v) and for the green image the corresponding spectrum G (u, v).

The spectra of the blue and green images are then normalized so that B (0,0) = G (0,0) = 0. This corresponds to the average value of the pixel intensity in each image being set to zero when the correlation is performed.

Modification of the sharpness in, for example, the blue image can then be calculated as BTk (u, v) = B (u, v) * Tk (u, v), where Tk (u, v) is the Fourier transform of the kzte Gaussian filter. Instead of making a computationally demanding fold, a multiplication is thus made instead.

The correlation between the sharp-modified blue image and the green image can be calculated as: 2 fl ßçu, mk (m) - GW, n] Jzzlßnlnqïknhnf -Zzkïnrnjz Similarly, the correlation between a PUf) = sharp-modified green image and a blue image is calculated: 2 Z | G (u, v) Tk (u, v) - B (u, v) 1 JZZ | G (u, v) Tk (u, v) | 2 -22 | B (u, v) | 2 fifl f) = 10 15 20 25 30 35 519 9fè2 21 The result of these calculations will be approximately the same as in the calculations shown above in the convolution modules. The value of the parameter k that results in maximum correlation is then used to calculate the distance sought.

For images with only or almost only blood cells in, there is an opportunity to make the above-mentioned Fourier- The reason for this is that the blood cells in an image look like small circles, there is no transformer faster. or almost no elongated objects in any particular direction in the image. Thanks to this, one can make an assumption that the Fourier transform of the image is rotationally symmetrical around the zero point (eg B (0,0)). Therefore, only one-dimensional Fourier transforms need calculation and G (u, O). morna then instead become single sums, which simplifies bones, such as B (u, O) The double sum calculation shown above. An even greater benefit of this procedure is that no two-dimensional, large intermediate results need to be stored during the calculation, which significantly simplifies the hardware that needs to be used. This enables the use of a relatively slow and inexpensive DSP, and is therefore a preferred embodiment.

It is also certainly not necessary to use all the pixels in a row in the calculations. For certain types of images, downsampling can be done, ie only every nth pixel along a row is used in the calculations. This can give reliable results even though the calculations can be done faster.

Fig. 6 shows a flow chart for a method according to the invention. In a first step, a first set of secondary images is then generated from a first image.

In a second step, a second set of secondary images is generated from a second image.

In a third step, correlations are calculated between, on the one hand, the first image and, on the other hand, at least a subset of the second set of secondary images. In a fourth step, correlations are calculated between, on the one hand, the second image and, on the other hand, at least a subset of the first set of secondary images.

In a fifth step, the degree of modification of the first or the second image that results in the largest correlation value that has emerged at the third and fourth steps is identified.

In a sixth step, an estimate is made of the distance between the first image focus plane and the imaged object, at which estimate the identified degree of modification is input.

The mutual order between several of the above steps can be reversed. The entire first set of secondary images does not need to be created simultaneously.

It is preferable to generate secondary images continuously and at the same time determine correlations for created secondary images. The parameter calculator shown in Fig. 4 then controls the degree of sharpness modification until a relevant correlation maximum is found. The set of secondary images then does not need to be made larger.

The invention is not limited by the embodiments shown above, but can be modified in a number of ways within the scope of the appended claims.

Claims (17)

10 15 20 25 30 35 4.1; n 519 9 Yes. BJ 23 PATENT CLAIMS 1. 1. A method in an optical system for estimating (zl) between an object (6) and an associated a first image which depicts the focus deviation focus plane (3) of the object, characterized in that a second image, which image has a image, the focus plane (4) is imaged from the (212) first image focus plane, in that the sharpness of the first image located at a known distance is modified by signal processing by a degree so that the first and second images are substantially equally sharp, and that the focus deviation between the object (6) and the focus plane of the first image is estimated on the basis of this degree of modification and said known distance (zu). A method according to claim 1, (151) comprising the steps of: - generating a first set of secondary images from the first image, by varying degrees of modification of the sharpness of the first image; (154) relations between overlapping parts of one side at least - to calculate a first set of correlations a subset of said first set of secondary images and on the other hand the second image; (155) the sharpness resulting in the largest of said corrections - to identify the degree of modification of lations; after which the identified degree is used in estimating the focus deviation between the object and the focus plane of the first image. The method of claim 2, wherein the method further comprises the steps of: (152) images from the second image, by varying degrees of - generating a second set of secondary modifications to the sharpness of the second image; and (153) interactions between overlapping portions of, on the one hand, the mean - calculating a second set correlating a subset of said second set of secondary images and, on the other hand, the first image; 10 15 20 25 30 35 519 942 24 - wherein said identification (155) also takes place on the basis of said second set of correlations; and (156), also to which of said first and second sets - wherein, in said estimation, correlations involving the largest value are taken into account. A method according to any one of the preceding claims, wherein said modification of the sharpness means at least reduction of the sharpness, by means of a Gaussian filter. A method according to any one of the preceding claims, in which said modification of the sharpness means at least increasing the sharpness, by means of an approximately inverse Gaussian filter. A method according to any one of the preceding claims, wherein said modification of the sharpness is performed by means of a folding. A method according to any one of claims 2-5, wherein said first and second images are Fourier transformed, after which said modification of the sharpness and said estimation of correlations is performed in the frequency domain. A method according to any one of the preceding claims, wherein the first image is generated by means of light within a first wavelength range and the second image is generated by means of light within a second wavelength range at least partially separate from the first wavelength range. A method according to any one of claims 1-7, wherein the first image is generated before an adjustment of the system optics known to the amount and the techniques and the second image is generated after said adjustment. A method according to any one of claims 1-7, wherein the first image is generated before a movement of the object known in terms of amount and character in relation to a focus plane of the optical system and the second image is generated after said movement. 11. ll. Method according to one of Claims 1 to 7, in which the first image is registered by means of a first image sensor. . ~ n,. -. In the .2- ..- 25 sensor and the second image is recorded by means of a second image sensor. Arrangement in an optical system for estimating the focus deviation between an object and a focus plane belonging to a first image depicting the object, characterized in that the arrangement uses a second image, which at least partially overlaps the first image, and which has a focus plane located at a known distance from the focus plane of the first image, by means (116) for modifying the sharpness of the first image by signal processing so that the first and second images are substantially equally sharp, and by means (126) to estimate the focus deviation between the object and the focus plane of the first image on the basis of this degree of modification and said known distances. The arrangement of claim 12, further comprising means (116) for generating a first set of secondary images from the first image, by varying degrees of modification of the sharpness of the first image; means (122) for calculating a first set of correlations between overlapping portions of one side at least a subset of said first set of secondary images (126) for identifying the degree of sharpness modification as a result and on the other hand the second image; means in the largest of said correlations, which identified degree is used by said means to estimate the focus deviation between the object and the focus plane of the first image. The arrangement of claim 13, further comprising: means (120) for generating a second set of secondary images from the second image, by varying degrees of modification of the sharpness of the second image; means (124) for calculating a second set of correlations between overlapping portions of on the one hand at least a subset of said second set of secondary images and on the other hand the first image; wherein said (126) means for identifying is arranged to operate with 519 942. . . . i. | - «n 26 starting point also from said second set of correlation- (126) estimation of the focus deviation is arranged to take into account; and wherein at said means for up to which of said first and second sets of correlations includes the largest value. 15. A digital memory medium comprising a computer program for estimating in an optical system the focus deviation between an object and a focus plane belonging to a first image depicting the object, characterized in that the computer program utilizes a second image which at least partially overlaps the first image, and having a focus plane located at a known distance from the focus plane of the first image, in that the computer program includes instructions for modifying the sharpness of the first image by a signal processing so that the first and second images are substantially as sharp and instructions for to estimate the focus deviation between the object and the focus plane of the first image on the basis of said degree of modification and said known distances. The digital memory medium according to claim 15, wherein the computer program further comprises instructions for the steps: - generating a first set of secondary images from the first image, by varying degrees of modification of the sharpness of the first image; - calculating a first set of correlations between overlapping parts of on the one hand at least a subset of said first set of secondary images and on the other hand the second image; - to identify the degree of modification of the sharpness which results in the largest among said correlation, after which this degree is used in the estimation of the focus deviation between the object and the focus plane of the first image. A microscope system with a microscope and a computer system, which comprises an arrangement according to any one of claims 12-14.