Disclosure of Invention
Therefore, the technical problem to be solved by the present invention is to provide a micro-droplet observation device and a micro-droplet image recognition method based on shape matching, in which the flow velocity of micro-droplets in an observation cavity is reduced, thereby facilitating the acquisition of clear images of micro-droplets in the observation cavity by a camera, reducing the requirements on the performance of the camera, and reducing the cost of the micro-droplet observation device.
In order to solve the above problems, the present invention provides a micro-droplet observation device including:
micro-droplet generating means for generating micro-droplets;
the micro-droplet generating device comprises a micro-droplet chip, wherein an observation cavity is formed on the micro-droplet chip, micro-droplets generated by the micro-droplet generating device can enter the observation cavity through corresponding micro-pipelines and are perpendicular to the flow direction of the micro-droplets, and the width of the micro-pipeline is smaller than that of the observation cavity;
and the optical imaging component is used for imaging the micro-liquid drop in the observation cavity into a first image.
Preferably, the height of the observation cavity is H, the diameter of the micro-droplet is D, and H is more than or equal to 1.2D and less than or equal to D, wherein the height of the observation cavity is H, and the diameter of the micro-droplet is D; and/or a plurality of observation cavities are adjacently arranged in parallel to the flowing direction of the micro-droplets.
Preferably, the micro-droplet observation device further comprises a micro-droplet collection device, and the micro-droplet collection device is positioned in the observation cavity and is positioned at the downstream of the micro-droplet flowing direction.
Preferably, the optical imaging component comprises a camera and a white light source, wherein the camera and the white light source are respectively arranged on two opposite sides of the micro-droplet chip and correspond to the region of the observation cavity.
Preferably, the micro-droplet observation device further comprises an image processing and analyzing device which is in communication connection with the camera.
The invention also provides a micro-droplet image recognition method based on shape matching, which is used for processing the obtained first image and comprises the following steps:
a template specifying step for obtaining a matching template of the target shape;
and positioning and counting target micro-droplets, namely searching and identifying micro-droplets matched with the shape of the matching template in the first image by adopting the matching template.
Preferably, before the step of specifying the template, the method further comprises:
and a matching template generation step, namely selecting a single micro-droplet image meeting a preset target, and performing image processing on the single micro-droplet image to serve as a matching template of the target shape.
Preferably, after the template generating and specifying step before the target micro-droplet positioning and counting step, the method further comprises: and a matching parameter adjusting step, which is used for adjusting the matching parameters in the searching and identifying process so as to improve the identifying efficiency and/or the accuracy of the identifying result.
Preferably, the matching parameters include: at least one of a contrast threshold parameter, a visibility parameter, an image pyramid layer number, a rotation angle parameter, a zoom parameter, a search completeness parameter, an inter-object maximum overlapping area ratio parameter, and a rotation angle parameter.
Preferably, before the target micro-droplet positioning and counting step, the method further comprises: and an identification area framing step, which is used for framing a partial area of the first image to serve as an identification area so as to perform matching identification on the micro-droplets in the identification area.
According to the micro-droplet observation device and the micro-droplet image recognition method based on shape matching, the width of the observation cavity is larger than that of the micro-pipeline, so that the flow velocity of micro-droplets is reduced after the micro-droplets flow into the observation cavity from the micro-pipeline, a camera can conveniently acquire clear images of the micro-droplets in the observation cavity, the requirements on the performance of the camera are reduced, and the cost of the micro-droplet observation device is reduced.
Detailed Description
Referring to fig. 1 to 7 in combination, according to an embodiment of the present invention, there is provided a micro-droplet observation device including: a micro-droplet generating device 10 for generating micro-droplets; a micro droplet chip 20, wherein an observation cavity 21 is configured on the micro droplet chip 20, the micro droplet generated by the micro droplet generating device 10 can enter the observation cavity 21 through a corresponding micro pipeline, and is perpendicular to the flow direction of the micro droplet, the micro pipeline has a width smaller than that of the observation cavity 21, that is, the micro droplet enters the observation cavity 21 through the micro pipeline, and the micro droplet has an increasing trend in volume, so that the micro droplet entering the observation cavity 21 is prevented from being tightly attached to each other and deformed; and the optical imaging component is used for imaging the micro-droplets in the observation cavity 21 into a first image. In the technical scheme, the width of the observation cavity 21 is larger than that of the micro-pipeline, so that the flow velocity of micro-droplets is reduced after the micro-droplets flow into the observation cavity 21 through the micro-pipeline, the camera can conveniently acquire clear images of the micro-droplets in the observation cavity, the requirement on the performance of the camera is reduced, and the cost of the micro-droplet observation device is reduced. The first image may be a plurality of images, for example, the optical imaging component is controlled to acquire and acquire the images at preset intervals, for example, one image is acquired every 0.5 s.
Preferably, perpendicular to the flowing direction of the micro-droplets, the height of the observation cavity 21 is H, the diameter of the micro-droplets is D, 1.2D ≦ H ≦ D, and more preferably D ═ H, so that it can be ensured that a plurality of micro-droplets in the observation cavity 21 can be tiled without overlapping in the height direction, thereby making the final observation result more accurate; the observation cavities 21 are adjacently arranged in a plurality parallel to the flowing direction of the micro-droplets, that is, a plurality of observation cavities 21 with micro-droplets can be simultaneously presented in the first image, so that one image with more micro-droplets can be provided, and the number of identification samples can be increased. As a specific embodiment, as shown in fig. 2, the observation chamber 21 is provided with 4.
Further, the micro-droplet observing device further comprises a micro-droplet collecting device 30, wherein the micro-droplet collecting device 30 is located downstream in the flow direction of the micro-droplets in the observing cavity 21 to collect the micro-droplets flowing out of the observing cavity 21.
The optical imaging component comprises a camera 41 and a white light source 42, the camera 41 and the white light source 42 are respectively located at two opposite sides of the micro droplet chip 20 and correspond to the region of the observation cavity 21, and it can be understood that the imaging fields of view of the white light source 42 and the camera 41 are opposite to each other, so as to accurately acquire the image in the observation cavity 21, in a specific application, it is preferable that the camera 41 uses a telephoto lens to ensure the imaging quality, and the observation cavity 21 occupies most of the area in the field of view, so as to ensure the effective field of view utilization; further, the droplet observing apparatus further includes an image processing and analyzing device 50, which is in communication connection with the camera 41, for example, the camera 41 may be connected with the image processing and analyzing device 50 through a wired USB interface, and of course, the image processing and analyzing device may also be connected in a wireless communication manner such as bluetooth and wifi, which is not particularly limited in the present invention, but it is understood that the image processing and analyzing device 50 may be, for example, a computer, which is configured to receive the first image (which may be multiple images) acquired by the camera 41 and identify, analyze and count the first image.
According to an embodiment of the present invention, there is also provided a method for identifying microdroplet images based on shape matching, which is used for processing the first image obtained above, and it can be understood that the method can be specifically performed by using the image processing and analyzing apparatus 50 described above, and includes the following steps:
a template specifying step for obtaining a matching template of the target shape; and positioning and counting target micro-droplets, namely searching and identifying micro-droplets matched with the shape of the matching template in the first image by adopting the matching template.
In the technical scheme, the object can be searched and positioned based on a single template or a template image based on shape matching, and the method has stronger robustness on noise, clutter, shading and any nonlinear illumination change; the precision of positioning the two-dimensional object can reach a sub-pixel level, and the two-dimensional object can be matched even if the two-dimensional object is rotated or zoomed; the method can be applied to standard 8-bit gray value images, images with the depth larger than 8-bit gray value and color (generally, multi-channel) images, has strong adaptability, is suitable for searching objects with fixed characteristics based on a shape matching algorithm, is very suitable for identifying scenes of micro-droplets, and the outlines of the micro-droplets in the observation cavity 21 shot by the camera 41 are similar circles. In addition, the micro-droplet image recognition method based on shape matching is high in recognition accuracy and relatively low in calculation complexity.
Preferably, before the step of specifying the template, the method further comprises: and a matching template generation step, namely selecting a single micro-droplet image meeting a preset target, and performing image processing on the single micro-droplet image to serve as a matching template of the target shape. Specifically, a certain number of representative microdroplet images are collected, a part of the microdroplets in the microdroplets is artificially selected and labeled, the selected microdroplets are used as templates, and for example, in fig. 2, a microdroplet image meeting the preset requirement is selected from an acquired first image, so that the matching template shown in fig. 3 is formed. It is important to construct a suitable template, which also gives good accuracy to the final match, and which can be saved in a file for reuse in different applications. More specifically, for example, the resolution of the acquired image is 2448 × 2048. In the middle stage of droplet generation, the size of the micro droplets is relatively uniform and the shape is most regular, so that a certain conventional droplet at this stage is selected, as shown in fig. 3, the outline of the droplet can be abstracted into a ring, the inner diameter of the ring is 30 pixels, the outer diameter of the ring is 36 pixels, at this time, a template picture with the specification of 40 × 40 pixels can be created, two circles with the radii of 15 and 18 respectively are drawn on the picture, in order to reflect the gray level difference of the outline of the micro droplet, the outer circle is filled with the gray level of 80, the inner circle is filled with the gray level of 210, a template in the shape of a ring is obtained, as shown in fig. 4, and the next matching process uses the template or the affine transformation form thereof.
Preferably, before the target micro-droplet positioning and counting step, the method further comprises: and an identification Region framing step, configured to frame a partial Region of the first image as an identification Region to perform matching identification on the microdroplets in the identification Region, where the identification Region may also be regarded as a Region of interest (ROI), and specifically, a center point of the ROI is used as a reference point of the template for estimating a position, rotation, and scaling, and selecting the ROI may avoid finding microdroplets on the entire image, so as to save matching time.
Further, after the template generating and specifying step before the target micro-droplet positioning and counting step, the method further comprises: and a matching parameter adjusting step, which is used for adjusting the matching parameters in the searching and identifying process so as to improve the identifying efficiency and/or the accuracy of the identifying result. For example, the matching parameters may include: at least one of a contrast threshold parameter, a visibility parameter, an image pyramid layer number, a rotation angle parameter, a zoom parameter, a search completeness parameter, and an inter-object maximum overlap area ratio parameter.
Specifically, since the droplet may be squeezed instead of a perfect circle, but rather an ellipse, an anisotropic matching mode is adopted, since the ellipse is an axisymmetric image, the rotation angle parameter allowed by matching is set to 0-90 ° (angle extend), the zoom range of the vertical axis and the zoom range of the horizontal axis (zoom parameter) are both set to 0.5-1.3(scaleR, scaleC), and an excessively large or excessively small recognition object is generally considered as a bubble, so that a meaningful zoom range is required to find the micro-droplet object in the range near the size of the template. To speed up the matching process, an image pyramid is used, which consists of an initial full-size image and a series of down-sampled images. The template will be generated and searched at different pyramid levels. The image pyramid is configured using an image pyramid layer number (NumLevels) parameter. Specifically, when the size of the template is only 40 × 40, the number of pyramid layers (NumLevels) of the image may be 1.
The above parameters are relatively fixed for identification matching of the liquid drop, and the parameters described below need to be adjusted to determine a relatively proper value through identification effect, so that the adjustment of the parameters is more universal. A Contrast threshold parameter (Contrast) specifies what Contrast points on the search image must be compared to the template. The main effect of this parameter is to exclude noise (e.g. grey value fluctuations during matching). The visibility parameter (MinScore) specifies how much of the template must be visible, the larger the visibility parameter (MinScore), the faster the search; the search thoroughness parameter (Greediness) determines the thoroughness of the search, and the parameters are adjusted to obtain the best micro-droplet matching effect, generally have a large influence on the search matching speed, and take different values for images with different overall gray levels. The following arrangement enables a successful match in all test images, i.e. all object instances are found. In some cases, if the search completeness parameter (Greediness) is too large, it may not be possible to find an object that is completely visible. Selecting the value 0 forces a thorough search. And (3) taking 1 for the micro-droplet identification search completeness parameter (greeness) does not influence the matching result at all. If the object is still to be recognized in a state where the Contrast of the recognition object is low, the Contrast threshold parameter (Contrast) is decreased. Fig. 5 shows the search results when the Contrast threshold parameter (Contrast) is equal to 50 and 30, respectively. The maximum overlap area ratio parameter (maxooverlap) determines the ratio of the overlapped part between the objects to the size of the template, and in this embodiment, the outline of the droplet is basically not overlapped, but the template image for matching is a rectangle with the size of 40 × 40, so that the template rectangles of the matched objects may be partially overlapped, and the parameter cannot be set too small. As shown in fig. 6, which is a search result when the maximum overlap area ratio parameter (maxooverlap) is equal to 0.1 and 0.3, respectively, it can be found that the close microdroplets stuck in the left graph are not recognized.
The parameters are adjusted in order to optimize the matching speed. As long as the matching is successful, the visibility parameter (MinScore) is increased as much as possible, and as shown in fig. 7, the search results when the visibility parameter (MinScore) is equal to 0.9 and 0.7, respectively, the completeness of the search (greeningess) is increased until the matching fails, and the visibility parameter (MinScore) is tried to be decreased if the previous value is not helped to be restored. The Min Contrast threshold parameter (Contrast) is increased as much as possible as long as the matching is successful. Under the condition of good matching effect (the complete microdroplet objects in the allowable size range can be identified), the speed of optimizing search matching can reach 800ms for one image. The method obtains a better identification effect, simultaneously controls the calculation time within a reasonable range, and better balances the relation between the micro-droplet image identification time and the efficiency.
It is readily understood by a person skilled in the art that the advantageous ways described above can be freely combined, superimposed without conflict.
The present invention is not limited to the above preferred embodiments, and any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention. The above is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the technical principle of the present invention, and these improvements and modifications should also be regarded as the protection scope of the present invention.