CN115268051B - Automatic high-flux image acquisition and splicing system for micro-nano structure - Google Patents

Abstract

The embodiment of the invention provides a micro-nano structure image high-flux automatic acquisition and splicing system, wherein two paths of electric push rods are connected outside an XY two-axis translation objective table of a micro-nano microscope, a manually moved shot sample is changed into electric motion, the micro-distance movement control and image acquisition are carried out on the micro-nano microscope through an image acquisition program, a micro-nano structure image sample can be automatically acquired while the objective table is moved at equal intervals with small errors, a sample panoramic image is automatically spliced through a microscopic image panoramic image splicing algorithm, the full automation of acquisition and splicing of a high-flux microscopic image of a large-size micro-nano structure sample is realized, and the efficiency and the precision of a subsequent detection and analysis link are improved.

Description

A high-throughput automatic collection and stitching system for micro-nano structure images

technical field

Embodiments of the present invention relate to the technical field of micro-nano structure image analysis, in particular to a high-throughput automatic collection and stitching system for micro-nano structure images.

Background technique

In the prior art, a microscope is usually used to magnify the pattern of the micro-nano structure, and the characteristic information of the micro-nano structure is collected by manually moving the stage to take pictures of the micro-nano structure, which is convenient for subsequent analysis. After the micro-nano structure pattern is magnified by the microscope, the overall view of the micro-nano structure sample is reduced, and it is difficult to locate the current micro-nano structure pattern in the specific position of the sample, which affects the analysis of the micro-nano structure; During the process of taking picture samples by the object stage, it is necessary to record the approximate position of the current pattern in real time, so as to locate the problematic position of the micro-nano structure sample in the subsequent detection and analysis process.

However, manually moving the platform to take pictures of samples can only be used for low-throughput micro-nano structure pattern shooting and collection for small-sized micro-nano structure samples, for example, the micro-nano structure with a micro-nano unit size of 25×15 is magnified by 4 times. In this case, it is necessary to manually move the platform for at least 25 times, and it is impossible to ensure the equidistant movement with small errors during the movement process. The repeated position of the adjacent sample pattern will also affect the statistics of the final analysis results, and when the size of the micro-nano structure sample When it is necessary to take pictures of high-throughput samples, the efficiency of manually moving the platform will be very low. Therefore, it is urgent to provide a high-throughput automatic collection and stitching system for micro-nanostructure images, which can realize automatic high-throughput image capture of large-scale micro-nanostructure samples with small errors and equal intervals under high magnification The collected image samples are automatically stitched together. In the state of the panorama, it can provide a full picture of the sample for subsequent analysis, which is convenient for quickly and accurately locating the problematic position of the micro-nano structure sample during subsequent detection and analysis. The panorama also retains clear feature information when zoomed in, providing sufficient morphological data for subsequent detection and analysis, and improving the efficiency of subsequent detection and analysis as a whole.

Contents of the invention

An embodiment of the present invention provides a high-throughput automatic collection and stitching system of micro-nano structure images to solve the problem of low efficiency in high-throughput detection of large-sized micro-nano structure samples in the prior art.

In order to solve the above technical problems, the embodiment of the present invention provides a high-throughput automatic collection and stitching system of micro-nano structure images, including a microscope electric platform, a microscopic image acquisition unit and a microscopic image panoramic stitching unit;

The microscope electric platform includes a multi-axis translation stage, and the multi-axis translation stage is equipped with a micro-nano structure sample, which is used to carry the micro-nano structure sample for directional and fixed-distance movement according to the control command. The spatial position of the micro-nano structure sample is preliminarily adjusted, and the micro-nano structure sample is moved to the image acquisition range of the micro-nano structure microscope;

The microscopic image acquisition unit includes an image acquisition module and an electric platform macro movement module, and the electric platform macro movement module is used to control the movement of the multi-axis translational stage to carry out the micro-nano structure sample Macro movement, the image acquisition module is used to control the micro-nano structure microscope to perform high-throughput image acquisition on the micro-nano structure sample, and obtain the micro-nano structure at different positions of the micro-nano structure sample under the preset magnification Microscopic image;

The microscopic image panorama stitching unit is used for panoramic stitching of the micro-nanostructure microscopic images.

Preferably, the multi-axis translation stage includes an XY two-axis translation stage; the XY two-axis translation stage includes an X-axis moving guide rail, a Y-axis moving guide rail and an object stage installation position, The X-axis moving guide rail and the Y-axis moving guide rail form a continuously movable guide rail, the stage mounting position is installed on the continuously movable guide rail, and the stage mounting position is used for mounting a stage.

Preferably, the microscope electric platform also includes a motor drive structure and at least two stepping motors;

The motor transmission structure includes a conveyor belt, a gear and a push rod, and the push rod includes a first push rod and a second push rod; the mounting position of the stage is slidably installed on the continuously movable guide rail, and the The sliding direction of the stage mounting position is parallel to the X-axis moving guide rail or the Y-axis moving guide rail, and one end of the first push rod is connected to the object stage mounting position along a direction parallel to the X-axis moving guide rail or the Y-axis moving guide rail, so The other end of the first push rod is connected to the stepping motor; the stage is slidably mounted on the mounting position of the stage, and the sliding direction of the stage is in line with the Y-axis moving guide rail or the X-axis moving guide rail Parallel, one end of the second push rod is connected to the stage in a direction parallel to the Y-axis moving guide rail or the X-axis moving guide rail, and the other end of the second push rod is connected to a stepping motor;

The conveyor belt and the gears form a transmission group, and the transmission group is connected to the X-axis moving guide rail and the Y-axis moving guide rail to control the X-axis moving guide rail and the Y-axis moving guide rail to drive the moving platform.

Preferably, a first concave rail is provided at the bottom of the stage mounting position, a second concave rail is provided at the top of the stage mounting position, and the first concave rail and the second concave rail are perpendicular to each other; The stage installation position is slidably installed on the X-axis moving guide rail or the Y-axis movement guide rail through the first concave rail, and the object stage is installed on the object stage installation position through the second concave rail .

Preferably, the end of the push rod is provided with an n-type ferrule, and the n-type ferrule is fitted to both ends of the stage. When the push rod stretches, the movement of the stage in two directions is realized.

Preferably, the multi-axis translation stage further includes a lifting platform, the lifting platform is connected with a stepping motor, the stepping motor and the motor transmission structure are relatively fixed on the lifting platform, and the The positions of the stepping motor, the motor transmission structure and the object stage are relatively fixed.

As a preference, the stepper motor and the motor transmission structure are fixedly connected to the object stage through stainless steel corner brackets and stainless steel circular saddles.

Preferably, the stepping motor is a reactive stepping motor, a permanent magnet stepping motor or a hybrid stepping motor, and the minimum step angle of the stepping motor is millimeter level.

Preferably, the microscopic image panorama stitching unit includes a data shape construction module, a core stitching module, a lightweight stitching module and a multi-layer stitching module;

The data shape construction module is used to convert the collected micro-nano structure microscopic image from a one-dimensional vector into an M*N two-dimensional vector to store the micro-nano structure microscopic image;

The lightweight splicing module is used to splice each row/column image list in the M*N two-dimensional vector into a panorama, and when the quality of the panorama is not up to standard, input the corresponding row/column image list to the multi-layer splicing module ;

The core stitching module is used to stitch two images into a small panorama;

The multi-layer stitching module is used to use the input row/column image list as the first layer image, and combine every two adjacent images in the first layer image into the core stitching module, and combine the adjacent two images The generated small panorama constitutes the second layer image, and every two adjacent images in the second layer image are combined into the core stitching module, and the small panorama generated by the adjacent two images constitutes the third layer image, until The number of images in the last layer of images is 1; the generated list of combined images in each layer of images is input to the lightweight splicing module.

Preferably, the data shape construction module is specifically used to adaptively adjust the number of rows and columns of the constructed two-dimensional vector based on the number of micro-nanostructure microscopic images, so as to capture micro-nanostructure microscopic images taken under different sizes and different magnifications. When the number of images is different, M*N two-dimensional vectors of different sizes are adaptively constructed to store micro-nano structure microscopic images.

Preferably, the lightweight stitching module performs multi-image stitching based on the Stitcher class carried by OpenCV; if there is uneven brightness and lack of splicing in the stitched panorama, it is judged that the quality of the panorama is not up to standard.

Preferably, the core stitching module is used to find the key feature points of the two images through the scale-invariant feature transformation SIFI algorithm, and match the key feature points of the overlapping parts of the two images based on the fast nearest neighbor search FLANN algorithm to generate corresponding The transformation matrix between the feature points is used to perform multi-band fusion based on the relationship of the transformation matrix to stitch two adjacent images.

The embodiment of the present invention provides a high-throughput automatic collection and stitching system for micro-nano structure images. By connecting two electric push rods outside the XY two-axis translation stage of the micro-nano microscope, the manual movement of the photographed sample is changed to an electric one. , through the image acquisition program to control the micro-nano microscope's micro-nano movement and image acquisition, it can automatically collect micro-nano structure image samples while moving the stage at equal intervals with small errors, and automatically stitch the samples through the microscopic image panorama stitching algorithm The panorama image realizes full automation of high-throughput microscopic images of large-scale micro-nano structure samples from collection to splicing, which improves the efficiency and accuracy of subsequent detection and analysis links.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description These are some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

Fig. 1 is a connection structure diagram of an electric push rod and an experimental lifting platform according to an embodiment of the present invention;

Fig. 2 is a connection structure diagram between the end of the electric push rod and the XY two-axis translation stage according to the embodiment of the present invention;

FIG. 3 is a flowchart of a panorama stitching algorithm according to an embodiment of the present invention;

Fig. 4 is a flow chart of lightweight module splicing according to an embodiment of the present invention;

Fig. 5 is a flow chart of splicing a multi-layer splicing module according to an embodiment of the present invention;

FIG. 6 is a splicing flowchart of a core splicing module according to an embodiment of the present invention;

Fig. 7 is a schematic diagram of splicing of multi-layer splicing modules according to an embodiment of the present invention.

detailed description

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

The term "and/or" in the embodiment of the present application is only an association relationship describing associated objects, which means that there may be three relationships, for example, A and/or B, which can mean: A exists alone, and A and B exist at the same time , there are three cases of B alone.

The terms "first" and "second" in the embodiments of the present application are used for description purposes only, and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity of indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include at least one of these features. In the description of the present application, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a system, product or equipment comprising a series of components or units is not limited to the listed components or units, but optionally also includes components or units not listed, or optionally also includes Other parts or units inherent in equipment. In the description of the present application, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

Reference herein to an "embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The occurrences of this phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is understood explicitly and implicitly by those skilled in the art that the embodiments described herein can be combined with other embodiments.

In the field of micro-nano structure image acquisition and analysis, in the prior art, the manual movement of the platform is usually used to take pictures of samples, and only small-sized micro-nano structure samples can be photographed and collected with low-throughput micro-nano structure patterns, for example, for 25×15 micro-nano structure samples In the case of nano-unit-sized micro-nano structures at 4 times magnification, it is necessary to manually move the platform to take pictures at least 25 times. During the movement process, it is impossible to ensure equidistant movement with small errors, and the repeated positions of pictures of adjacent sample patterns will also affect the final The statistics of the analysis results are affected, and when the size of the micro-nano structure sample increases and high-throughput sample pictures need to be taken, the efficiency of manually moving the platform to take pictures will be very low. Therefore, an embodiment of the present invention provides a high-throughput automatic collection and stitching system for micro-nano structure images, which can realize automatic high-throughput image capture with small errors and equal intervals for large-scale micro-nano structure samples under high-magnification conditions, and Automatic panorama stitching of the collected image samples. In the state of the panorama, it can provide a full picture of the sample for subsequent analysis, which is convenient for quickly and accurately locating the problematic position of the micro-nano structure sample during subsequent detection and analysis. The panorama also retains clear feature information when zoomed in, providing sufficient morphological data for subsequent detection and analysis, and improving the efficiency of subsequent detection and analysis as a whole. The following will describe and introduce through multiple embodiments.

Figure 1 and Figure 2 are a high-throughput automatic collection and stitching system for micro-nano structure images provided according to an embodiment of the present invention, which is suitable for the detection of various micro-nano structure samples and can be applied in biomedicine, environmental science and imaging science field, the system includes a microscope electric platform, a microscopic image acquisition unit and a microscopic image panorama stitching unit;

The microscope electric platform includes a multi-axis translation stage, and the multi-axis translation stage is equipped with a micro-nano structure sample, which is used to carry the micro-nano structure sample for directional and fixed-distance movement according to the control command. The spatial position of the micro-nano structure sample is preliminarily adjusted, and the micro-nano structure sample is moved to the image acquisition range of the micro-nano structure microscope;

The multi-axis translational stage in this embodiment includes an XY two-axis translational stage 5, a stepping motor 1, a motor transmission structure and an experimental lifting platform 2, which are used to realize the electric control of the orientation of the microscope stage. The distance movement also includes a right-angle corner code 3 and a horse riding card 4, which are used to fix the transmission structure of the stepping motor 1 on the experimental lifting platform 2.

Specifically, as shown in Figures 1 and 2, the XY two-axis translation stage 5 includes an X-axis moving guide rail, a Y-axis moving guide rail, and an object stage mounting position, and also includes a Y-direction transmission rod 6, which It is in direct contact with the Y-axis moving guide rail, and the X-direction transmission rod 7 is in direct contact with the X-axis moving guide rail. The X-axis moving guide rail and the Y-axis moving guide rail are continuously movable guide rails. The X-axis moving guide rail and the Y-axis moving guide rail The moving guide rails form a continuously movable guide rail, and the mounting position of the object stage is installed on the continuously movable guide rail, and the mounting position of the object stage is used to install the object stage. The stage of the micro-nano microscope has a guide rail that can move continuously in the X and Y axes. It will be put into the preset round hole on the side of the XY two-axis translation stage 5, the preset round hole in the X direction transmission rod 7 vertically penetrates the screw hole 8, and the screw can pass through the circle of the X direction transmission rod 7. holes, so that the X-direction transmission rod 7 is connected and fixed with the XY two-axis translation stage 5.

The motor transmission structure includes a conveyor belt, a gear and a push rod, and the push rod includes a first push rod and a second push rod; the mounting position of the stage is slidably installed on the continuously movable guide rail, and the The sliding direction of the stage mounting position is parallel to the X-axis moving guide rail or the Y-axis moving guide rail, and one end of the first push rod is connected to the object stage mounting position along a direction parallel to the X-axis moving guide rail or the Y-axis moving guide rail, so The other end of the first push rod is connected to the stepper motor 1; the stage is slidably installed on the stage mounting position, and the sliding direction of the stage is in line with the Y-axis moving guide rail or the X-axis movement The guide rails are parallel, one end of the second push rod is connected to the stage in a direction parallel to the Y-axis moving guide rail or the X-axis moving guide rail, and the other end of the second push rod is connected to the stepping motor 1, in this embodiment Among them, the stepping motor 1 includes a first stepping motor and a second stepping motor, wherein the first push rod corresponds to the first stepping motor, and the second push rod corresponds to the second stepping motor; wherein the first push rod, The second push rod is an electric push rod.

The stage can be driven in different directions according to the rotation direction of the motor. For example, the end of the push rod is directly connected with the stage through screws to realize the transmission in both directions of push and pull; Both ends of the stage, when the push rod stretches and retracts, the movement of the stage in two directions is realized.

The conveyor belt and the gears form a transmission group, and the transmission group is connected to the X-axis moving guide rail and the Y-axis moving guide rail to control the X-axis moving guide rail and the Y-axis moving guide rail to drive the moving platform.

The motor transmission structure can be connected with the stage through the mechanical structure, so as to drive the stage to perform macro-directional movement, such as the push rod head is connected with the stage, and the stage is moved by pushing and pulling; the conveyor belt and gears are connected with the stage The X-Y shafts are connected to each other, and the stage is driven by rotation.

The bottom of the stage mounting position is provided with a first recessed rail, and the top of the stage mounting position is provided with a second recessed rail, and the first recessed rail and the second recessed rail are perpendicular to each other; The stage mounting position is slidably installed on the X-axis moving guide rail or the Y-axis moving guide rail through the first concave rail, and the object stage is installed on the object stage mounting position through the second concave rail.

The multi-axis translation stage also includes a lifting platform, the range of the moving guide rail is a circle outside the stage, and then there is a position for connecting the transmission structure on the stage, the lifting platform is connected with the stepping motor, and then the horizontal Put it on the desktop, with non-slip material or structure on the bottom, and the lifting platform as a whole is only connected to the stepping motor. The stepper motor 1 and the motor transmission structure are relatively fixed on the lifting platform, and the positions of the stepping motor 1, the motor transmission structure and the loading platform are relatively fixed; the lifting platform is used to adjust The stepping motor 1 and its transmission structure are at the required height, and the lifting platform has a structure connected with the stepping motor 1 and its transmission structure, so as to ensure the stepping motor 1 and its transmission structure, the loading platform and the experimental lifting platform when the system is in operation 2 Relative displacement will not occur, such as connecting the stainless steel corner code and the stainless steel circular riding card to fix the stepper motor 1, the push rod.

Wherein, the stepping motor 1 is a reactive stepping motor 1 , a permanent magnet stepping motor 1 or a hybrid stepping motor 1 , and the minimum step angle of the stepping motor 1 is millimeter level.

As shown in Figure 1 and Figure 2, there are two 32 two-phase stepping motors, electric push rods and improved lifting platforms with connection structures; among them, the first 32 two-phase stepping motors, the first electric push rod and the second 32 The two-phase stepping motor and the second electric push rod are fixed on the top of the experimental lifting platform 2, and the height can be adjusted according to different microscope platforms, and the end of the rod can be connected with the 5 walls of the XY two-axis translation stage; There are grooves and screw holes on the rear side of the moving stage, which can be used to install and connect the rod end of the electric push rod of 32 two-phase stepping motors. The schematic diagram of its structure is shown in Figure 2; ×120(mm), and screw holes can be set at the specified places to connect the stainless steel corner bracket and the stainless steel round riding card, so that the electric push rod of the motor can be fixed on its top surface. The schematic diagram of its structure is shown in Figure 3.

The microscopic image acquisition unit includes an image acquisition module and an electric platform macro movement module, and the electric platform macro movement module is used to control the movement of the multi-axis translational stage to carry out the micro-nano structure sample Macro movement, the image acquisition module is used to control the micro-nano structure microscope to perform high-throughput image acquisition on the micro-nano structure sample, and obtain the micro-nano structure at different positions of the micro-nano structure sample under the preset magnification microscopic image.

In this example, according to the size design of the micro-nano structure sample and the magnification of the micro-nano microscope, different acquisition schemes are adopted for the micro-nano structure sample, and the image acquisition program (algorithm) of the micro-image acquisition system and the electric platform micro Adjust different parameters from the mobile control program to complete the automatic image acquisition; among them, the image acquisition program of the PC-side upper computer (ie, the image acquisition module) and the electric platform macro movement program of the lower computer (ie, the electric platform macro movement module) , used to automatically collect high-throughput micro-nano structure microscopic images, the host computer on the PC end communicates with the lower-position computer through USB or serial ports, and the PC end collects images while the micro-nano structure micro-electric platform moves to realize the micro-nano structure High-throughput image acquisition is fully automated.

Specifically, different sizes of dielectrophoretic display devices are selected and placed on the motorized stage of the microscope. The PC end can clearly display the enlarged image of the dielectrophoretic display device through the CCD-driven imaging software, and the arduino uno is selected as the lower computer to run the motorized platform. Micro-distance mobile program, choose arduino A4988 two-way stepper motor drive expansion board to drive 32 two-phase stepper motor electric push rods, complete the signal circuit wiring of the upper computer, lower computer and actuator, and complete the energy circuit between the actuator and the DC power supply Wiring, according to the size of the dielectrophoretic display device, change the parameters of the image acquisition program and the macro movement program of the electric platform according to the acquisition plan form, and start the program to automatically complete the acquisition of high-throughput image samples of micro-nano structures. The acquisition plan is shown in Table 1 below.

The microscopic image panorama stitching unit is used for panoramic stitching of the micro-nanostructure microscopic images.

In this embodiment, the microscopic image panorama stitching unit includes a data shape construction module, a core stitching module, a lightweight stitching module and a multi-layer stitching module;

The data shape construction module is used to convert the collected micro-nano structure microscopic image from a one-dimensional vector to M*N two-dimensional vector to store the micro-nano structure microscopic image; based on the number of micro-nano structure microscopic images from Adapt to adjust the number of rows and columns of the constructed two-dimensional vector, so that when the number of micro-nano structure microscopic images taken under different sizes and different magnifications is different, adaptively construct M*N two-dimensional vectors of different sizes to store the micro-nano structure Microscopic image, its vector changes as follows:

p ₁ , p ₂ , p ₃ , p ₄

p ₁ , p ₂ , p ₃ , p ₄ , ...p _n-1 ,p _n ——> p ₅ , p ₆ , p ₇ , p ₈

...

p _n-3 , p _n-2 , p _n-1 , p _n

The position of each picture Pi in the two-dimensional vector should correspond to the panorama.

The lightweight splicing module is used to splice each row/column image list in the M*N two-dimensional vector into a panorama, and when the quality of the panorama is not up to standard, input the corresponding row/column image list to the multi-layer splicing module ; The lightweight stitching module is based on the Stitcher class that comes with OpenCV to stitch the images in each row/column image list in the M*N two-dimensional vector into multiple images, and stitch them into a panorama and store them separately; if the stitched panorama exists If the brightness is uneven and the splicing is missing, the quality of the panorama is judged to be substandard. OpenCV’s Stitcher class of multi-image stitching has a high splicing speed, but it is prone to uneven brightness and missing splicing. Therefore, the lightweight stitching module generates The panorama is inspected for quality. When there is serious brightness unevenness and lack of splicing, the row of image lists will be re-entered into the multi-layer stitching module for re-splicing, ensuring splicing speed and good splicing quality at the same time.

For the M*N two-dimensional vector constructed by the data shape construction module, the image list is read row by row (column) from the first row (column), and the images in the image list (that is, micro-nano structure microscopic images) are first Input to the lightweight splicing module for splicing. The lightweight module flow is shown in Figure 4. The Stitcher class based on OpenCV quickly stitches the input mth row (column) image list, and then outputs the splicing of the row (column) image picture;

Then perform quality inspection on the mosaic image to detect whether there is a serious brightness difference between each sub-image of the mosaic image, and judge whether there is an image missing in this light-weight mosaic by the length of the mosaic image. If the quality is up to standard, store the result of this mosaic , if not up to standard, the image list of the row (column) is input into the multi-layer splicing module for splicing.

The multi-layer stitching module is used to use the input row/column image list as the first layer image, and combine every two adjacent images in the first layer image into the core stitching module, and combine the adjacent two images The generated small panorama constitutes the second layer image, and every two adjacent images in the second layer image are combined into the core stitching module, and the small panorama generated by the adjacent two images constitutes the third layer image, until The number of images in the last layer of images is 1; the generated list of combined images in each layer of images is input to the lightweight splicing module.

In this embodiment, the implementation steps of the multi-layer splicing module include taking the original input row image list as the first layer image, using the step selector to continuously select two adjacent pictures and sending them to the core splicing module for splicing. The two generated small panoramas constitute the second layer of images. Assuming that the original input image list is finally stored for the n images to be stitched, the stitching process will generate n layers of images, and each layer except the first layer is larger than There is one less image to be stitched in the previous layer, and the overlapping area of adjacent images will increase layer by layer to ensure the high quality of multi-image fusion.

As shown in Figure 5 and Figure 7, the multi-layer splicing module uses a step selector to continuously select two adjacent pictures from the input m row (column) image list and send them to the core splicing module for splicing. The two generated small panoramas form the second layer of images. In this specific embodiment, the original input image list stores 5 images to be spliced, and the splicing process will generate 5 layers of images. Each layer except the first layer There is one less image to be stitched than the previous layer, and the overlapping area of adjacent images will increase layer by layer to ensure the high quality of multi-image fusion. The flow chart of the stitching process is shown in Figure 5, and the schematic diagram is shown in Figure 7.

The core stitching module is used to stitch two images into a small panorama; the core stitching module is used to find the key feature points of the two images through the scale-invariant feature transformation SIFI algorithm, and based on the fast nearest neighbor search FLANN The algorithm matches the key feature points of the overlapping parts of the two images to generate a transformation matrix between the corresponding feature points, and performs multi-band fusion based on the relationship of the transformation matrix to stitch two adjacent images.

Finally, the stitching result images of each row (column) are formed into a new image list, and the above steps are repeated for the image list to perform the final stitching.

Various embodiments of the present invention can be combined arbitrarily to achieve different technical effects.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the present application will be generated in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server, or data center by wired (eg, coaxial cable, optical fiber, DSL) or wireless (eg, infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available media may be magnetic media (eg, floppy disk, hard disk, magnetic tape), optical media (eg, DVD), or semiconductor media (eg, SolidState Disk).

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments are realized. The processes can be completed by computer programs to instruct related hardware. The programs can be stored in computer-readable storage media. When the programs are executed , may include the processes of the foregoing method embodiments. The aforementioned storage medium includes: ROM or random access memory RAM, magnetic disk or optical disk, and other various media that can store program codes.

Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: it can still be Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent replacements are made to some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims (11)

1. A micro-nano structure image high-flux automatic acquisition and splicing system is characterized by comprising a microscope electric platform, a microscopic image acquisition unit and a microscopic image panoramic splicing unit;

the microscope electric platform comprises a multi-axis translation objective table, wherein a micro-nano structure sample is carried on the multi-axis translation objective table and used for carrying the micro-nano structure sample to move at a fixed direction and a fixed distance according to a control instruction, preliminarily adjusting the spatial position of the micro-nano structure sample and moving the micro-nano structure sample to the image acquisition range of the micro-nano structure microscope;

the microscopic image acquisition unit comprises an image acquisition module and an electric platform micro-distance moving module, the electric platform micro-distance moving module is used for controlling the multi-axis translation object stage to move so as to carry out micro-distance movement on the micro-nano structure sample, and the image acquisition module is used for controlling the micro-nano structure microscope to carry out high-flux image acquisition on the micro-nano structure sample so as to acquire micro-nano structure microscopic images of the micro-nano structure sample at different positions under the condition of preset magnification;

the microscopic image panoramic stitching unit is used for performing panoramic stitching on the micro-nano structure microscopic image; the microscopic image panoramic stitching unit comprises a data shape construction module, a core stitching module, a lightweight stitching module and a multilayer stitching module;

the data shape construction module is used for converting the collected micro-nano structure microscopic image from a one-dimensional vector to an M x N two-dimensional vector so as to store the micro-nano structure microscopic image;

the lightweight splicing module is used for splicing each row/column image list in the M-by-N two-dimensional vector into a panorama, and when the quality of the panorama does not reach the standard, the corresponding row/column image list is input into the multilayer splicing module;

the core splicing module is used for splicing the two images into a small panoramic image;

the multi-layer splicing module is used for taking an input line/column image list as a first layer image, combining and inputting every two adjacent images in the first layer image into the core splicing module, forming a second layer image by using small panoramic pictures generated by the two adjacent images, combining and inputting every two adjacent images in the second layer image into the core splicing module, and forming a third layer image by using the small panoramic pictures generated by the two adjacent images until the number of images in the last layer image is 1; and inputting a list consisting of combined images in the generated images of each layer into the lightweight splicing module.

2. The micro-nano structure image high-flux automatic acquisition and splicing system according to claim 1, wherein the multi-axis translation stage comprises an XY two-axis translation stage; the XY two-axis translation objective table comprises an X-axis moving guide rail, a Y-axis moving guide rail and an objective table mounting position, the X-axis moving guide rail and the Y-axis moving guide rail form a continuously movable guide rail, the objective table mounting position is mounted on the continuously movable guide rail, and the objective table mounting position is used for mounting the objective table.

3. The micro-nano structure image high-flux automatic acquisition and splicing system according to claim 2, wherein the microscope electric platform further comprises a motor transmission structure and at least two stepping motors;

the motor transmission structure comprises a conveyor belt, a gear and a push rod, and the push rod comprises a first push rod and a second push rod; the object stage installation position is slidably installed on the continuously movable guide rail, the sliding direction of the object stage installation position is parallel to the X-axis movable guide rail or the Y-axis movable guide rail, one end of the first push rod is connected with the object stage installation position along the direction parallel to the X-axis movable guide rail or the Y-axis movable guide rail, and the other end of the first push rod is connected to the stepping motor; the object stage is arranged on the object stage installation position in a sliding mode, the sliding direction of the object stage is parallel to the Y-axis moving guide rail or the X-axis moving guide rail, one end of the second push rod is connected with the object stage along the direction parallel to the Y-axis moving guide rail or the X-axis moving guide rail, and the other end of the second push rod is connected to the stepping motor;

the transmission group is connected with the X-axis moving guide rail and the Y-axis moving guide rail so as to control the X-axis moving guide rail and the Y-axis moving guide rail to drive the objective table to move.

4. The micro-nano structure image high-throughput automatic acquisition and splicing system according to claim 3, wherein a first concave rail is arranged at the bottom of the object stage installation position, a second concave rail is arranged at the top of the object stage installation position, and the first concave rail and the second concave rail are perpendicular to each other; the object stage installation position is arranged on the X-axis moving guide rail or the Y-axis moving guide rail in a sliding mode through the first concave rail, and the object stage is arranged on the object stage installation position through the second concave rail.

5. The micro-nano structure image high-throughput automatic acquisition and splicing system according to claim 3, wherein an n-type clamping sleeve is arranged at the tail end of the push rod, the n-type clamping sleeve is sleeved at two ends of the objective table, and the objective table moves in two directions when the push rod stretches.

6. The micro-nano structure image high-flux automatic acquisition and splicing system according to claim 3, wherein the multi-axis translation stage further comprises a lifting table, the lifting table is connected with a stepping motor, the stepping motor and the motor transmission structure are relatively fixed on the lifting table, and the positions of the stepping motor, the motor transmission structure and the stage are relatively fixed.

7. The micro-nano structure image high-flux automatic acquisition and splicing system according to claim 6, wherein the stepping motor and the motor transmission structure are fixedly connected with the objective table through a stainless steel corner brace and a stainless steel circular horse riding.

8. The micro-nano structure image high-flux automatic acquisition and splicing system according to claim 3, wherein the stepping motor is a reaction type stepping motor, a permanent magnet type stepping motor or a hybrid type stepping motor, and the minimum stepping angle of the stepping motor is in millimeter level.

9. The system for automatically acquiring and splicing the micro-nano structure image at high flux according to claim 1, wherein the data shape construction module is specifically used for adaptively adjusting the number of rows and columns of constructed two-dimensional vectors based on the number of micro-nano structure microscopic images, so that when the number of the micro-nano structure microscopic images shot under different sizes and different magnification factors is different, M x N two-dimensional vectors with different sizes are adaptively constructed to store the micro-nano structure microscopic images.

10. The micro-nano structure image high-throughput automatic acquisition and splicing system according to claim 1, wherein the lightweight splicing module performs multi-graph splicing based on a Stitcher class of OpenCV; and if the spliced panoramic image has uneven brightness and splicing loss, judging that the quality of the panoramic image does not reach the standard.

11. The micro-nano structure image high-throughput automatic acquisition and splicing system according to claim 9, wherein the core splicing module is used for searching key feature points of two images through a scale invariant feature transform (SIFI) algorithm, matching the key feature points of the overlapped part of the two images based on a fast nearest neighbor search (FLANN) algorithm to generate a transformation matrix between the corresponding feature points, and performing multi-band fusion based on the relation of the transformation matrix to splice the two adjacent images.

Images