CN113391441B

CN113391441B - Method and device for acquiring microscopic image information of sample

Info

Publication number: CN113391441B
Application number: CN202010171385.3A
Authority: CN
Inventors: 张大庆
Original assignee: Pinghu Laidun Optical Instrument Manufacturing Co ltd
Current assignee: Pinghu Laidun Optical Instrument Manufacturing Co ltd
Priority date: 2020-03-12
Filing date: 2020-03-12
Publication date: 2024-02-02
Anticipated expiration: 2040-03-12
Also published as: CN113391441A

Abstract

It is an object of the present application to provide a method and apparatus for acquiring microscopic image information of a sample, comprising: and sending image acquisition instruction information about a sample to a corresponding microscopic device, wherein the image acquisition instruction information is used for controlling the microscopic device so as to enable an objective lens of the microscopic device to generate relative motion relative to the sample, and acquiring microscopic sub-image information about the sample through the microscopic device in the process of the relative motion, wherein the relative motion comprises relative plane motion of the objective lens relative to a stage corresponding to the sample in a stage corresponding plane direction, and relative back and forth motion of the objective lens relative to the stage in a perpendicular direction of the stage corresponding plane, and the relative plane motion at least partially overlaps with the relative back and forth motion. The microscopic sub-image information can be obtained rapidly and accurately, the precision of the microscopic image information is improved, and the use experience is improved.

Description

Method and device for acquiring microscopic image information of sample

Technical Field

The present application relates to the field of microscopic imaging, and in particular to a technique for acquiring microscopic image information of a sample.

Background

Microscopic optical imaging, also commonly referred to as "optical Microscopy", or "optical Microscopy" (Optical Microscopy, or Light Microscopy), refers to a technique whereby an enlarged image of a tiny sample can be obtained after passing through or reflecting visible Light back from the sample through one or more lenses. The obtained image can be directly observed by eyes through an ocular lens, can be recorded by a photosensitive plate or a digital image detector such as CCD (charge coupled device) and CMOS (complementary metal oxide semiconductor), and can be displayed and analyzed on a computer. Of course, by combining with the image pickup device, it is also possible to record a video or the like concerning the sample in the field of view. However, the scope of view that the microscope can observe is limited, and when the size of the observed sample exceeds the current scope of view, it is difficult to observe the condition of the whole sample, and when there is a certain height difference between the samples in the current scope of view, the observation result at a certain focal plane height is also not ideal.

Disclosure of Invention

It is an object of the present application to provide a method and apparatus for acquiring microscopic image information of a specimen.

According to one aspect of the present application, there is provided a method for acquiring microscopic image information of a sample, for use in a user device, the method comprising:

And sending image acquisition instruction information about a sample to a corresponding microscopic device, wherein the image acquisition instruction information is used for controlling the microscopic device so as to enable an objective lens of the microscopic device to generate relative motion relative to the sample, and acquiring microscopic sub-image information about the sample through the microscopic device in the process of the relative motion, wherein the relative motion comprises relative plane motion of the objective lens relative to a stage corresponding to the sample in the direction of the stage corresponding to the stage, and relative back and forth motion of the objective lens relative to the stage in the vertical direction of the stage corresponding to the plane, and the relative plane motion at least partially overlaps with the relative back and forth motion in the process of the relative motion.

According to another aspect of the present application, there is provided a method for acquiring microscopic images of a specimen, applied to a microscopic apparatus, the method comprising:

receiving image acquisition instruction information about a sample, which is sent by corresponding user equipment;

and controlling the microscopic equipment according to the image acquisition instruction information so as to enable the objective lens of the microscopic equipment to generate relative motion relative to the sample, and acquiring a plurality of microscopic sub-image information about the sample through the microscopic equipment in the relative motion process, wherein the relative motion comprises relative plane motion of the objective lens relative to a stage corresponding to the sample in the direction of the stage corresponding to the sample, and relative back and forth motion of the objective lens relative to the stage in the vertical direction of the stage corresponding to the plane, and the relative plane motion is at least partially overlapped with the relative back and forth motion in the relative motion process.

According to one aspect of the present application, there is provided a method for acquiring a microscopic image of a specimen, wherein the method comprises:

the user equipment sends image acquisition instruction information about a sample to corresponding microscopic equipment, wherein the image acquisition instruction information is used for controlling the microscopic equipment so as to enable an objective lens of the microscopic equipment to generate relative motion relative to the sample, and microscopic sub-image information about the sample is acquired through the microscopic equipment in the process of the relative motion, the relative motion comprises relative plane motion of the objective lens relative to an objective table in the direction of the corresponding plane of the objective table for accommodating the sample, and relative back and forth motion of the objective lens relative to the objective table in the vertical direction of the corresponding plane of the objective table, and the relative plane motion and the relative back and forth motion at least partially overlap in the process of the relative motion;

the microscopic equipment receives image acquisition instruction information about the sample, which is sent by the corresponding user equipment, controls the objective lens to keep relative movement relative to the objective table according to the image acquisition instruction information, and acquires a plurality of pieces of sub-image information about the sample through the microscopic equipment in the relative movement process.

According to one aspect of the present application, there is provided a user device for acquiring microscopic image information of a sample, the device comprising:

the device comprises a one-to-one module, a microscope device and a microscope device, wherein the one-to-one module is used for sending image acquisition instruction information about a sample to a corresponding microscope device, the image acquisition instruction information is used for controlling the microscope device so that an objective lens of the microscope device moves relatively to the sample, and microscopic sub-image information about the sample is acquired through the microscope device in the process of the relative movement, the relative movement comprises relative plane movement of the objective lens relative to an objective table in the direction of the corresponding plane of the objective table for accommodating the sample, and relative back-and-forth movement of the objective lens relative to the objective table in the vertical direction of the corresponding plane of the objective table, and the relative plane movement and the relative back-and-forth movement are at least partially overlapped in the process of the relative movement.

According to another aspect of the present application, there is provided a microscopy apparatus for acquiring microscopic image information of a sample, the apparatus comprising:

the second module is used for receiving image acquisition instruction information about the sample, which is sent by the corresponding user equipment;

And the second module is used for controlling the microscopic equipment according to the image acquisition instruction information so as to enable the objective lens of the microscopic equipment to generate relative motion relative to the sample, and acquiring a plurality of microscopic sub-image information about the sample through the microscopic equipment in the relative motion process, wherein the relative motion comprises relative plane motion of the objective lens relative to the objective table in the direction of the corresponding plane of the objective table for accommodating the sample, and relative reciprocating motion of the objective lens relative to the objective table in the vertical direction of the corresponding plane of the objective table, and the relative plane motion and the relative reciprocating motion are at least partially overlapped in the relative motion process.

According to one aspect of the present application, there is provided an apparatus for acquiring microscopic image information of a sample, wherein the apparatus comprises:

a processor; and

a memory arranged to store computer executable instructions that, when executed, cause the processor to perform the operations of any of the methods described above.

According to one aspect of the present application, there is provided a computer readable medium storing instructions that, when executed, cause a system to perform the operations of any of the methods described above.

Compared with the prior art, the image acquisition instruction information about the sample is sent to the corresponding microscopic equipment, wherein the image acquisition instruction information is used for controlling the microscopic equipment so as to enable the objective lens of the microscopic equipment to generate relative motion relative to the sample, microscopic sub-image information about the sample is acquired through the microscopic equipment in the process of the relative motion, the relative motion comprises relative plane motion of the objective lens relative to the objective table in the direction of a corresponding plane of the objective table for accommodating the sample, and relative back and forth motion of the objective lens relative to the objective table in the vertical direction of a corresponding plane of the objective table, and the relative plane motion is at least partially overlapped with the relative back and forth motion in the process of the relative motion; according to the method, the microscopic sub-image information about the sample can be acquired rapidly and accurately through relative movement, the acquisition efficiency is high, the acquired microscopic sub-image information is a continuous image with high overlapping degree, the precision of generating the whole microscopic image information of the sample is improved, and the use experience of a user is improved.

Drawings

Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the following drawings, in which:

FIG. 1 illustrates a flow chart of a method for acquiring microscopic image information of a specimen in accordance with one embodiment of the present application;

FIG. 2 illustrates a flow chart of a system method for acquiring microscopic image information of a specimen in accordance with one embodiment of the present application;

FIG. 3 illustrates a flow chart of a method for acquiring microscopic image information of a sample in accordance with one embodiment of the present application;

FIG. 4 illustrates an example of establishing a spatial coordinate system based on a center of a sample, according to one embodiment of the present application;

FIG. 5 illustrates an example of controlling relative movement of a microscopy apparatus according to one embodiment of the present application;

FIG. 6 illustrates an example of a graph of the speed of a relative round trip movement over time over a period, according to one embodiment of the present application;

fig. 7 shows step S102 and step S103 following step S101 in fig. 3 according to one embodiment of the present application;

FIG. 8 illustrates a flow chart of a method for acquiring microscopic image information of a specimen in accordance with one embodiment of the present application;

fig. 9 shows functional modules of a user equipment 100 according to one embodiment of the present application;

FIG. 10 illustrates functional blocks of a microscopy apparatus 200 according to one embodiment of the present application;

FIG. 11 illustrates an exemplary system that may be used to implement various embodiments described herein.

The same or similar reference numbers in the drawings refer to the same or similar parts.

Detailed Description

The present application is described in further detail below with reference to the accompanying drawings.

In one typical configuration of the present application, the terminal, the devices of the services network, and the trusted party each include one or more processors (e.g., central processing units (Central Processing Unit, CPU)), input/output interfaces, network interfaces, and memory.

The Memory may include non-volatile Memory in a computer readable medium, random access Memory (Random Access Memory, RAM) and/or non-volatile Memory, etc., such as Read Only Memory (ROM) or Flash Memory (Flash Memory). Memory is an example of computer-readable media.

Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase-Change Memory (PCM), programmable Random Access Memory (Programmable Random Access Memory, PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (Dynamic Random Access Memory, DRAM), other types of Random Access Memory (RAM), read-Only Memory (ROM), electrically erasable programmable read-Only Memory (EEPROM), flash Memory or other Memory technology, read-Only Memory (Compact Disc Read-Only Memory, CD-ROM), digital versatile disks (Digital Versatile Disc, DVD) or other optical storage, magnetic cassettes, magnetic tape storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by the computing device.

The device referred to in the present application includes, but is not limited to, a user device, a network device, or a device formed by integrating a user device and a network device through a network. The user equipment includes, but is not limited to, any mobile electronic product which can perform man-machine interaction with a user (for example, perform man-machine interaction through a touch pad), such as a smart phone, a tablet computer and the like, and the mobile electronic product can adopt any operating system, such as an Android operating system, an iOS operating system and the like. The network device includes an electronic device capable of automatically performing numerical calculation and information processing according to a preset or stored instruction, and the hardware includes, but is not limited to, a microprocessor, an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a programmable logic device (Programmable Logic Device, PLD), a field programmable gate array (Field Programmable Gate Array, FPGA), a digital signal processor (Digital Signal Processor, DSP), an embedded device, and the like. The network device includes, but is not limited to, a computer, a network host, a single network server, a plurality of network server sets, or a cloud of servers; here, the Cloud is composed of a large number of computers or network servers based on Cloud Computing (Cloud Computing), which is a kind of distributed Computing, a virtual supercomputer composed of a group of loosely coupled computer sets. Including but not limited to the internet, wide area networks, metropolitan area networks, local area networks, VPN networks, wireless Ad Hoc networks (Ad Hoc networks), and the like. Preferably, the device may be a program running on the user device, the network device, or a device formed by integrating the user device and the network device, the touch terminal, or the network device and the touch terminal through a network.

Of course, those skilled in the art will appreciate that the above-described devices are merely examples, and that other devices now known or hereafter may be present as appropriate for the application, are intended to be within the scope of the present application and are incorporated herein by reference.

In the description of the present application, the meaning of "a plurality" is two or more, unless explicitly defined otherwise.

Fig. 1 shows a typical scenario of the present application, where a user device 100 establishes a communication connection with a microscope device 200, and the user device 100 sends corresponding image acquisition instruction information to the microscope device 200, and the microscope device 200 acquires a plurality of microscopic sub-image information about a sample based on the image acquisition instruction information, where the image acquisition instruction information is used to control the microscope device so that an objective lens of the microscope device generates relative movement with respect to the sample, and the microscopic sub-image information about the sample is acquired by the microscope device during the relative movement, where the relative movement includes a relative planar movement of the objective lens with respect to a stage corresponding to the sample in a direction of the stage corresponding to the plane, and a relative reciprocal movement of the objective lens with respect to the stage in a vertical direction of the stage corresponding to the plane, where the relative planar movement is at least partially overlapped with the relative reciprocal movement. Here, the user device 100 includes, but is not limited to, any mobile electronic product that can perform man-machine interaction with a user (e.g., perform man-machine interaction through a touch pad), such as PC (personal computer), a tablet computer, a smart phone, etc.; the microscopic device includes, but is not limited to, an optical instrument consisting of a lens or a combination of several lenses for magnifying a tiny object into a human eye, including, but not limited to, an optical microscope, an electron microscope, a video microscope, etc. The microscopic equipment is usually combined with a camera system, user equipment and the like to achieve the aim of amplifying and observing a measured object, an image obtained under a microscope is projected onto a photosensitive photo through the principle of small hole imaging, so that microscopic sub-image information under the corresponding field of view of the current objective lens is obtained, or a camera is directly in butt joint with the microscope to shoot corresponding microscopic sub-image information.

In connection with the system topology shown in fig. 1, fig. 2 shows a system method for acquiring microscopic image information of a sample according to an aspect of the present application, applied to the user device 100 and the microscopic device 200 described in fig. 1, the method comprising:

in step S101, the user equipment sends image acquisition instruction information about a sample to a corresponding microscopic device, where the image acquisition instruction information is used to control the microscopic device so as to make an objective lens of the microscopic device generate relative motion with respect to the sample, and acquire microscopic sub-image information about the sample by the microscopic device during the relative motion, where the relative motion includes a relative planar motion of the objective lens with respect to a stage corresponding to the sample in a direction of the stage corresponding to the sample, and a relative reciprocal motion of the objective lens with respect to the stage in a direction perpendicular to the stage corresponding to the stage, where the relative planar motion at least partially overlaps with the relative reciprocal motion during the relative motion;

in step S201, the microscope device receives image acquisition instruction information about a sample sent by a corresponding user device;

In step S202, the microscope apparatus controls the objective lens to maintain the relative movement with respect to the stage according to the image acquisition instruction information, and acquires a plurality of sub-image information about the sample through the microscope apparatus during the relative movement.

With reference to the system topology shown in fig. 1 and the system method shown in fig. 2, from the perspective of interaction between the user equipment and the microscopic equipment, the present application further provides embodiments corresponding to each method for acquiring microscopic sub-image information of the sample, and the following description will be given by way of example with reference to fig. 3.

Fig. 3 shows a method for acquiring microscopic image information of a sample according to an aspect of the present application, applied to a user device 100, which is applicable to the system topology shown in fig. 1, comprising step S101. In step S101, a user device sends image acquisition instruction information about a sample to a corresponding microscope device, where the image acquisition instruction information is used to control the microscope device so as to make an objective lens of the microscope device generate relative motion with respect to the sample, and acquire microscopic sub-image information about the sample by the microscope device during the relative motion, where the relative motion includes a relative planar motion of the objective lens with respect to a stage corresponding to the sample in a direction of the stage corresponding to the sample, and a relative reciprocal motion of the objective lens with respect to the stage in a direction perpendicular to the stage corresponding to the stage, where the relative planar motion at least partially overlaps with the relative reciprocal motion during the relative motion. For example, the user device establishes a communication connection with the corresponding microscopic device, such as by a wired or wireless connection, and the user device transmits acquisition instruction information about the sample to the microscopic device via the communication connection. The microscope device comprises a control system, a lens barrel, a lens arm, a displacement sensor and a lens barrel, wherein the control system is used for controlling the movement of an objective lens and/or an objective table of the microscope device, for example, a lens barrel where the objective lens is positioned is connected with the lens arm through a moving device, the movement of the lens arm is controlled through the control of the moving device, so that the objective lens is moved, and the moving device is connected with the displacement sensor, wherein the displacement sensor comprises but is not limited to a screw rod, a grating ruler and the like; similarly, the objective table is connected with the lens seat through the moving device, the movement of the character stage is controlled through the control of the moving device, and the sample is fixed on the objective table. The relative movement of the objective lens with respect to the sample in the microscope device may be accomplished by controlling the movement of the objective lens alone or by controlling the movement of the stage alone, or by controlling the relative movement of the objective lens and the stage simultaneously. The microscopic equipment further comprises an image pick-up device, wherein the image pick-up device is used for collecting microscopic sub-image information about the sample, the resolution of each microscopic sub-image information is consistent, the size of each pixel is determined according to the magnification of the objective lens, and accordingly the size of each microscopic sub-image information is determined, and the actual size of each microscopic sub-image information corresponding to the range of the sample is taken as a unit range. The image acquisition instruction information is used for controlling the microscopic equipment, the microscopic equipment adjusts a corresponding control system according to the image acquisition instruction information, so that the objective lens of the microscopic equipment generates relative motion relative to the sample, a space coordinate system which is established by taking the center of the sample or the object table as an origin is shown in fig. 4, corresponding X axis and Y axis are respectively established in a plane where the object table is located, a corresponding Z axis is established in a direction perpendicular to the plane where the object table is located, the origin of the space coordinate system moves along with the movement of the center of the sample or the object table, the corresponding relative plane moves to the movement of the objective lens on a corresponding plane of X0Y in the coordinate system, the corresponding relative reciprocating motion is the reciprocating motion of the objective lens in the Z axis direction in the coordinate system, and the like; the relative plane movement is used for adjusting the area of the sample shot by the objective lens, so that the area of the current shot microscopic sub-image information corresponding to the sample is changed; the relative back and forth movement is used for adjusting the height of the objective lens relative to the sample so as to change the corresponding height of the focal plane of the current photographed microscopic sub-image information in the sample. Fig. 5 shows an example of relative movement, as shown, by controlling the up and down movement of the lens barrel to achieve relative back and forth movement, by controlling the movement of the stage in the X-axis direction and the Y-axis direction to achieve relative planar movement, etc., although it will be understood by those skilled in the art that the above-described relative movement is merely exemplary, and that other relative movement patterns that may occur in the present application are intended to be encompassed within the scope of the present application and are incorporated herein by reference.

The relative planar motion at least partially overlaps the relative reciprocating motion during the relative motion. For example, the relative planar movement and the relative reciprocating movement are performed simultaneously during the relative movement, for example, the lens barrel is controlled to make the objective lens reciprocate in the Z-axis direction while controlling the stage to move in the planar direction, so that the focal plane of the photographed microscopic sub-image information also reciprocates in the height of the specimen while the region of the microscopic sub-image information varies in the whole range of the specimen. When the relative planar motion and the relative reciprocating motion are not overlapped in the relative motion process, the relative planar motion and the relative reciprocating motion can be only corresponding relative planar motion or only corresponding relative reciprocating motion; for example, in some cases, the sample is collected by relative planar motion only in the peripheral region of the sample based on the user's needs, by overlapping relative planar motion with relative back and forth motion in the central region of the sample, by a user circling (e.g., by viewing a microscopic image under a low magnification objective, marking a region of interest in the microscopic image, etc.), and so forth. According to the scheme, the relative plane motion and the relative reciprocating motion are combined, so that efficiency and precision are considered under different demands of users, balance of acquisition efficiency and acquisition precision is realized, more optimized acquisition schemes are provided for the users, and the use experience of the users is improved.

In some embodiments, when the relative planar motion overlaps the relative reciprocal motion and there is the relative reciprocal motion of the objective lens and the stage, there is a displacement variation of the objective lens relative to the stage in the direction of the stage's corresponding planar motion. For example, the overlapping of the relative planar motion and the relative reciprocating motion means that, while the region of the photographed microscopic sub-image information varies over the entire range of the sample, the focal plane of the microscopic sub-image information also varies reciprocally in height in the sample, etc., such as in the temporal neighborhood of each overlapping time in partial overlapping, the objective lens and the stage follow corresponding relative reciprocating motions in the Z-axis direction, and there is a displacement variation of the objective lens relative to the stage in the planar direction of the stage (i.e., XOY planar direction), i.e., there is a displacement variation of the objective lens in the planar direction in the temporal neighborhood of each overlapping time of the overlapping time, and the speed of the corresponding relative planar motion is not zero. Through the requirement to the speed on the plane direction for the collection mode of this overlap portion can reach the collection effect of quick scanning, has promoted the efficiency of collection.

In some embodiments, the relative planar motion overlaps the relative back and forth motion, and the microscopy apparatus is controlled to acquire microscopic sub-image information about the specimen at a first frequency. For example, the image acquisition instruction information further includes a first frequency for instructing the microscopy apparatus to control the corresponding imaging device to acquire microscopic sub-image information about the specimen at the first frequency when the relative planar motion overlaps with the relative reciprocating motion. In some embodiments, the microscopic sub-image information satisfies: the overlapping portion of adjacent images is greater than a first threshold and less than a second threshold. For example, since the movement speed in the plane direction in the overlapping portion is not zero, the overlapping degree between adjacent images in the acquired microscopic sub-image information is smaller than a certain upper limit threshold, and the upper limit threshold is generally 100%, that is, the images are completely overlapped, where the second threshold may be smaller than the upper limit threshold, for example, 95%, so that the speed in the plane direction is not too low, and thus the acquisition efficiency is affected; in addition, considering the requirements on the image overlapping degree in image matching and the like, the overlapping degree between adjacent images should be larger than a first threshold value, such as 65%, so that the image acquisition efficiency is ensured, and meanwhile, the acquired images can be ensured to meet the requirements on the precision of the composite microscopic image information and the like.

In some embodiments, the image acquisition instruction information does not include the acquisition frequency of other areas, so that the microscopy equipment does not acquire related microscopy sub-image information in other areas except the overlapping area, or acquires the microscopy sub-image information of the areas according to a default frequency, and so on. In other embodiments, the image acquisition instruction information further includes an acquisition frequency of other areas, such as a second frequency, and the like, for instructing the microscopic device to control the corresponding camera to acquire microscopic sub-image information about the sample at the second acquisition frequency when the relative planar motion and the relative reciprocating motion are not overlapped; the microscopy apparatus is controlled to acquire microscopic sub-image information about the specimen at a second frequency, wherein the second frequency is less than the first frequency, if the relative planar motion and the relative back-and-forth motion do not overlap. For example, the microscopy apparatus is controlled to acquire microscopy sub-image information of other areas than the overlapping area at a second frequency, which is lower than the first frequency, as the other areas may not have the overlapping of the relative back and forth movement and the relative planar movement, and the accuracy requirements for the microscopy image information of the other areas are relatively lower. The data quantity of the collected sample is controlled by controlling the collection frequency, and the data quantity of the collected image can be controlled to a certain extent under the requirements of controlling the collection efficiency and the image precision.

In some embodiments, the image acquisition instruction information further includes a first start position and a first end position of the relative planar motion, the relative planar motion including movement of the objective lens relative to the stage in a direction of the stage corresponding plane from the first start position to the second end position. For example, in the above-established spatial coordinate system, the acquired range of the sample may be determined in the planar direction of the stage, for example, the image acquisition instruction information includes a first start position and a first end position of the relative planar movement, and the objective lens of the microscope device is controlled to move from the first start position to the first end position in the planar direction relative to the stage by the image acquisition instruction information, and the movement mode is usually linear movement, and of course, a corresponding acquisition route may be set according to the user requirement, such as a fold line type or a curve type; the movement speed of the corresponding relative planar movement can be generally regarded as a uniform speed, such as initial acceleration, uniform speed after acceleration to a certain speed, and the like, until the first end position, or the corresponding speed can be set according to the user requirement, such as the speed changing with time or the speed changing with the movement position, and the like.

In some embodiments, the image acquisition instruction information further includes a relative planar motion speed during the relative planar motion, a relative reciprocating motion speed within one period of the relative reciprocating motion. For example, the acquired range of the sample may be controlled by setting the acquisition speed of the microscope device, for example, the image acquisition instruction information includes a relative movement speed during the relative plane movement and a movement speed of the relative back and forth movement in one period of the relative back and forth movement, and the relative plane speed of the relative plane movement includes a speed value of the corresponding movement speed, where the speed value may be a numerical value (such as a constant speed value that keeps constant speed movement throughout the whole course, etc.), or a speed value that varies with time, etc.; the relative reciprocating speed of the relative reciprocating motion includes a speed value of the objective lens moving in the vertical plane direction relative to the objective table in a reciprocating period, the speed value being a value periodically changing with time in a period, such as a speed change curve like a sine curve or acceleration/deceleration. In some embodiments, the relative reciprocating speed in one period of the relative reciprocating motion includes an acceleration process, a uniform speed process, a deceleration process, and the like. For example, fig. 6 shows an example of a curve of relative round-trip speed periodically varying with time over a period of relative round-trip movement, such as in an acceleration phase, acceleration to speed V0 moving at constant speed over time t1-t2, followed by deceleration movement over time t2-t3 until the speed is zero, followed by reverse acceleration to speed V0 over time t3-t4, i.e., reverse acceleration to speed value V0, and maintaining constant speed movement over time t4-t5, followed by deceleration over time t5-t6 until the speed is zero, during which time periods the objective lens moves relative to the stage from one end point (e.g., highest point, etc.) to the opposite other end point (e.g., lowest point, etc.) over time period 0-t3, followed by reverse movement from the other end point (e.g., lowest point, etc.) to the end point that just started (e.g., highest point, etc.) over time period t3-t6, thereby effecting round-trip movement. By the mode, simple and convenient speed control can be realized, so that reciprocating motion is realized, the focal plane of the acquired microscopic sub-image information can be ensured to be uniform in the height change of the sample, and better microscopic image information and the like can be obtained. Of course, the speed of the reciprocating motion in the application can also be changed at will, for example, the corresponding speed is adjusted to accelerate, decelerate or uniform according to the requirement of the user.

In some embodiments, when the relative planar motion overlaps the relative reciprocating motion, the objective lens performs a relative planar motion with respect to the stage at a first relative planar motion speed in a planar direction corresponding to the stage. For example, the precision requirement of the peripheral area of the sample is not high, and the important area usually enables the relative plane motion to overlap with the relative reciprocating motion so as to improve the density of image acquisition and ensure the acquisition efficiency, for example, the image acquisition instruction information contains a first relative plane motion speed, and when the relative plane motion overlaps with the relative reciprocating motion, the microscopic equipment performs relative plane motion in the plane direction through the first relative plane motion speed. In some embodiments, when the relative planar motion and the relative back-and-forth motion do not overlap, the objective lens performs a relative planar motion with respect to the stage in a planar direction corresponding to the stage at a second relative planar motion speed, wherein the second relative planar motion speed is greater than the first relative planar motion speed. For example, the microscopy apparatus is controlled to acquire other areas than the overlapping area, etc. at a second relative planar velocity where the microscopy sub-image density requirements are low, so that the corresponding second relative planar velocity is greater than the first relative planar velocity, e.g. when the relative planar motion is non-overlapping with the relative back-and-forth motion and the relative planar motion is performed at the second relative planar motion velocity, the objective lens is moved only in the planar direction with respect to the stage, acquiring microscopy sub-image information about the sample at the same focal plane height.

In some embodiments, the time of one cycle of the relative back and forth movement corresponds to the first relative planar movement speed. For example, a period of time of one cycle of the relative reciprocal movement as shown in fig. 6 is 0 to t6, which corresponds to the first relative planar movement, the first relative planar movement speed moves by one unit area in the planar direction as the relative reciprocal movement overlaps with the relative planar movement, the relative reciprocal movement moves by just one cycle, or the like; or the first relative plane movement speed moves by one unit area in the plane direction, and the time of the relative back and forth movement is just equal to the time of half a cycle, etc. In some embodiments, the first relative planar movement speed of the relative planar movement is greater than or equal to the length of the unit area corresponding to the objective lens in the half cycle time of the relative back and forth movement, wherein the relative movement path includes a path along which the objective lens performs relative planar movement with respect to the objective table in the planar direction corresponding to the objective table. For example, if the objective lens moves from one end point to the other end point in the vertical direction relative to the stage within a half period of the relative reciprocating movement, and the focal plane corresponding to microscopic sub-image information passes through all the collection heights set by the sample, the length of the objective lens moving in the plane direction relative to the stage by one unit area is greater than or equal to the length of the unit area in the movement direction, and the unit area is determined according to the magnification of the objective lens and the resolution of the microscopic sub-image information. By the method, the collection rate is guaranteed under the condition that the collected microscopic sub-images are comprehensive.

In some embodiments, as shown in fig. 7, the method further comprises step S102 and step S103. In step S102, the user device receives a plurality of microscopic sub-image information about the specimen returned by the microscopic device; in step S103, the user device generates microscopic image information about the specimen from the plurality of microscopic sub-image information. For example, after the user device sends the image acquisition instruction information to the corresponding microscopic device through communication connection, the microscopic device receives the image acquisition instruction information, controls the corresponding objective lens and the objective table to generate relative motion based on the image acquisition instruction information, and acquires microscopic sub-image information about the sample in the process of performing the relative motion, then the microscopic device returns the acquired multiple pieces of microscopic sub-image information to the user device, and the user device receives the multiple pieces of microscopic sub-image information, generates microscopic image information about the sample according to the microscopic sub-image information, for example, generates microscopic image information about the sample according to image matching or coordinate calculation. In some embodiments, in step S103, the user device determines spatial coordinate information of each pixel point in the sample according to the plurality of microscopic sub-image information, and generates microscopic image information of the sample using the spatial coordinate information of each pixel point. For example, based on the above-mentioned established spatial coordinate system, each piece of microscopic image information may determine the coordinate of the center of the objective lens during shooting, based on the coordinate of the center of the objective lens and the distance between the center of the objective lens and the corresponding piece of microscopic sub-image information, the coordinate information of the image center corresponding to the microscopic sub-image information may be determined, so as to obtain the spatial coordinate information of each pixel point in the microscopic sub-image information, and then, image fusion or the like is performed according to the spatial coordinate information of each pixel point in the microscopic sub-image information, for example, the clearer pixel point in each piece of microscopic sub-image information is taken, and the microscopic image information about the sample is generated by combining the spatial coordinate information of the pixel points, where the microscopic image information refers to the microscopic image about the whole range of the sample generated by fusing the microscopic sub-image information of each unit area. In other embodiments, in step S103, the user device performs image matching on the plurality of microscopic sub-image information, and generates microscopic image information about the sample according to the image matching result. For example, image matching refers to a method of searching for similar image targets by analyzing correspondence of image content, features, structures, relationships, textures, gray scales, and the like, similarity and consistency; based on the overlapping part of adjacent images in the microscopic sub-image information, the spatial relationship between the microscopic sub-images can be determined through image matching, and image fusion is carried out based on the spatial relationship, so that microscopic image information about the sample is generated. In other embodiments, in step S103, the user device determines spatial coordinate information of an image center of each of the plurality of microscopic sub-image information according to the plurality of microscopic sub-image information; and determining adjacent microscopic sub-image information corresponding to each microscopic sub-image information according to the space coordinate information of the image center of each microscopic sub-image information, performing image matching based on the adjacent microscopic sub-image information of the microscopic sub-image information, and generating microscopic image information about the sample according to an image matching result. For example, the spatial coordinate information of each pixel point calculated by coordinates is limited by the moving device of the microscope equipment and the accuracy of calculation, and the accuracy of image fusion of each microscopic sub-image is also limited by the moving device of the microscope equipment and the accuracy of calculation; when image fusion is carried out through image matching, the similarity of characteristic points between images with far distances is high, a matching result with large errors easily occurs, and good microscopic image information can be obtained by combining two modes, such as determining spatial coordinate information of an image center of current microscopic sub-image information through coordinate calculation, and determining the microscopic sub-image information as neighboring microscopic sub-image information of the microscopic sub-image information if the image center of the microscopic sub-image information exists in the neighborhood (such as within 100 microns) based on the microscopic sub-image information in the image neighborhood, and then carrying out image matching on the neighboring microscopic sub-image information and the microscopic sub-image information to generate corresponding microscopic image information. The method overcomes the defect that larger errors occur in image matching, and simultaneously enables the acquired microscopic image information with good precision requirements.

In some embodiments, the microscopic image information includes, but is not limited to: two-dimensional microscopic image information, three-dimensional microscopic image information, two-dimensional three-dimensional comprehensive image information, and the like. For example, the two-dimensional microscopic image information includes microscopic images in which the pixel points are arranged in image coordinates; for example, the clearer pixel points in each microscopic sub-image information are extracted, and depth of field fusion is performed based on the pixel points, so that corresponding two-dimensional microscopic image information is obtained. For example, the three-dimensional microscopic image information comprises three-dimensional images and the like which are distinct in level and bright in color in visual sense and have strong visual impact force with respect to a stereoscopic image of the target object, the three-dimensional images comprise three-dimensional coordinate information of the whole outline of the target object, a corresponding three-dimensional image coordinate system is built in the three-dimensional microscopic image information, corresponding three-dimensional coordinates and the like exist in the three-dimensional image coordinate system at each point on the outline corresponding to the target object, and the three-dimensional microscopic image information can present three-dimensional outlines and the like of different angles of the target object through specific plug-ins or application and the like; for example, clearer pixel points in each microscopic sub-image are extracted, corresponding spatial relations are established according to spatial coordinate information of the pixel points, and corresponding three-dimensional microscopic image information is generated through image fusion, wherein the three-dimensional microscopic image information can be generated only based on microscopic sub-image information at different heights of the overlapped part. For another example, the two-dimensional three-dimensional integrated image information includes that a partial area related to the target image is presented in a three-dimensional image form, the partial area is presented in a two-dimensional image form, and a reference plane corresponding to the presentation of the two-dimensional image is flush with a plane with a zero height in the three-dimensional image, and of course, the reference plane of the two-dimensional image can be aligned based on a plane corresponding to a certain height in the three-dimensional image set by a user, and the like. For example, the overlapped part has microscopic sub-image information under different focal plane heights, corresponding three-dimensional microscopic image information can be generated, the microscopic sub-image information of the non-overlapped part can generate corresponding two-dimensional microscopic image information, the boundaries of the three-dimensional microscopic image information and the two-dimensional microscopic image information are matched based on image matching, image fusion is performed based on a zero reference plane of the three-dimensional microscopic image information, and the like.

In some embodiments, the method further includes step S104 (not shown), in step S104, the user equipment performs feature matching on the latest microscopic sub-image information in the plurality of microscopic sub-image information, and if matching fails, corresponding image acquisition stop instruction information is generated; and sending the image acquisition stop instruction information to the microscopic equipment. For example, after receiving the microscopic sub-image information collected by the microscopic device, the user device may perform matching based on the microscopic sub-image information, for example, matching is performed according to the latest received microscopic sub-image information, whether the collection area matching the microscopic sub-image information leaves the collection area of the sample, if matching fails, it indicates that the collection area has already considered the collection area of the sample, and the user device generates corresponding image collection stop instruction information, and sends the collection stop instruction information to the corresponding microscopic device, so as to control the microscopic device to stop collecting the microscopic sub-image information about the sample. In some embodiments, the feature matching includes, but is not limited to: performing feature matching on the latest microscopic sub-image information and adjacent microscopic sub-image information of the latest microscopic sub-image information; and carrying out feature matching on the latest microscopic sub-image information and template feature information of the sample. For example, the user equipment determines adjacent microscopic sub-image information (such as microscopic sub-image information within 100 micrometers) of the current latest microscopic sub-image information based on the spatial coordinate relation of the microscopic sub-image information, and performs image matching in the several microscopic sub-images, if the microscopic sub-image information matched with the latest microscopic sub-image information is not matched, determines that the matching fails, and generates corresponding image acquisition stop instruction information. For another example, the user equipment end stores template characteristic information of the sample, and the latest microscopic sub-image information is matched with the template characteristic information, if the latest microscopic sub-image information is not matched with the template characteristic information, the current acquisition area is indicated to exceed the acquisition area of the sample, and corresponding image acquisition stopping instruction information is generated.

The foregoing illustrates various embodiments of methods that can be implemented from the perspective of a user device, and further, embodiments of methods that can be implemented from the perspective of a microscopy device are provided, as exemplified below in connection with fig. 8.

Fig. 8 illustrates a method of acquiring microscopic image information of a specimen according to an aspect of the present application, applicable to a microscope apparatus 200, capable of adapting to the system topology illustrated in fig. 1, including step S201 and step S202. In step S201, the microscopy device receives image acquisition instruction information about the sample sent by the corresponding user device; in step S202, the microscopy apparatus controls the microscopy apparatus according to the image acquisition instruction information, so that a relative motion of an objective lens of the microscopy apparatus with respect to the sample is generated, and a plurality of microscopic sub-image information about the sample is acquired by the microscopy apparatus during the relative motion, wherein the relative motion includes a relative planar motion of the objective lens with respect to a stage corresponding to the sample in a direction of the stage corresponding to the sample, and a relative reciprocal motion of the objective lens with respect to the stage in a vertical direction of the stage corresponding to the stage, and the relative planar motion at least partially overlaps with the relative reciprocal motion during the relative motion. For example, the user device establishes a communication connection with the corresponding microscopic device, such as by a wired or wireless connection, and the user device transmits acquisition instruction information about the sample to the microscopic device via the communication connection. The microscope device comprises a control system, a lens barrel, a lens arm, a displacement sensor and a lens barrel, wherein the control system is used for controlling the movement of an objective lens and/or an objective table of the microscope device, for example, a lens barrel where the objective lens is positioned is connected with the lens arm through a moving device, the movement of the lens arm is controlled through the control of the moving device, so that the objective lens is moved, and the moving device is connected with the displacement sensor, wherein the displacement sensor comprises but is not limited to a screw rod, a grating ruler and the like; similarly, the objective table is connected with the lens seat through the moving device, the movement of the character stage is controlled through the control of the moving device, and the sample is fixed on the objective table. The relative movement of the objective lens with respect to the sample in the microscope device may be accomplished by controlling the movement of the objective lens alone or by controlling the movement of the stage alone, or by controlling the relative movement of the objective lens and the stage simultaneously. The microscopic equipment further comprises an image pick-up device, wherein the image pick-up device is used for collecting microscopic sub-image information about the sample, the resolution of each microscopic sub-image information is consistent, the size of each pixel is determined according to the magnification of the objective lens, and accordingly the size of each microscopic sub-image information is determined, and the actual size of each microscopic sub-image information corresponding to the range of the sample is taken as a unit range. The image acquisition instruction information is used for controlling the microscopic equipment, the microscopic equipment adjusts a corresponding control system according to the image acquisition instruction information, so that the objective lens of the microscopic equipment generates relative motion relative to the sample, a space coordinate system which is established by taking the center of the sample or the object table as an origin is shown in fig. 3, corresponding X-axis and Y-axis are respectively established in a plane where the object table is located, a corresponding Z-axis is established in a direction perpendicular to the plane where the object table is located, the origin of the space coordinate system moves along with the movement of the center of the sample or the object table, the corresponding relative plane moves to the movement of the objective lens on a corresponding plane of X0Y in the coordinate system, the corresponding relative reciprocating motion is the reciprocating motion of the objective lens in the Z-axis direction in the coordinate system, and the like; the relative plane movement is used for adjusting the area of the sample shot by the objective lens, so that the area of the current shot microscopic sub-image information corresponding to the sample is changed; the relative back and forth movement is used for adjusting the height of the objective lens relative to the sample so as to change the corresponding height of the focal plane of the current photographed microscopic sub-image information in the sample. Fig. 4 shows an example of relative movement, as shown, by controlling the up and down movement of the lens barrel to achieve relative back and forth movement, by controlling the movement of the stage in the X-axis direction and the Y-axis direction to achieve relative planar movement, etc., although it will be understood by those skilled in the art that the above-described relative movement is merely exemplary, and that other relative movement patterns that may occur in the present application are intended to be encompassed within the scope of the present application and are incorporated herein by reference.

In some embodiments, the method further comprises step S203 (not shown), in step S203 the microscopy device returns the plurality of microscopy sub-image information to the user device. For example, based on the above-mentioned established spatial coordinate system, each piece of microscopic image information may determine the coordinate of the center of the objective lens during shooting, based on the coordinate of the center of the objective lens and the distance between the center of the objective lens and the corresponding piece of microscopic sub-image information, the coordinate information of the image center corresponding to the microscopic sub-image information may be determined, so as to obtain the spatial coordinate information of each pixel point in the microscopic sub-image information, and then, image fusion or the like is performed according to the spatial coordinate information of each pixel point in the microscopic sub-image information, for example, the clearer pixel point in each piece of microscopic sub-image information is taken, and the microscopic image information about the sample is generated by combining the spatial coordinate information of the pixel points, where the microscopic image information refers to the microscopic image about the whole range of the sample generated by fusing the microscopic sub-image information of each unit area.

In some embodiments, the method further includes step S204 (not shown), in which the microscopy device receives the image acquisition stop instruction information sent by the user device in step S204; and controlling the microscopic equipment to pause the current acquisition process based on the image acquisition stop instruction information. For example, after receiving the microscopic sub-image information collected by the microscopic device, the user device may perform matching based on the microscopic sub-image information, for example, matching is performed according to the latest received microscopic sub-image information, whether the collection area matching the microscopic sub-image information leaves the collection area of the sample, if matching fails, it indicates that the collection area has already considered the collection area of the sample, and the user device generates corresponding image collection stop instruction information, and sends the collection stop instruction information to the corresponding microscopic device, so as to control the microscopic device to stop collecting the microscopic sub-image information about the sample. In some embodiments, the feature matching includes, but is not limited to: performing feature matching on the latest microscopic sub-image information and adjacent microscopic sub-image information of the latest microscopic sub-image information; and carrying out feature matching on the latest microscopic sub-image information and template feature information of the sample. For example, the user equipment determines adjacent microscopic sub-image information (such as microscopic sub-image information within 100 micrometers) of the current latest microscopic sub-image information based on the spatial coordinate relation of the microscopic sub-image information, and performs image matching in the several microscopic sub-images, if the microscopic sub-image information matched with the latest microscopic sub-image information is not matched, determines that the matching fails, and generates corresponding image acquisition stop instruction information. For another example, the user equipment end stores template characteristic information of the sample, and the latest microscopic sub-image information is matched with the template characteristic information, if the latest microscopic sub-image information is not matched with the template characteristic information, the current acquisition area is indicated to exceed the acquisition area of the sample, and corresponding image acquisition stopping instruction information is generated.

With reference to the system topology shown in fig. 1 and the embodiments corresponding to the methods described above, the present application further provides corresponding devices capable of implementing the embodiments described above, and we will describe, by way of example, the user device 100 in connection with fig. 9.

Fig. 9 shows a user device 100 for acquiring microscopic image information of a sample, according to one aspect of the present application, which is adapted for the system topology shown in fig. 1, comprising a one-to-one module 101. And a one-to-one module 101, configured to send image acquisition instruction information about a sample to a corresponding microscopic device, where the image acquisition instruction information is configured to control the microscopic device so that an objective lens of the microscopic device generates relative movement with respect to the sample, and acquire microscopic sub-image information about the sample by the microscopic device during the relative movement, where the relative movement includes a relative planar movement of the objective lens with respect to a stage corresponding to the sample in a direction of the stage corresponding to the sample, and a relative reciprocal movement of the objective lens with respect to the stage in a direction perpendicular to the stage corresponding to the stage, where the relative planar movement at least partially overlaps with the relative reciprocal movement during the relative movement. Here, the implementation manner of the one-to-one module 101 is the same as or similar to the embodiment corresponding to step S101 in fig. 3, which is not described in detail and is incorporated herein by reference.

In some embodiments, when the relative planar motion overlaps the relative reciprocal motion and there is the relative reciprocal motion of the objective lens and the stage, there is a displacement variation of the objective lens relative to the stage in the direction of the stage's corresponding planar motion. The embodiment of the determination of overlapping of the relative planar motion and the relative reciprocal motion is the same as or similar to the embodiment of the determination of overlapping of the relative planar motion and the relative reciprocal motion described above, and is not described in detail and is incorporated herein by reference.

In some embodiments, the relative planar motion overlaps the relative back and forth motion, and the microscopy apparatus is controlled to acquire microscopic sub-image information about the specimen at a first frequency. In some embodiments, the microscopic sub-image information satisfies: the overlapping portion of adjacent images is greater than a first threshold and less than a second threshold.

In other embodiments, the microscopy apparatus is controlled to acquire microscopic sub-image information about the specimen at a second frequency when the relative planar motion and the relative back-and-forth motion do not overlap, wherein the second frequency is less than the first frequency. Here, the embodiments of the first frequency and the second frequency related to each other are the same as or similar to the foregoing embodiments of the first frequency and the second frequency related to each other, which are not described in detail and are incorporated herein by reference.

In some embodiments, the image acquisition instruction information further includes a first start position and a first end position of the relative planar motion, the relative planar motion including movement of the objective lens relative to the stage in a direction of the stage corresponding plane from the first start position to the second end position. Here, the embodiment of the image capturing instruction information further including the first start position and the first end position of the relative plane movement is the same as or similar to the embodiment of the image capturing instruction information further including the first start position and the first end position of the relative plane movement, which are corresponding to each other, and are not described in detail herein, and are incorporated by reference.

In some embodiments, the image acquisition instruction information further includes a relative planar motion speed during the relative planar motion, a relative reciprocating motion speed within one period of the relative reciprocating motion. In some embodiments, the relative reciprocating speed in one period of the relative reciprocating motion includes an acceleration process, a uniform speed process, a deceleration process, and the like. In some embodiments, when the relative planar motion overlaps the relative reciprocating motion, the objective lens performs a relative planar motion with respect to the stage at a first relative planar motion speed in a planar direction corresponding to the stage. In some embodiments, when the relative planar motion and the relative back-and-forth motion do not overlap, the objective lens performs a relative planar motion with respect to the stage in a planar direction corresponding to the stage at a second relative planar motion speed, wherein the second relative planar motion speed is greater than the first relative planar motion speed. In some embodiments, the time of one cycle of the relative back and forth movement corresponds to the first relative planar movement speed. In some embodiments, the first relative planar movement speed of the relative planar movement has a relative movement path equal to a length of a unit area corresponding to the objective lens within a half cycle time of the relative reciprocal movement, wherein the relative movement path includes a path along which the objective lens performs relative planar movement with respect to the objective table in a planar direction corresponding to the objective table. Here, the embodiments related to the relative planar motion speed, the relative reciprocating motion speed and the period are the same as or similar to the embodiments corresponding to the relative planar motion speed, the relative reciprocating motion speed and the period, and are not described in detail, and are incorporated herein by reference.

In some embodiments, the apparatus further comprises a two module 102 (not shown) and a three module 103 (not shown). A second module 102 for receiving a plurality of microscopic sub-image information about the specimen returned by the microscopic apparatus; and a three-module 103 for generating microscopic image information about the specimen based on the plurality of microscopic sub-image information. In some embodiments, in step S103, the user device determines spatial coordinate information of each pixel point in the sample according to the plurality of microscopic sub-image information, and generates microscopic image information of the sample using the spatial coordinate information of each pixel point. In other embodiments, in step S103, the user device performs image matching on the plurality of microscopic sub-image information, and generates microscopic image information about the sample according to the image matching result. In other embodiments, in step S103, the user device determines neighboring microscopic sub-image information corresponding to each microscopic sub-image information according to the plurality of microscopic sub-image information, performs image matching based on the neighboring microscopic sub-image information of the microscopic sub-image information, and generates microscopic image information about the sample according to the image matching result. Here, the embodiments of the two modules 102 and the three modules 103 are the same as or similar to the embodiments corresponding to the steps S102 and S103, and are not described in detail herein, and are incorporated by reference.

In some embodiments, the microscopic image information includes, but is not limited to: two-dimensional microscopic image information, three-dimensional microscopic image information, two-dimensional three-dimensional comprehensive image information, and the like. Here, the microscopic image information includes, but is not limited to: embodiments of the two-dimensional microscopic image information, the three-dimensional microscopic image information, and the two-dimensional three-dimensional integrated image information include, but are not limited to: the embodiments corresponding to the two-dimensional microscopic image information, the three-dimensional microscopic image information and the two-dimensional and three-dimensional comprehensive image information are the same or similar, and are not described in detail and are contained herein in a quoted manner.

In some embodiments, the apparatus further includes a fourth module 104 (not shown) configured to perform feature matching on the latest microscopic sub-image information in the plurality of microscopic sub-image information, and if matching fails, generate corresponding image acquisition stop instruction information; and sending the image acquisition stop instruction information to the microscopic equipment. The implementation of the four modules 104 is the same as or similar to the embodiment corresponding to the step S104, and is not described in detail herein, but is incorporated by reference.

In addition, the present application further provides a microscope apparatus capable of implementing embodiments of the method of the present application, as exemplified below in connection with fig. 10.

Fig. 10 illustrates a microscopy apparatus 200 for acquiring microscopic image information of a sample according to one aspect of the present application, which can be adapted to the system topology illustrated in fig. 1, includes a two-in-one module 201 and a two-in-two module 202. A second module 201, configured to receive image acquisition instruction information about a sample sent by a corresponding user equipment; and the second module 202 is configured to control the microscope device according to the image acquisition instruction information, so that a relative motion of an objective lens of the microscope device with respect to the sample is generated, and a plurality of microscopic sub-image information about the sample is acquired by the microscope device during the relative motion, where the relative motion includes a relative planar motion of the objective lens with respect to a stage holding the sample in a direction of a stage corresponding plane, and a relative reciprocal motion of the objective lens with respect to the stage in a direction perpendicular to the stage corresponding plane, and the relative planar motion and the relative reciprocal motion at least partially overlap during the relative motion. The embodiments of the second module 201 and the second module 202 are the same as or similar to the embodiments corresponding to the step S201 and the step S202 in fig. 8, and are not described in detail herein, and are incorporated herein by reference.

In some embodiments, the apparatus further comprises a bi-tri module 203 (not shown) for returning the plurality of microscopic sub-image information to the user device. Here, the implementation of the bi-tri module 203 is the same as or similar to the embodiment corresponding to the step S203, and is not described in detail herein, and is incorporated by reference.

In some embodiments, the apparatus further includes a second and fourth modules 204 (not shown) configured to receive image acquisition stop instruction information sent by the user equipment; and controlling the microscopic equipment to pause the current acquisition process based on the image acquisition stop instruction information. The implementation of the two-four module 204 is the same as or similar to the embodiment corresponding to the step S204, and is not described in detail herein, but is incorporated by reference.

In addition to the methods and apparatus described in the above embodiments, the present application also provides a computer-readable storage medium storing computer code which, when executed, performs a method as described in any one of the preceding claims.

The present application also provides a computer program product which, when executed by a computer device, performs a method as claimed in any preceding claim.

The present application also provides a computer device comprising:

one or more processors;

a memory for storing one or more computer programs;

the one or more computer programs, when executed by the one or more processors, cause the one or more processors to implement the method of any preceding claim.

FIG. 11 illustrates an exemplary system that may be used to implement various embodiments described herein;

in some embodiments, as shown in fig. 11, the system 300 can function as any of the above-described devices of the various described embodiments. In some embodiments, system 300 can include one or more computer-readable media (e.g., system memory or NVM/storage 320) having instructions and one or more processors (e.g., processor(s) 305) coupled with the one or more computer-readable media and configured to execute the instructions to implement the modules to perform the actions described herein.

For one embodiment, the system control module 310 may include any suitable interface controller to provide any suitable interface to at least one of the processor(s) 305 and/or any suitable device or component in communication with the system control module 310.

The system control module 310 may include a memory controller module 330 to provide an interface to the system memory 315. Memory controller module 330 may be a hardware module, a software module, and/or a firmware module.

The system memory 315 may be used, for example, to load and store data and/or instructions for the system 300. For one embodiment, system memory 315 may include any suitable volatile memory, such as, for example, a suitable DRAM. In some embodiments, the system memory 315 may comprise a double data rate type four synchronous dynamic random access memory (DDR 4 SDRAM).

For one embodiment, system control module 310 may include one or more input/output (I/O) controllers to provide an interface to NVM/storage 320 and communication interface(s) 325.

For example, NVM/storage 320 may be used to store data and/or instructions. NVM/storage 320 may include any suitable nonvolatile memory (e.g., flash memory) and/or may include any suitable nonvolatile storage device(s) (e.g., one or more Hard Disk Drives (HDDs), one or more Compact Disc (CD) drives, and/or one or more Digital Versatile Disc (DVD) drives).

NVM/storage 320 may include storage resources that are physically part of the device on which system 300 is installed or which may be accessed by the device without being part of the device. For example, NVM/storage 320 may be accessed over a network via communication interface(s) 325.

Communication interface(s) 325 may provide an interface for system 300 to communicate over one or more networks and/or with any other suitable device. The system 300 may wirelessly communicate with one or more components of a wireless network in accordance with any of one or more wireless network standards and/or protocols.

For one embodiment, at least one of the processor(s) 305 may be packaged together with logic of one or more controllers (e.g., memory controller module 330) of the system control module 310. For one embodiment, at least one of the processor(s) 305 may be packaged together with logic of one or more controllers of the system control module 310 to form a System In Package (SiP). For one embodiment, at least one of the processor(s) 305 may be integrated on the same die as logic of one or more controllers of the system control module 310. For one embodiment, at least one of the processor(s) 305 may be integrated on the same die with logic of one or more controllers of the system control module 310 to form a system on chip (SoC).

In various embodiments, the system 300 may be, but is not limited to being: a server, workstation, desktop computing device, or mobile computing device (e.g., laptop computing device, handheld computing device, tablet, netbook, etc.). In various embodiments, system 300 may have more or fewer components and/or different architectures. For example, in some embodiments, system 300 includes one or more cameras, keyboards, liquid Crystal Display (LCD) screens (including touch screen displays), non-volatile memory ports, multiple antennas, graphics chips, application Specific Integrated Circuits (ASICs), and speakers.

It should be noted that the present application may be implemented in software and/or a combination of software and hardware, for example, using Application Specific Integrated Circuits (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software programs of the present application may be executed by a processor to implement the steps or functions as described above. Likewise, the software programs of the present application (including associated data structures) may be stored on a computer readable recording medium, such as RAM memory, magnetic or optical drive or diskette and the like. In addition, some steps or functions of the present application may be implemented in hardware, for example, as circuitry that cooperates with the processor to perform various steps or functions.

Furthermore, portions of the present application may be implemented as a computer program product, such as computer program instructions, which when executed by a computer, may invoke or provide methods and/or techniques in accordance with the present application by way of operation of the computer. Those skilled in the art will appreciate that the form of computer program instructions present in a computer readable medium includes, but is not limited to, source files, executable files, installation package files, etc., and accordingly, the manner in which the computer program instructions are executed by a computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Herein, a computer-readable medium may be any available computer-readable storage medium or communication medium that can be accessed by a computer.

Communication media includes media whereby a communication signal containing, for example, computer readable instructions, data structures, program modules, or other data, is transferred from one system to another. Communication media may include conductive transmission media such as electrical cables and wires (e.g., optical fibers, coaxial, etc.) and wireless (non-conductive transmission) media capable of transmitting energy waves, such as acoustic, electromagnetic, RF, microwave, and infrared. Computer readable instructions, data structures, program modules, or other data may be embodied as a modulated data signal, for example, in a wireless medium, such as a carrier wave or similar mechanism, such as that embodied as part of spread spectrum technology. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. The modulation may be analog, digital or hybrid modulation techniques.

By way of example, and not limitation, computer-readable storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. For example, computer-readable storage media include, but are not limited to, volatile memory, such as random access memory (RAM, DRAM, SRAM); and nonvolatile memory such as flash memory, various read only memory (ROM, PROM, EPROM, EEPROM), magnetic and ferromagnetic/ferroelectric memory (MRAM, feRAM); and magnetic and optical storage devices (hard disk, tape, CD, DVD); or other now known media or later developed computer-readable information/data that can be stored for use by a computer system.

An embodiment according to the present application comprises an apparatus comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein the computer program instructions, when executed by the processor, trigger the apparatus to operate a method and/or a solution according to the embodiments of the present application as described above.

It will be evident to those skilled in the art that the present application is not limited to the details of the foregoing illustrative embodiments, and that the present application may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned. Furthermore, it is evident that the word "comprising" does not exclude other elements or steps, and that the singular does not exclude a plurality. A plurality of units or means recited in the apparatus claims can also be implemented by means of one unit or means in software or hardware. The terms first, second, etc. are used to denote a name, but not any particular order.

Claims

1. A method for acquiring microscopic image information of a sample, applied to a user equipment, wherein the method comprises:

transmitting image acquisition instruction information about a sample to a corresponding microscopic device, wherein the image acquisition instruction information is used for controlling the microscopic device so as to enable an objective lens of the microscopic device to generate relative motion relative to the sample, and acquiring microscopic sub-image information about the sample through the microscopic device in the process of the relative motion, wherein the relative motion comprises relative plane motion of the objective lens relative to a stage corresponding to the sample in a direction of the stage corresponding to the stage, and relative back and forth motion of the objective lens relative to the stage in a vertical direction of the stage corresponding to the plane, and the relative plane motion at least partially overlaps with the relative back and forth motion in the process of the relative motion;

wherein when the relative planar movement overlaps with the relative reciprocal movement, the microscopy apparatus is controlled to acquire microscopic sub-image information about the specimen at a first frequency, the relative planar movement having a displacement variation in a planar direction within a temporal neighborhood of each overlapping time corresponding to an overlapping time of the overlapping movement, a speed of the relative planar movement being non-zero, during the relative planar movement overlapping with the relative reciprocal movement, the microscopy apparatus is controlled to acquire at least two microscopic sub-image information about the specimen at the first frequency, the at least two microscopic sub-image information including microscopic sub-image information having different photographed areas and focal plane heights;

The image acquisition instruction information further comprises a relative plane movement speed in the relative plane movement process and a relative round-trip movement speed in one period, when the relative plane movement is overlapped with the relative round-trip movement, the objective lens performs relative plane movement relative to the objective table at a first relative plane movement speed in the plane direction corresponding to the objective table, the time of one period or half period of the relative round-trip movement is equal to the time of one unit area of movement of the first relative plane movement speed in the plane direction, and the unit area is determined by the multiplying power of the objective lens and the resolution of microscopic sub-image information;

receiving a plurality of microscopic sub-image information about the specimen returned by the microscopy apparatus;

generating microscopic image information about the sample according to the microscopic sub-image information, wherein the microscopic image information comprises two-dimensional and three-dimensional integrated image information, wherein an overlapping area corresponding to the partial overlapping of the two-dimensional and three-dimensional integrated image information is a three-dimensional image, other areas except the overlapping area are two-dimensional images, a reference plane presented by the two-dimensional images is level to a plane with zero height in the three-dimensional images, and the three-dimensional images are generated by the microscopic sub-image information under different focal plane heights of the partial overlapping corresponding overlapping areas.

2. The method of claim 1, wherein when the relative planar motion and the relative back and forth motion do not overlap, the microscopy apparatus is controlled to acquire microscopic sub-image information about the specimen at a second frequency, wherein the second frequency is less than the first frequency.

3. The method of claim 1, wherein the microscopic sub-image information satisfies: the overlapping portion of adjacent images is greater than a first threshold and less than a second threshold.

4. The method of claim 1, wherein when the relative planar motion overlaps the relative reciprocal motion and the relative reciprocal motion of the objective lens and the stage exists, the objective lens has a displacement variation amount with respect to the stage in a direction of a corresponding plane of the stage.

5. The method of any of claims 1-4, wherein the image acquisition instruction information further comprises a first start position and a first end position of the relative planar motion, the relative planar motion comprising movement of the objective lens relative to the stage in a direction of the stage corresponding plane from the first start position to the first end position.

6. The method of claim 1, wherein the objective lens performs a relative planar motion with respect to the stage in a planar direction corresponding to the stage at a second relative planar motion speed when the relative planar motion and the relative back-and-forth motion do not overlap, wherein the second relative planar motion speed is greater than the first relative planar motion speed.

7. The method of claim 1, wherein the first relative planar movement speed of the relative planar movement has a relative movement path that is greater than or equal to a length of a unit area corresponding to the objective lens during a half cycle time of the relative reciprocating movement, wherein the relative movement path includes a path of the relative planar movement of the objective lens relative to the stage in a planar direction corresponding to the stage.

8. The method of claim 1, wherein the generating microscopic image information about the specimen from the plurality of microscopic sub-image information comprises:

and determining the space coordinate information of each pixel point in the sample according to the plurality of microscopic sub-image information, and generating microscopic image information of the sample by utilizing the space coordinate information of each pixel point.

9. The method of claim 1, wherein the generating microscopic image information about the specimen from the plurality of microscopic sub-image information comprises:

and performing image matching on the plurality of microscopic sub-image information, and generating microscopic image information about the sample according to an image matching result.

10. The method of claim 1, wherein the generating microscopic image information about the specimen from the plurality of microscopic sub-image information comprises:

determining the space coordinate information of the image center of each microscopic sub-image information according to the microscopic sub-image information;

and determining adjacent microscopic sub-image information corresponding to each microscopic sub-image information according to the space coordinate information of the image center of each microscopic sub-image information, performing image matching on the microscopic sub-image information based on the adjacent microscopic sub-image information of the microscopic sub-image information, and generating microscopic image information about the sample according to an image matching result.

11. The method of any one of claims 1, 8 to 10, wherein the microscopic image information further comprises at least any one of:

two-dimensional microscopic image information;

Three-dimensional microscopic image information.

12. The method of claim 1, wherein the method further comprises:

performing feature matching on the latest microscopic sub-image information in the plurality of microscopic sub-image information, and if matching fails, generating corresponding image acquisition stop instruction information;

and sending the image acquisition stop instruction information to the microscopic equipment.

13. The method of claim 12, wherein the feature matching comprises at least any one of:

performing feature matching on the latest microscopic sub-image information and the microscopic sub-image information adjacent to the latest microscopic sub-image information;

and carrying out feature matching on the latest microscopic sub-image information and template feature information of the sample.

14. A method for acquiring microscopic image information of a sample, applied to a microscopic device, wherein the method comprises:

controlling the microscopic equipment according to the image acquisition instruction information so as to enable an objective lens of the microscopic equipment to generate relative motion relative to the sample, and acquiring a plurality of microscopic sub-image information about the sample through the microscopic equipment in the relative motion process, wherein the relative motion comprises relative plane motion of the objective lens relative to a stage corresponding to the sample in the direction of the stage corresponding to the sample, and relative back and forth motion of the objective lens relative to the stage in the vertical direction of the stage corresponding to the plane, and the relative plane motion is at least partially overlapped with the relative back and forth motion in the relative motion process;

Returning the plurality of microscopic sub-image information to the user device, wherein the plurality of microscopic sub-image information is used to perform the steps of:

15. The method of claim 14, wherein the method further comprises:

receiving image acquisition stopping instruction information sent by the user equipment;

and controlling the microscopic equipment to pause the current acquisition process based on the image acquisition stop instruction information.

16. A method for acquiring microscopic image information of a sample, wherein the method comprises:

the method comprises the steps that user equipment sends image acquisition instruction information about a sample to corresponding microscopic equipment, wherein the image acquisition instruction information is used for controlling the microscopic equipment so that an objective lens of the microscopic equipment moves relatively to the sample, microscopic sub-image information about the sample is acquired through the microscopic equipment in the process of the relative movement, the relative movement comprises relative plane movement of the objective lens relative to an objective table in the direction of a corresponding plane of the objective table for accommodating the sample, and relative back and forth movement of the objective lens relative to the objective table in the vertical direction of the corresponding plane of the objective table, and the relative plane movement and the relative back and forth movement at least partially overlap in the process of the relative movement; wherein when the relative planar movement overlaps with the relative reciprocal movement, the microscopy apparatus is controlled to acquire microscopic sub-image information about the specimen at a first frequency, the relative planar movement having a displacement variation in a planar direction within a temporal neighborhood of each overlapping time corresponding to an overlapping time of the overlapping movement, a speed of the relative planar movement being non-zero, during the relative planar movement overlapping with the relative reciprocal movement, the microscopy apparatus is controlled to acquire at least two microscopic sub-image information about the specimen at the first frequency, the at least two microscopic sub-image information including microscopic sub-image information having different photographed areas and focal plane heights;

The microscopic equipment receives image acquisition instruction information about a sample, which is sent by corresponding user equipment, controls the objective lens to keep relative motion relative to the objective table according to the image acquisition instruction information, and acquires a plurality of pieces of sub-image information about the sample through the microscopic equipment in the relative motion process; the microscopy equipment returns the plurality of microscopy sub-image information to the user equipment;

The user equipment receives a plurality of microscopic sub-image information about the sample returned by the microscopic equipment; generating microscopic image information about the sample according to the microscopic sub-image information, wherein the microscopic image information comprises two-dimensional and three-dimensional integrated image information, wherein an overlapping area corresponding to the partial overlapping of the two-dimensional and three-dimensional integrated image information is a three-dimensional image, other areas except the overlapping area are two-dimensional images, a reference plane presented by the two-dimensional images is level to a plane with zero height in the three-dimensional images, and the three-dimensional images are generated by the microscopic sub-image information under different focal plane heights of the partial overlapping corresponding overlapping areas.

17. A user device for acquiring microscopic image information of a sample, wherein the device comprises:

the device comprises a one-to-one module, a microscope device and a sample receiving module, wherein the one-to-one module is used for sending image acquisition instruction information about a sample to a corresponding microscope device, the image acquisition instruction information is used for controlling the microscope device so as to enable an objective lens of the microscope device to generate relative motion relative to the sample, and microscopic sub-image information about the sample is acquired through the microscope device in the process of the relative motion, the relative motion comprises relative plane motion of the objective lens relative to an objective table in the direction of a corresponding plane of the objective table for accommodating the sample, and relative back-and-forth motion of the objective lens relative to the objective table in the vertical direction of the corresponding plane of the objective table, and the relative plane motion and the relative back-and-forth motion are at least partially overlapped in the process of the relative motion;

the image acquisition instruction information further comprises a relative plane movement speed in the relative plane movement process and a relative round-trip movement speed in one period, when the relative plane movement overlaps with the relative round-trip movement, the objective lens performs relative plane movement relative to the objective table at a first relative plane movement speed in the plane direction corresponding to the objective table, the time of one period or half period of the relative round-trip movement is equal to the time of one unit area of the first relative plane movement speed in the plane direction, and the unit area comprises the actual size of the microscopic sub-image information corresponding to the sample range;

18. A microscopy apparatus for acquiring microscopic image information of a sample, wherein the apparatus comprises:

the second-to-second module is used for controlling the microscopic equipment according to the image acquisition instruction information so as to enable the objective lens of the microscopic equipment to generate relative motion relative to the sample, and acquiring a plurality of microscopic sub-image information about the sample through the microscopic equipment in the relative motion process, wherein the relative motion comprises relative plane motion of the objective lens relative to the objective table in the direction of the corresponding plane of the objective table for accommodating the sample, and relative back and forth motion of the objective lens relative to the objective table in the vertical direction of the corresponding plane of the objective table, and the relative plane motion and the relative back and forth motion are at least partially overlapped in the relative motion process;

19. An apparatus for acquiring microscopic image information of a specimen, wherein the apparatus comprises:

a processor; and

a memory arranged to store computer executable instructions which, when executed, cause the processor to perform the operations of the method of any one of claims 1 to 15.

20. A computer readable medium storing instructions that, when executed, cause a system to perform the operations of the method of any one of claims 1 to 15.