Disclosure of Invention
The embodiment of the invention aims to provide an ultrasonic scanning microscope global focusing method and device combined with a depth camera, so as to solve the technical problem that the traditional high-resolution ultrasonic scanning microscope is difficult to automatically focus a warped sample.
According to a first aspect of the embodiments of the present application, there is provided a global focusing method for an ultrasound scanning microscope with a depth camera, applied to a terminal, the method including:
acquiring a three-dimensional coordinate matrix of the upper surface of the sample through a depth camera;
calculating a coordinate position point matrix of the moving position of the ultrasonic probe according to the three-dimensional coordinate matrix, wherein the coordinate position point matrix is composed of a plurality of coordinate position points;
and controlling an XYZ displacement table according to the coordinate position point matrix to enable the ultrasonic probe to sequentially move to the coordinate position points, thereby finishing overall focusing.
Further, acquiring a three-dimensional coordinate matrix of the upper surface of the sample by a depth camera, comprising:
and extracting coordinates of the upper surface of the sample in the expected scanning range from the three-dimensional data obtained by the depth camera, wherein the coordinates comprise an X-direction coordinate array X1 and a Y-direction coordinate array Y1, an upper surface Z-axis coordinate matrix Z1 corresponding to each XY plane coordinate, and the coordinate origin is arranged on a horizontal plane on which the sample is placed.
Further, according to the three-dimensional coordinate matrix, calculating a coordinate position point matrix of the moving position of the ultrasonic probe, including:
according to an X-direction coordinate array X1, a Y-direction coordinate array Y1 and a Z-axis coordinate matrix Z1, calculating to obtain a coordinate position point matrix of the moving position of the ultrasonic probe, wherein the coordinate position point matrix comprises: an X-direction coordinate array X1, a Y-direction coordinate array Y1 and a Z-axis coordinate matrix Z2 corresponding to each XY coordinate point, wherein the relation between Z2 and Z1 is as follows: and Z2 is Z1+ a-b, wherein a is the focal length of the ultrasonic probe, and b is the relative distance between the target focal position and the upper surface of the sample.
According to a second aspect of the embodiments of the present application, there is provided an ultrasonic scanning microscope global focusing apparatus combined with a depth camera, applied to a terminal, the apparatus including:
the acquisition module is used for acquiring a three-dimensional coordinate matrix of the upper surface of the sample through the depth camera;
the calculation module is used for calculating a coordinate position point matrix of the moving position of the ultrasonic probe according to the three-dimensional coordinate matrix, and the coordinate position point matrix is composed of a plurality of coordinate position points;
and the control module is used for controlling the XYZ displacement table according to the coordinate position point matrix to enable the ultrasonic probe to sequentially move to the coordinate position points so as to complete global focusing.
According to a third aspect of embodiments of the present application, there is provided an ultrasonic scanning microscope global focusing system in combination with a depth camera, comprising: an XYZ displacement stage; the motor driver is electrically connected with the XYZ displacement table and is used for controlling the movement of the XYZ displacement table; the depth camera is used for acquiring a three-dimensional coordinate matrix of the upper surface of the sample; an ultrasonic probe mounted on the XYZ displacement table; and the terminal is respectively electrically connected with the motor driver and the depth camera and is used for outputting a control signal to the motor driver, receiving the three-dimensional coordinate matrix, calculating a coordinate position point matrix of the moving position of the ultrasonic probe according to the three-dimensional coordinate matrix, wherein the coordinate position point matrix is formed by a plurality of coordinate position points, and controlling the XYZ displacement table through the motor driver according to the coordinate position point matrix to enable the ultrasonic probe to sequentially move to the coordinate position points.
According to a fourth aspect of embodiments of the present application, there is provided an electronic apparatus, including: one or more processors; a memory for storing one or more programs; when executed by the one or more processors, cause the one or more processors to implement a method as described in the first aspect.
According to a fifth aspect of embodiments herein, there is provided a computer readable storage medium having stored thereon computer instructions which, when executed by a processor, implement the steps of the method according to the first aspect.
Compared with the prior art, the invention has the following beneficial effects: because the three-dimensional appearance of the upper surface of the sample is obtained by adopting the depth camera, the invention can quickly calculate the global focusing position coordinate, so that the global focusing can be realized by one-time formal scanning. After the global focusing is completed by the method provided by the embodiment of the invention, the method can be used for uniformly and clearly imaging warped and inclined samples, the traditional C scanning and multilayer image splicing of different depths are not needed for multiple times, and the sample detection speed is higher.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, embodiments accompanying the present application are described in detail below with reference to the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. This application is capable of embodiments in many different forms than those described herein and those skilled in the art will be able to make similar modifications without departing from the spirit of the application and it is therefore not intended to be limited to the embodiments disclosed below.
FIG. 1 is a flow chart illustrating a method for global focusing of an ultrasound scanning microscope in conjunction with a depth camera in accordance with an exemplary embodiment. The embodiment of the invention provides an ultrasonic scanning microscope global focusing method combined with a depth camera, which can be applied to a terminal and comprises the following steps:
step S11, acquiring a three-dimensional coordinate matrix of the upper surface of the sample through a depth camera;
step S12, calculating a coordinate position point matrix of the moving position of the ultrasonic probe according to the three-dimensional coordinate matrix, wherein the coordinate position point matrix is composed of a plurality of coordinate position points;
and step S13, controlling an XYZ displacement table according to the coordinate position point matrix, and enabling the ultrasonic probe to sequentially move to the coordinate position points to complete global focusing.
According to the technical scheme, as the three-dimensional morphology (namely the three-dimensional coordinate matrix) of the upper surface of the sample is obtained by adopting the depth camera, the method can quickly calculate the global focusing position coordinate, so that the global focusing can be realized through one-time formal scanning. After the global focusing is completed by the method provided by the embodiment of the invention, the method is suitable for uniformly and clearly imaging warped and inclined samples, the traditional C scanning and multilayer image splicing of different depths are not needed for multiple times, and the sample detection speed is higher.
In a specific implementation of step S11, acquiring a three-dimensional coordinate matrix of the upper surface of the sample by the depth camera;
specifically, from the three-dimensional data obtained by the depth camera, the coordinates of the upper surface of the sample within the expected scanning range are extracted, including an X-direction coordinate array X1 and a Y-direction coordinate array Y1, an upper surface Z-axis coordinate matrix Z1 corresponding to each XY plane coordinate, with the origin of coordinates set on the horizontal plane on which the sample is placed.
In a specific implementation of step S12, calculating a coordinate position point matrix of the ultrasonic probe moving position according to the three-dimensional coordinate matrix;
specifically, a coordinate position point matrix of the moving position of the ultrasonic probe is calculated according to an X-direction coordinate array X1, a Y-direction coordinate array Y1 and a Z-axis coordinate matrix Z1, where the coordinate position point matrix includes: an X-direction coordinate array X1, a Y-direction coordinate array Y1 and a Z-axis coordinate matrix Z2 corresponding to each XY coordinate point, wherein the relation between Z2 and Z1 is as follows: z2 is Z1+ a-b, where a is the focal length of the probe and b is the relative distance of the target focal position 9 from the sample upper surface 8.
In a specific implementation of step S13, the XYZ stage is controlled according to the coordinate location point matrix, so that the ultrasound probe sequentially moves to the coordinate location points, thereby completing global focusing.
Specifically, the XYZ stage is controlled to sequentially move the ultrasound probe to the calculated coordinate position points, i.e., the coordinate sequence determined by X1, Y1, Z2.
Corresponding to the foregoing embodiment of the method for global focusing of an ultrasound scanning microscope with a depth camera, the present application also provides an embodiment of a device for global focusing of an ultrasound scanning microscope with a depth camera.
FIG. 2 is a block diagram illustrating an ultrasonic scanning microscope global focus device in combination with a depth camera in accordance with an exemplary embodiment. Referring to fig. 2, the apparatus includes:
the acquisition module 11 is used for acquiring a three-dimensional coordinate matrix of the upper surface of the sample through the depth camera;
the calculation module 12 is configured to calculate a coordinate position point matrix of the moving position of the ultrasonic probe according to the three-dimensional coordinate matrix, where the coordinate position point matrix is formed by a plurality of coordinate position points;
and the control module 13 is used for controlling the XYZ displacement table according to the coordinate position point matrix, so that the ultrasonic probe is sequentially moved to the coordinate position points to complete global focusing.
With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.
For the device embodiments, since they substantially correspond to the method embodiments, reference may be made to the partial description of the method embodiments for relevant points. The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of the application. One of ordinary skill in the art can understand and implement it without inventive effort.
Correspondingly, the present application also provides an electronic device, comprising: one or more processors; a memory for storing one or more programs; when executed by the one or more processors, cause the one or more processors to implement a method of global focus for an ultrasound scanning microscope in conjunction with a depth camera as described above.
Accordingly, the present application also provides a computer readable storage medium having stored thereon computer instructions, wherein the instructions, when executed by a processor, implement a method for global focusing of an ultrasound scanning microscope in combination with a depth camera as described above.
Fig. 3 is a schematic diagram of an overall focusing system of an ultrasonic scanning microscope with a depth camera according to an exemplary embodiment of the present invention. The embodiment of the present application further provides an ultrasonic scanning microscope global focusing system combined with a depth camera, which includes: an XYZ displacement stage 1; a motor driver 2 electrically connected to the XYZ stage 1, the motor driver 2 controlling movement of the XYZ stage 1; the depth camera 3 is used for acquiring a three-dimensional coordinate matrix of the upper surface of the sample 4; an ultrasonic probe 5 mounted on the XYZ stage 1; and the terminal 6 is respectively electrically connected with the motor driver 2 and the depth camera 3 and is used for outputting a control signal to the motor driver 2, receiving the three-dimensional coordinate matrix, calculating a coordinate position point matrix of the moving position of the ultrasonic probe according to the three-dimensional coordinate matrix, wherein the coordinate position point matrix is formed by a plurality of coordinate position points, and controlling the XYZ displacement table 1 through the motor driver 2 according to the coordinate position point matrix to enable the ultrasonic probe 5 to sequentially move to the coordinate position points.
The system can clearly image the warped sample. The warped sample is characterized in that the upper surface is not a horizontal plane, and the relative distance between the target detection structure and the upper surface is fixed. In particular, the sample may be a warped wafer, a squeezed sample, a flexible composite, a chip placed with a tilt, or the like. The depth camera 3 is fixed right above the sample, can acquire the three-dimensional appearance of the upper surface of the sample, and transmits the result to an upper computer for processing. The depth camera is also called a 3D camera, can acquire distance information from an object to the camera, and can calculate three-dimensional coordinates of each point by adding X, Y coordinates of a 2D plane, and can be applied to three-dimensional reconstruction, target positioning and the like. At present, mainstream depth cameras are classified in principle, and include three types, namely structured light, a time flight method and binocular stereo. The depth camera can construct the three-dimensional appearance of the surface of an article in a very short time, and provides three-dimensional coordinate information for the global focusing of the ultrasonic scanning microscope.
The ultrasonic probe is fixed on an XYZ displacement table, and the upper computer can control the displacement table to perform linear movement in the XYZ three-axis direction through a motor driver, so that the ultrasonic probe is driven to perform XY plane scanning and Z-direction focusing adjustment. And an ultrasonic transmitting and receiving device is also connected between the ultrasonic probe and the terminal and is responsible for receiving an ultrasonic trigger signal of the upper computer, controlling the ultrasonic probe to transmit ultrasonic waves, receiving echo signals of a sample and transmitting the echo signals to the upper computer.
Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention following, in general, the principles of the application and including such departures from the present disclosure as come within known or customary practice within the art to which the invention pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the application being indicated by the following claims.
It will be understood that the present application is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the application is limited only by the appended claims.