CN113526125B - A cell specimen sample handling system and method based on multi-label positioning - Google Patents

Abstract

The utility model discloses a cell specimen sample carrying system and method based on multi-label positioning, which comprises a mobile robot and a robot internal control host; a laser radar and a camera are arranged on a mechanical arm of the mobile robot; the label is arranged on the tray for placing the cell specimen sample wafer; according to the cell specimen sample handling robot, the coordinate deviation is extracted by using the two-dimensional code label which is easy to recognize, the repositioning correction of a world coordinate system is carried out on the robot, and the problem of grabbing deviation of the mechanical arm caused by insufficient self-navigation precision is solved, so that the purpose that the mechanical arm accurately carries cell specimen samples among different sample trays on different object carrying tables is guaranteed.

Description

A cell specimen sample handling system and method based on multi-label positioning

technical field

The disclosure belongs to the technical field of robot handling, and in particular relates to a multi-label localization-based cell specimen sample handling system and method.

Background technique

The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

With the rapid development of artificial intelligence, the intelligence of all walks of life is becoming more and more obvious; the application of handling robots is becoming more and more extensive. In some places such as hospitals or biological research institutes that require a large number of observation and analysis of cells, mobile robots are used to carry cell specimens Samples will save a lot of manpower and material resources.

In the process of moving cell specimens, the mobile robot has high requirements for operational stability. If some misoperations occur, the cell samples will be polluted or damaged; because the cell specimen tray groove requires high precision for picking and placing, and the mobile robot does not Unlike ordinary manipulators that are fixed at a fixed position, the mobile robot has certain navigation errors during the movement process, which causes the position of the manipulator coordinate system of the mobile robot in the world coordinate system to be unstable, and the target extraction algorithm based on machine vision cannot do it. high-precision target positioning, which leads to the inability to guarantee the handling accuracy, and it is impossible to put the cell specimens into the grooves of the sample trays every time; Stability is lacking.

Contents of the invention

In order to solve the above problems, the present disclosure proposes a cell specimen sample handling system and method based on multi-label positioning; the present disclosure uses easily identifiable two-dimensional code labels to extract coordinate deviations, and repositions and corrects the world coordinate system of the robot, which solves the problem of The grasping deviation problem caused by the lack of autonomous navigation accuracy of the robotic arm ensures that the robotic arm can accurately transport cell specimen samples between different sample trays on different stages.

In order to achieve the above purpose, in the first aspect, the present disclosure provides a cell specimen sample handling system based on multi-label positioning, which adopts the following technical solution:

A cell sample sample handling system based on multi-label positioning, including a mobile robot and an internal control host of the robot; a laser radar and a camera are arranged on the mechanical arm of the mobile robot; the tags are arranged on a tray for placing cell sample samples;

The radar is used to perceive the surrounding environment of the mobile robot;

The camera is used to capture the image of the tray;

The internal control host of the robot is configured as:

Obtain the coordinate information of the label in the image captured by the camera when the mobile robot is docked at the standard position, and calculate the standard position vector information according to the coordinate information;

On the basis of perceiving the surrounding environment of the mobile robot, the SLAM algorithm is used to establish a global map, and the position of the mobile robot in the global map is determined, and an autonomous navigation route is established;

According to the autonomous navigation route, control the mobile robot to move to the designated grasping position of the first stage; perform target recognition on the image captured by the camera at the designated grasping position, read the position coordinate information of the label in the image, and calculate the current position vector information;

Calculate the correction amount of the world coordinate system according to the standard position vector information and the current position vector information;

Use machine vision technology to detect the target of the image, determine the cell specimen sample in the tray, and determine the grabbing coordinates in combination with the position information of the tray groove where the cell specimen sample is located;

Correcting the grasping coordinates by the correction amount, and controlling the mechanical arm to grasp the cell specimen sample at the target position according to the corrected grasping coordinates;

Controlling the mobile robot to carry the cell sample sheet.

Further, the mechanical arm is also provided with a pneumatic gripper suction cup for grabbing the cell sample piece.

Further, the labels are two-dimensional code labels, and there are three in number, which are respectively arranged at three corners of the tray.

Further, when the mobile robot is at a specified grab position, the camera takes pictures continuously, performs machine vision processing on the images, captures the two-dimensional code label information, and slowly controls the body of the mobile robot through the characteristics of the differential wheel of the vehicle body Slowly rotate on the spot to fine-tune the direction until three QR code labels appear simultaneously in the field of view of the camera.

Further, when the mobile robot stops at the standard position, it also obtains the grabbing coordinates and the grabbing and placing height of the storage groove of each cell specimen sample on the tray.

Further, after determining the grabbing coordinates, calculate the correction amount and offset angle between the grabbing coordinates and the world coordinate system; after determining the current grabbing coordinates, control the mechanical arm to move on the pallet at a speed of 1 mm/s Move to the current grasping coordinates in the upper two-dimensional plane until it moves above the current grasping coordinates, stop the horizontal movement, and vertically lower the Z-axis height of the robotic arm to the grasping height, and execute the grasping action.

Further, after performing the grabbing action, move the end of the mechanical arm to the photographing position, acquire the image information of the tray, and perform target detection of the cell specimen in the groove of the grabbing position; if no cell is detected If it is a sample sample, it is judged that the cell sample sample capture is successful, otherwise it is judged that the cell sample sample capture fails; if the analysis shows that the cell sample sample capture fails, the capture operation is re-executed until the capture is successful.

Further, after the cell specimen sample is successfully captured, the mobile robot is controlled to move to the specified capture position in front of the second stage according to the autonomous navigation route; the target recognition is performed on the image captured by the camera at the specified capture position, and the reading Get the position coordinate information of the label in the image, and calculate the current position vector information;

Calculate the correction amount of the world coordinate system according to the standard position vector information and the current position vector information;

Use machine vision technology to detect the target of the image, determine the vacant groove in the tray, and determine the placement coordinates in combination with the position information of the tray groove where the cell specimen is located;

The placement coordinates are corrected by the correction amount, and according to the corrected placement coordinates, the mechanical arm is controlled to place the cell sample piece into the target vacant groove.

Further, the correction amount includes an offset and an offset angle between the standard position vector and the current position vector.

In order to achieve the above purpose, in the second aspect, the present disclosure provides a cell specimen sample handling method based on multi-label positioning, which adopts the following technical scheme:

Obtain the coordinate information of the label in the image at the standard position, and calculate the standard position vector information according to the coordinate information;

Utilize the SLAM algorithm to establish a global map, and determine the position of the mobile robot in the global map, and establish an autonomous navigation route;

According to the autonomous navigation route, control the mobile robot to move to the designated grab position; perform target recognition on the image taken at the designated grab position, read the position coordinate information of the label in the image, and calculate the current position vector information;

Calculate the correction amount of the world coordinate system according to the standard position vector information and the current position vector information;

Use machine vision technology to detect the target of the image, determine the cell specimen sample in the tray, and determine the grabbing coordinates in combination with the position information of the tray groove where the cell specimen sample is located;

The grasping coordinates are corrected by the correction amount, and according to the corrected grasping coordinates, the mechanical arm is controlled to grasp the cell specimen sample at the target position, and the mobile robot is controlled to transport the cell specimen specimen.

Compared with the prior art, the beneficial effects of the present disclosure are:

1. This disclosure uses an easily identifiable two-dimensional code label to extract coordinate deviations, and repositions and corrects the world coordinate system of the robot, which solves the problem of grasping deviation caused by insufficient autonomous navigation accuracy of the robotic arm, thereby ensuring that the robotic arm is in different positions. The purpose of accurately transporting cell specimen samples between different sample trays on the stage.

Description of drawings

The accompanying drawings constituting a part of this embodiment are used to provide a further understanding of this embodiment, and the schematic embodiments and descriptions of this embodiment are used to explain this embodiment and do not constitute an improper limitation to this embodiment.

FIG. 1 is a schematic structural diagram of Embodiment 1 of the present disclosure;

FIG. 2 is a label layout diagram of Embodiment 1 of the present disclosure;

FIG. 3 is a schematic diagram of coordinate system conversion in Embodiment 1 of the present disclosure;

Fig. 4 is the work flowchart of embodiment 1 of the present disclosure;

Among them, 1. Mobile robot, 2. Mechanical arm, 3. Camera, 4. Pneumatic gripper suction cup, 5. Tray, 6. Object stage.

Detailed ways:

The present disclosure will be further described below in conjunction with the accompanying drawings and embodiments.

It should be pointed out that the following detailed description is exemplary and intended to provide further explanation to the present application. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

Example 1:

As shown in FIG. 1 , the present disclosure provides a cell specimen sample handling system based on multi-label positioning, including a mobile robot 1 and a robot internal control host; the robotic arm 2 of the mobile robot 1 is provided with a laser radar and a camera 3 ; The label is set on the tray 5 where the cell specimen sample is placed;

Specifically, the camera is a 3-bit monocular camera, the end of the mechanical arm 2 is provided with a pneumatic gripper suction cup 4, the lidar is arranged on the base of the mobile robot 1, and the surrounding environment is sensed through the lidar, The interior of the mobile robot 1 is provided with an internal control host, which establishes a global navigation map through the SLAM algorithm, and can know its position in the map at all times through the SLAM algorithm, and completes the process of moving the mobile robot 1 from the stop point to the load. autonomous navigation of the stage; the tray 5 of the cell specimen sample on the stage is photographed by the monocular camera to obtain image information, and the position information of the target cell specimen sample is obtained through the target detection algorithm; the mechanical arm 2 and the The pneumatic gripper suction cup 4 grabs and moves the cell specimen sample at the target position; it can be understood that the internal control host establishes a connection with the camera 3 and the laser radar, and provides all actions of the mobile robot Excuting an order.

The working process or principle of this embodiment are:

和

且定义

由此我们得到标准位置矢量信息，该标准位置矢量信息用于后续对世界坐标系进行重定位修正的计算；然后对所述托盘5上每个细胞标本样片的存放凹槽分别当以为：凹槽1、凹槽2、……、凹槽6，并取每个凹槽的中心位置为抓取坐标，分别定义为(x₁,y₁)，(x₂,y₂)，……，(x₆,y₆)。As shown in Figure 2, the two-dimensional code label is used for the label, and a two-dimensional code label is respectively fixed on the upper left, lower left, and lower right triangles of the tray 5, which are respectively defined as three points of coordinates A, B, and C, and Calculated vector

and

and define

Thus we obtain the standard position vector information, which is used for the subsequent calculation of the relocation correction of the world coordinate system; 1. Groove 2, ..., groove 6, and take the center position of each groove as the grabbing coordinates, which are defined as (x ₁ ,y ₁ ), (x ₂ ,y ₂ ),..., ( x ₆ ,y ₆ ).

和

以及

定义

为当前位置矢量，由此我们得到当前位置矢量信息。When the mobile robot 1 autonomously navigates and stops before the loading platform (the first loading platform), the tray 5 is photographed, and the current image is read to obtain the position coordinate information of the three two-dimensional code labels in the image. , respectively defined as A', B', C', the calculated vector

and

as well as

definition

is the current position vector, from which we get the current position vector information.

和前位置矢量

的关系，结合

和

进行矢量计算，求两向量的偏移量(a,b)和偏移角θ：As shown in Figure 3, it is the standard position vector

and the front position vector

relationship, combined

and

Perform vector calculations to find the offset (a, b) and offset angle θ of the two vectors:

Offset:

Offset angle:

The offset and angle between the two vectors are defined as the correction of the world coordinate system, from which the offset error of the current docking position of the mobile robot 1 relative to the standard docking position can be calculated, and then the world coordinate system can be corrected.

The mobile robot 1 performs target detection on the tray 5 through the monocular camera, determines the position information of the groove of the tray 5 where the cell specimen sample is located, and then determines the number i of the groove that needs to be grasped, and finds the corresponding grasping position. Take the coordinates (x _i , y _i ). Calculate the grasping coordinates ( _xi , y _i ) with the correction amount (a, b) of the world coordinate system and the offset angle θ to obtain the current grasping coordinates (x' _i , y' _i ):

After determining the current grabbing coordinates (xi _' , y _i '), control the mechanical arm 2 to move at a speed of 1 mm/s in the two-dimensional plane above the pallet 5 to the current grabbing coordinates (xi _' , y _i ') move until it moves above the current grabbing coordinates, stop the horizontal movement, and vertically lower the Z-axis height of the robotic arm to the grabbing height z, execute the grabbing action, and control the gripper suction cup to grab the target cell specimen sample.

After performing the grasping action, move the end of the mechanical arm 2 to the photographing position, take a photograph of the tray 5 to obtain image information, and perform target detection on the cell specimen sample in the groove of the grasping position, if no cell is detected The specimen sample is determined as the cell specimen sample capture is successful, otherwise it is determined that the cell specimen sample capture fails. If the grasping of the analyzed cell specimen sample fails, the grasping operation is performed again until the grasping is successful.

After successfully grabbing the cell specimen sample, the mobile robot 1 moves to the designated grabbing position in front of the second stage; then the mobile robot 1 turns on the camera function of the monocular camera at the end of the mechanical arm 2 and takes pictures continuously , machine vision processing is performed on the photo to capture the two-dimensional code label information, and at the same time, through the characteristics of the differential wheel of the mobile robot 1, the body is slowly controlled to rotate slowly in place to fine-tune the direction until three two-dimensional codes are captured in the camera field of view. The two-dimensional code label appears at the same time, and the car body of the mobile robot 1 stops rotating.

After the mobile robot finds the correct photographing position, the pallet 5 is photographed by the camera at the end of the mechanical arm 2 . Perform target recognition on the image captured by the camera, read the position coordinate information of the three two-dimensional code labels in the image, repeat the above process, and obtain the correction amount of the world coordinate system of the mobile robot 1 in front of the second stage.

The mobile robot 1 takes pictures of the tray 5 with a monocular camera to obtain image information, and uses machine vision technology to perform target detection on the image to find empty grooves in the tray 5 . Through the center coordinates of the target candidate frame, the position information of the pallet 5 groove where the vacant groove is located can be roughly judged, and then the groove number j is judged, and the corresponding placement coordinates (x _j , y _j ) are found, and passed The correction amount of the world coordinate system is corrected to obtain the current placement coordinates (x _j ', y _j '). Then control the mechanical arm to move at a speed of 1 mm/s to move to the current placement coordinates (x _j ', y _j ') in the two-dimensional plane above the tray 5 until it moves above the current placement coordinates, stop the horizontal movement, and Vertically lower the height of the Z-axis of the mechanical arm to the placement height z, perform the placement action, and control the gripper suction cup to place the cell specimen sample into the target vacant groove.

After performing the placement action, move the end of the mechanical arm 2 to the photographing position, take a photograph of the tray 5 to obtain image information, and perform target detection on the cell specimen sample in the groove of the placement position. If the cell specimen specimen is detected, then It is judged that the placement of the cell specimen is successful, otherwise it is judged that the placement of the cell specimen is failed. If the placement of the analyzed cell specimen sample fails, the placement operation is performed again until the placement is successful.

After performing the above business, it is judged that the handling of the cell sample piece has been completed.

和凹槽抓取坐标(x₁,y₁)，(x₂,y₂)，……，(x₆,y₆)，定义该位置为标准位置，该位置矢量为标准位置矢量，该抓取坐标为标准抓取坐标；Initialize, dock the mobile robot 1 at any position close to the stage 6, take a photo of the tray 5, read the coordinate information of the two-dimensional code label in the photo, and obtain the position vector

and groove grab coordinates (x ₁ , y ₁ ), (x ₂ , y ₂ ), ..., (x ₆ , y ₆ ), define the position as the standard position, the position vector as the standard position vector, the grab Take the coordinates as the standard capture coordinates;

After the mobile robot 1 receives the handling task, it first navigates the mobile robot 1 through the SLAM algorithm and moves to the loading platform 1 to specify a grabbing position;

The mobile robot 1 captures the two-dimensional code label information through a monocular camera, and controls the vehicle body of the mobile robot 1 to slowly rotate in situ to fine-tune the direction until three two-dimensional code labels are captured simultaneously in the camera field of view;

After the mobile robot 1 finds the correct photographing position, it captures three two-dimensional code label position coordinate information through a monocular camera to obtain the current position vector

和

进行矢量计算，求两向量的偏移量(a,b)和偏移角θ，作为世界坐标系的修正量；combine

and

Carry out vector calculation, find the offset (a, b) and offset angle θ of the two vectors, as the correction amount of the world coordinate system;

The mobile robot 1 performs target detection on the cell specimen sample through a monocular camera, and obtains the grasping coordinates ( _xi , y _i ) of the cell specimen sample, and if no cell specimen sample is detected, then navigates the mobile robot 1 back to Waiting area, waiting for the next handling task;

Correct the grab coordinates (xi, y _i ) of the cell specimen in the world coordinate system through the offset (a, b) and offset angle θ to obtain the current grab coordinates (x _{' i ,} y' _i );

Grabbing action, control the mechanical arm 2 to move to the sky above the current grabbing coordinates at a speed of 1mm/s, and vertically lower the Z-axis height of the robotic arm to the grabbing height z, perform the grabbing action, and control the gripper suction cup to grab Take target cell specimen samples;

After performing the grasping action, move the end of the mechanical arm 2 to the photographing position, take a photograph of the tray 5 to obtain image information, and perform target detection on the cell specimen sample in the groove of the grasping position, if no cell is detected The specimen sample is determined as the cell specimen sample capture is successful, otherwise it is determined that the cell specimen sample capture fails. If it is analyzed that the grabbing of the cell specimen sample fails, the grabbing operation is performed again until the grabbing is successful;

After the execution of the grabbing is successful, the mobile robot 1 is navigated to the loading platform 2, and the placement action is performed.

Example 2:

This embodiment provides a cell sample sample handling method based on multi-label positioning, using the cell sample sample handling system based on multi-label positioning as described in Example 1, the main contents include:

Obtain the coordinate information of the label in the image at the standard position, and calculate the standard position vector information according to the coordinate information;

Utilize the SLAM algorithm to establish a global map, and determine the position of the mobile robot in the global map, and establish an autonomous navigation route;

According to the autonomous navigation route, control the mobile robot to move to a designated grab position; perform target recognition on the image taken at the designated grab position, read the position coordinate information of the label in the image, and calculate the current position vector information;

Calculate the correction amount of the world coordinate system according to the standard position vector information and the current position vector information;

Use machine vision technology to detect the target of the image, determine the cell specimen sample in the tray, and determine the grabbing coordinates in combination with the position information of the tray groove where the cell specimen sample is located;

The grasping coordinates are corrected by the correction amount, and according to the corrected grasping coordinates, the mechanical arm 2 is controlled to grasp the cell specimen sample at the target position, and the mobile robot is controlled to transport the cell specimen specimen.

The above descriptions are only preferred embodiments of this embodiment, and are not intended to limit this embodiment. For those skilled in the art, various modifications and changes may be made to this embodiment. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this embodiment shall be included in the protection scope of this embodiment.

Claims (10)

1. A cell sample sample handling system based on multi-label positioning, characterized in that it includes a mobile robot and a robot internal control host; the mechanical arm of the mobile robot is provided with a laser radar and a camera; the label is arranged on the cell on the tray of specimen slides; The lidar is used to perceive the surrounding environment of the mobile robot; The camera is used to capture the image of the tray; The internal control host of the robot is configured as: Obtain the coordinate information of the label in the image captured by the camera when the mobile robot is docked at the standard position, and calculate the standard position vector information according to the coordinate information; On the basis of perceiving the surrounding environment of the mobile robot, the SLAM algorithm is used to establish a global map, and the position of the mobile robot in the global map is determined, and an autonomous navigation route is established; According to the autonomous navigation route, control the mobile robot to move to the designated grasping position of the first stage; perform target recognition on the image captured by the camera at the designated grasping position, read the position coordinate information of the label in the image, and calculate the current position vector information; Calculate the correction amount of the world coordinate system according to the standard position vector information and the current position vector information; Use machine vision technology to detect the target of the image, determine the cell specimen sample in the tray, and determine the grabbing coordinates in combination with the position information of the tray groove where the cell specimen sample is located; Correcting the grasping coordinates by the correction amount, and controlling the mechanical arm to grasp the cell specimen sample at the target position according to the corrected grasping coordinates; Controlling the mobile robot to carry the cell sample sheet. 2. A cell sample sample handling system based on multi-label positioning according to claim 1, wherein the mechanical arm is also provided with a pneumatic gripper suction cup for grabbing the cell sample sample. 3. A cell specimen sample handling system based on multi-label positioning as claimed in claim 1, wherein the labels are two-dimensional code labels, three in number, respectively arranged at three corners of the tray place. 4. A cell specimen sample handling system based on multi-label positioning as claimed in claim 3, characterized in that, when the mobile robot is at a specified grabbing position, the camera takes pictures continuously, and performs machine vision processing on the images, Capture the two-dimensional code label information, and slowly control the body to rotate slowly in place through the characteristics of the differential wheel of the mobile robot to fine-tune the direction until three two-dimensional code labels are captured simultaneously in the camera field of view. 5. A cell sample sample handling system based on multi-label positioning as claimed in claim 1, wherein when the mobile robot stops at the standard position, it also obtains the storage recess of each cell sample sample on the tray. The grab coordinates of the slot and the pick and place height. 6. A cell specimen sample handling system based on multi-label positioning as claimed in claim 5, characterized in that, after determining the capture coordinates, the correction amount and offset angle between the capture coordinates and the world coordinate system are calculated; After determining the current grabbing coordinates, control the robotic arm to move at a speed of 1mm/s to the current grabbing coordinates in the two-dimensional plane above the pallet until it moves above the current grabbing coordinates, stop the horizontal movement, and drop vertically The Z-axis height of the mechanical arm reaches the grabbing height, and the grabbing action is performed. 7. A cell specimen sample handling system based on multi-label positioning as claimed in claim 6, characterized in that, after the grabbing action is performed, the end of the mechanical arm is moved to the photographing position to obtain the image of the tray. For the image information, the target detection of the cell specimen sample is carried out in the groove of the grasping position; if the cell specimen sample is not detected, it is judged that the cell specimen sample has been captured successfully, otherwise it is judged that the cell specimen sample has failed to be captured; if the cell specimen is analyzed If the capture of the sample piece fails, the capture operation will be performed again until the capture is successful. 8. A cell sample sample handling system based on multi-label positioning as claimed in claim 7, wherein after the cell sample sample is successfully captured, the mobile robot is controlled to move to the second carrier according to the autonomous navigation route. Specify the capture position in front of the object stage; perform target recognition on the image captured by the camera at the specified capture position, read the position coordinate information of the label in the image, and calculate the current position vector information; Calculate the correction amount of the world coordinate system according to the standard position vector information and the current position vector information; Use machine vision technology to detect the target of the image, determine the vacant groove in the tray, and determine the placement coordinates in combination with the position information of the tray groove where the cell specimen is located; The placement coordinates are corrected by the correction amount, and according to the corrected placement coordinates, the mechanical arm is controlled to place the cell sample piece into the target vacant groove. 9 . The multi-label localization-based cell sample sample handling system according to claim 1 , wherein the correction amount includes an offset and an offset angle between the standard position vector and the current position vector. 10 . 10. A cell sample sample handling method based on multi-label positioning, characterized in that the cell sample sample handling system based on multi-label positioning according to any one of claims 1-9 is adopted; the main contents include: Obtain the coordinate information of the label in the image at the standard position, and calculate the standard position vector information according to the coordinate information; Utilize the SLAM algorithm to establish a global map, and determine the position of the mobile robot in the global map, and establish an autonomous navigation route; According to the autonomous navigation route, control the mobile robot to move to the designated grab position; perform target recognition on the image taken at the designated grab position, read the position coordinate information of the label in the image, and calculate the current position vector information; Calculate the correction amount of the world coordinate system according to the standard position vector information and the current position vector information; Use machine vision technology to detect the target of the image, determine the cell specimen sample in the tray, and determine the grabbing coordinates in combination with the position information of the tray groove where the cell specimen sample is located; The grasping coordinates are corrected by the correction amount, and according to the corrected grasping coordinates, the mechanical arm is controlled to grasp the cell specimen sample at the target position, and the mobile robot is controlled to transport the cell specimen specimen.

Images