CN110866903A - Ping-pong ball identification method based on Hough circle transformation technology - Google Patents

Abstract

The invention discloses a table tennis identification method based on the Hough circle transform technology, which uses the gradient Hough transform method to improve the identification speed. It includes the following steps: 1) image grayscale processing; 2) filtering processing; 3) edge extraction; 4) table tennis ball marking; Compared with the prior art, the present invention can realize real-time identification of the table tennis balls contained in each frame of pictures for the pictures read in the camera, and can accurately identify the table tennis balls in complex situations.

Description

Table tennis recognition method based on Hough circle transform technology

technical field

The invention belongs to the technical field of robots, and in particular relates to a method for identifying a table tennis ball through the Hough circle transformation technology.

Background technique

In order to cooperate with the ping-pong ball picking robot and realize the function of automatic recognition of ping-pong balls by the ping-pong ball-picking robot, it is particularly important to design a method that can recognize and track table tennis balls in continuous images. Currently, most algorithms for identifying ping-pong balls are limited to a given image, from which ping-pong balls can be labeled. This method is easily disturbed by debris in the table tennis court with complex background, and cannot guarantee the accuracy of identifying table tennis balls when there is slight tearing and noise in the picture.

In view of the above problems, the present invention proposes a table tennis ball recognition method based on the Hough circle transform technology, which can cooperate with a table tennis ball picking robot to realize real-time recognition of table tennis balls on video. This algorithm can eliminate the interference of debris in the table tennis court with complex background, and still ensure the accuracy of identifying table tennis balls in the case of slight tearing and noise in the picture.

The principle of Hough transform method to identify circles is as follows:

Assuming that the image collected by the vision sensor with a resolution of M*N is f(x,y), the general equation of the circle can be expressed by the formula

(xa) ² +(yb) ² =r ²

to represent, where (a, b) is the center of the circle and r is the radius of the circle. The classic Hough transform converts the circle on the X-Y plane to the a-b-r parameter space, then all the three-dimensional cones in the parameter space corresponding to the points on the same circle in the image space must intersect at a point, and then the judgment result can be obtained by voting on this point. Whether it is a circle.

This method has high reliability and is less affected by noise, deformation and curve discontinuity, but it also has obvious defects: large memory usage and large amount of calculation, so it is difficult to meet the real-time detection of circles. At the same time, because the present invention is applied to the mobile robot platform, limited by hardware conditions, and has high requirements for real-time detection, the gradient Hough transform method is adopted in this example to improve the recognition speed.

SUMMARY OF THE INVENTION

In order to realize the function of automatically recognizing the ping-pong ball detected by the camera by the ping-pong ball picking robot, the patent of the present invention proposes a table-tennis ball recognition method based on the Hough circle transform technology. Mark it on the screen to realize the function of identifying table tennis.

The table tennis recognition method based on Hough circle transform technology adopts the gradient Hough transform method to improve the recognition speed. in polar coordinates

A=x-r cosθ, B=y-r sinθ (1)

to represent a circle, where x and y are the coordinates of the current pixel point, r is the radius, θ is the gradient direction angle, and A and B are the calculated possible circle center coordinates.

The table tennis identification method based on Hough transform technology includes the following steps:

Step 1. Image grayscale processing

The robot's processor can continuously obtain images of the front through the vision sensor, and then preprocess the read images. The image is grayed by the average method. Assuming that the three components of the color image are R(x,y), G(x,y), and B(x,y) respectively, after the grayscale processing, the grayscale value of the image Gray(x,y) can be expressed as:

Step 2. Filter processing

After the grayscale processing, the image is filtered to suppress noise, reduce environmental interference, and improve the edge definition of the target table tennis ball. The present invention adopts the mode of median filtering, sorts the data in the template from small to large, and takes the data in the middle position as the final processing result. The output after median filtering is:

g(x,y)=Med{f(x-k,y-l),(k,l)∈W} (3)

Among them, f(x,y) and g(x,y) are the original image and the processed image respectively, and W is a two-dimensional sliding window, which is generally selected from 3*3 and 5*5.

Step 3. Edge Extraction

The invention adopts the method based on the gradient Hough transform to find the center of the circle, and then realizes the extraction of the edge of the table tennis ball through the Canny algorithm. The steps of gradient Hough transform to find the coordinates of the circle center are:

计算原图像的梯度值，并将图像进行二值化。3.1 Using the gradient formula

Calculate the gradient value of the original image and binarize the image.

3.2 Find the gradient direction angle θ of the pixels whose gradient value is not 0 in the image, and then use the formula (1) to find the coordinates of the pixels that may be the center of the circle.

Step 3.3 traverse all the pixels in the image whose gradient value is not 0, find all the points that may be the center of the circle according to the method of step 3.2, and vote for each point.

3.4 After scanning the entire image, the point with the highest votes is the center of the circle to be detected.

Step 4. Ping Pong Ball Markers

For a certain frame of image in which a ping-pong ball is detected, take the upper left corner of the image as the origin and each pixel as a unit length, and mark the coordinates and radius of each circle in the image on the image. According to the physical rules of near big and far small, the table tennis ball closest to the robot is screened out, and the x, y coordinates and radius of the ball are passed to the processor for collection operation.

Step 5. Special processing of table tennis identification and setting of relevant parameters

5.1 Processing method of table tennis occlusion

In practical situations, it may happen that multiple ping pong balls block each other, or are too far or too close to the robot. Therefore, after Hough transform is performed, it is necessary to set parameters such as the minimum distance between the centers of the two circles, the threshold for determining whether the accumulated plane pair is a circle, and the maximum and minimum radius of the detected circles.

When adjusting the parameters, the present invention selects a black rubber field of 2m*2m; the entrance of the ball picking device is 15cm wide, the maximum resolution of the visual sensor is 640*480, the irradiation width is 70°, and the irradiation length is 50cm, which can identify the distance of the table tennis ball. The range is 20cm-70cm in front of the robot. Within this range, the maximum radius of the ping-pong ball that can be recognized is 80 pixels, and the minimum radius is 15 pixels. Since the collection success rate is not high when the table tennis ball is too far away from the robot, in this example, the maximum pixel value and the minimum pixel value of the radius of the detected circle are set to 80 and 30. Because the field used in the test in this example is a black solid-color rubber pad, the reflection is not serious, and the non-ping-pong object has little influence on the detection efficiency, so the judgment thresholds of whether the accumulated plane is a circle are set to 1, which is in a relatively low level. Normal moderate state. If the two ping-pong balls overlap too much, the camera may not recognize the captured ping-pong ball, and under this condition, the radius of the ping-pong ball is usually 80 pixels, so the minimum distance between the two circle centers is set to 120 pixels, does not affect the recognition effect, and can well limit the overlap of table tennis balls.

5.2 Parameter setting when working with the table tennis robot

In order to better cooperate with the ball pick-up device to complete the pick-up action and improve the pick-up efficiency, on this basis, it is required that the position where the table tennis ball enters the pick-up device is as centered as possible to avoid the table tennis ball touching the left and right edges of the pick-up device. And was pushed away from the robot, resulting in a situation where the ball could not be picked up. In this example, because the entrance of the ball picking device is 15cm, the pixels of the camera are 640*480, and the irradiation width is 70°, the irradiable range is larger than the entrance range of the ball picking device; the ball picking range is 20cm to 70cm in front of the robot, and the real straight line The picture illuminated by the camera is a trapezoid. In order to enable the table tennis balls with the farthest distance of 70cm to be successfully collected, after many experiments, it is finally determined that the table tennis ball center is located between 160 pixels and 480 pixels on the x-axis. collect. If the table tennis ball is recognized outside the collectible range, the robot is driven to adjust the posture accordingly, so that the center of the table tennis ball enters the collectible range and is collected.

5.3 Optimization Processing of Table Tennis Recognition

There are few interference factors in the actual table tennis field, but in order to ensure the accuracy of table tennis detection in a more complex environment, the picture can be selectively further processed to eliminate the influence of the background, such as corrosion , expansion, etc.

Erosion and expansion of grayscale images can be expressed by formulas (4) and (5) respectively:

fob=min{f _x -b(x),x∈D(b)} (4)

fgb=max{f _x +b(x),x∈D(b)} (5)

where D(b) is the template region of structural element b.

Open and close operations are used to achieve the purpose of improving the accuracy of identifying table tennis balls. The open operation is a process of first eroding and then expanding the image. It can be used to eliminate small black spots, separate objects at thin points, and smooth the boundaries of larger objects without significantly changing their area. The closed operation first expands and then erodes the image. process that can be used to exclude small black holes.

Description of drawings

Fig. 1 is the method flow chart of the present invention

Figure 2 is a grayscale image of a table tennis ball

Figure 3 is the median filter diagram after the table tennis ball is processed by the median filter

Figure 4 shows the edge extraction result of the Canny algorithm

Figure 5 is the effect diagram after modifying the double threshold

Figure 6 is an example diagram of table tennis marking

Figure 7 shows the detection results without using the opening and closing operation

Figure 8 shows the detection results after using the opening and closing operation

Detailed ways

The table tennis identification method based on the Hough transform technology, the flow chart of the specific implementation method is shown in Figure 1. Include the following steps:

Step 1. Image grayscale processing

In the example of the present invention, the resolution of the vision sensor is 640*480 pixels. Through the vision sensor, the robot's processor can continuously obtain the image in front of the vision sensor, and then preprocess the read image. First, grayscale processing is performed on the picture, and the present invention adopts the grayscale method of the average value. Assuming that the three components of the color image are R(x,y), G(x,y), and B(x,y) respectively, after the grayscale processing, the grayscale value of the image Gray(x,y) can be expressed as:

The principle is to average the brightness of the three components in the color image to obtain the gray value, that is, in the RGB model, calculate the average value of the three colors of R, G, and B of a pixel, and use this value to represent this value. The gray value of the pixel point, so that the color image is converted into a gray image, which reduces a certain amount of calculation.

Read the original image of the RGB three-color components of a table tennis ball. At this time, the color of each pixel is represented by three values of R(x,y), G(x,y), and B(x,y). Grayscale processing is performed on this image to obtain Figure 2. At this time, the color of each pixel is represented by only one value, Gray(x, y), which can reduce the computational complexity of image processing.

Step 2. Filter processing

After grayscale processing, considering that there is light interference in the actual scene, and the visual sensor captures, compresses the image, and brings noise during the transmission process, it is necessary to filter the image. The purpose is to suppress noise and reduce environmental interference. Improves the edge definition of the target ping pong ball. In the present invention, the method of median filtering is adopted, the data in the template is sorted from small to large, and the data in the middle position is taken as the final processing result. The output after median filtering is:

g(x,y)=Med{f(x-k,y-l),(k,l)∈W} (3)

Among them, f(x,y), g(x,y) are the original image and the processed image respectively, W is a two-dimensional sliding window, generally selected from 3*3, 5*5, in this example, 3* 3 sliding windows for filtering. It can be seen from Figure 3 that after the median filtering process, the noise on the image is much less, and the image is also much smoother, which is convenient for the next processing.

Step 3. Edge Extraction

Next, edge extraction can be performed on the ping-pong ball in the figure. In the example of the present invention, the method based on the gradient Hough transform is used to find the center of the circle, and then the edge of the table tennis ball is extracted by the Canny algorithm. The idea of gradient Hough transform to find the coordinates of the center of the circle is:

计算原图像的梯度值，并将图像进行二值化。3.1 Using the gradient formula

Calculate the gradient value of the original image and binarize the image.

3.2 Find the gradient direction angle θ of the pixels whose gradient value is not 0 in the image, and then use the formula (1) to find the coordinates of the pixels that may be the center of the circle.

3.3 Traverse all the pixels whose gradient value is not 0 in the image, find all the points that may be the center of the circle according to the method of step 3.2, and vote for each point.

3.4 After scanning the entire image, the point with the highest votes is the center of the circle to be detected.

It can be seen from Figure 4 that the rough outline of the ping pong ball has been extracted by the gradient Hough transform method and the edge extraction process by the Canny algorithm, but there are also blurred and discontinuous boundaries. In this example, after many tests, the high and low thresholds in the Canny algorithm are finally set to 100 and 30, respectively, and the edge extraction effect will be much improved as shown in Figure 5.

Step 4. Ping Pong Ball Markers

When a certain frame of ping-pong ball is detected, take the upper left corner of the image as the origin and each pixel as a unit length, and mark the coordinates and radius of each circle in the image on the image. Finally, according to the physical rules of near big and far small, the table tennis ball closest to the robot is screened out, and the x, y coordinates and radius of the ball are passed to the processor for collection operation.

Figure 6 shows an example of marking a ping pong ball. In the figure, the ping pong ball is circled with a circle, and the number on the ping pong ball represents the radius of the ping pong ball. The ping pong ball in the upper left is far from the camera, so the radius value is smaller than the ping pong ball in the lower right. In the actual situation, the ping pong ball in the lower right is close to the camera, and the robot will decide to collect this ping pong ball first.

Step 5. Special processing of table tennis identification and setting of relevant parameters

5.1 Processing method of table tennis occlusion

In practical situations, it may happen that multiple ping pong balls block each other, or are too far or too close to the robot. Therefore, after Hough transform is performed, it is necessary to set parameters such as the minimum distance between the centers of the two circles, the threshold for determining whether the accumulated plane pair is a circle, and the maximum and minimum radius of the detected circles.

When adjusting the parameters, the present invention selects a black rubber field of 2m*2m; the entrance of the ball picking device is 15cm wide, the maximum resolution of the visual sensor is 640*480, the irradiation width is 70°, and the irradiation length is 50cm, which can identify the distance of the table tennis ball. The range is 20cm-70cm in front of the robot. Within this range, the maximum radius of the ping-pong ball that can be recognized is 80 pixels, and the minimum radius is 15 pixels. Since the collection success rate is not high when the table tennis ball is too far away from the robot, in this example, the maximum pixel value and the minimum pixel value of the radius of the detected circle are set to 80 and 30. Because the field used in the test in this example is a black solid-color rubber pad, the reflection is not serious, and the non-ping-pong object has little influence on the detection efficiency, so the judgment thresholds of whether the accumulated plane is a circle are set to 1, which is in a relatively low level. Normal moderate state. If the two ping-pong balls overlap too much, the camera may not recognize the captured ping-pong ball, and under this condition, the radius of the ping-pong ball is usually 80 pixels, so the minimum distance between the two circle centers is set to 120 pixels, does not affect the recognition effect, and can well limit the overlap of table tennis balls.

5.2 Parameter setting when working with the table tennis robot

In order to better cooperate with the ball pick-up device to complete the pick-up action and improve the pick-up efficiency, on this basis, it is required that the position where the table tennis ball enters the pick-up device is as centered as possible to avoid the table tennis ball touching the left and right edges of the pick-up device. And was pushed away from the robot, resulting in a situation where the ball could not be picked up. In this example, because the entrance of the ball picking device is 15cm, the pixels of the camera are 640*480, and the irradiation width is 70°, the irradiable range is larger than the entrance range of the ball picking device; the ball picking range is 20cm to 70cm in front of the robot, and the real straight line The picture illuminated by the camera is a trapezoid. In order to enable the table tennis balls with the farthest distance of 70cm to be successfully collected, after many experiments, it is finally determined that the table tennis ball center is located between 160 pixels and 480 pixels on the x-axis. collect. If the table tennis ball is recognized outside the collectible range, the robot is driven to adjust the posture accordingly, so that the center of the table tennis ball enters the collectible range and is collected.

5.3 Optimization Processing of Table Tennis Recognition

There are few interference factors in the actual table tennis field, but in order to ensure the accuracy of table tennis detection in a more complex environment, the picture can be selectively further processed to eliminate the influence of the background, such as corrosion , expansion, etc.

Erosion and expansion of grayscale images can be expressed by formulas (4) and (5) respectively:

fob=min{f _x -b(x),x∈D(b)} (4)

fgb=max{f _x +b(x),x∈D(b)} (5)

where D(b) is the template region of structural element b.

In this example, the author uses the open and close operations to achieve the purpose of improving the accuracy of identifying table tennis. The open operation is a process of first eroding and then expanding the image. It can be used to eliminate small black spots, separate objects at thin points, and smooth the boundaries of larger objects without significantly changing their area. The closed operation first expands and then erodes the image. process that can be used to exclude small black holes.

Figure 7 shows the recognition without using open and close operations. It can be seen that although the method of Hough transform can identify ping pong balls in complex environments, there are also many misjudgments.

Figure 8 shows the recognition effect after using open and close operations. It can be seen that, compared with Fig. 8, all misjudgments are eliminated, and table tennis can still be stably recognized. Therefore, when the working environment of the robot is complex, this example eliminates the influence of complex background on the recognition of table tennis by adding open and close operations.

Claims (3)

1. The ping-pong ball identification method based on the Hough circle transformation technology is characterized in that: a gradient Hough transformation method is adopted to improve the identification speed; in polar equation

A＝x-rcosθ,B＝y-rsinθ (1)

Representing a circle, wherein x and y are coordinates of a current pixel point, r is a radius, theta is a gradient direction angle, and A and B are possible coordinates of a circle center obtained through calculation;

the method comprises the following steps:

step 1, image graying treatment

The processor of the robot can continuously obtain images in front through the vision sensor and then preprocess the read-in pictures; performing graying processing on the image by using an average value method, and after graying processing is performed on three components of the color image, namely R (x, y), G (x, y) and B (x, y), the Gray value Gray (x, y) of the image can be represented as follows:

step 2, filtering processing

After graying processing, filtering the image to inhibit noise, reduce environmental interference and improve the edge definition of the target table tennis ball; the invention adopts a median filtering mode to sort the data in the template from small to large, and takes the data arranged at the middle position as the final processing result; the median filtered output is:

g(x,y)＝Med{f(x-k,y-l),(k,l)∈W} (3)

wherein f (x, y) and g (x, y) are respectively an original image and a processed image, and W is a two-dimensional sliding window and is generally selected from 3 × 3 and 5 × 5;

step 3. edge extraction

The invention adopts a method based on gradient Hough transformation to find the circle center, and then extracts the edge of the table tennis through a Canny algorithm; the step of searching the coordinates of the circle center by gradient Hough transformation comprises the following steps:

3.1 Using gradient formula

Calculating the gradient value of the original image, and binarizing the image;

3.2 solving the gradient direction angle theta of a pixel point with gradient value not being 0 in the image, and then solving the coordinate of the pixel point which is possibly as the center of a circle by using the formula (1);

3.3, traversing all pixel points with gradient values not being 0 in the image, solving all points which can be the circle center according to the mode of the step 3.2, and voting each point;

3.4 after the whole image is scanned, the point with the highest ticket is the center of the circle to be detected;

step 4. Table tennis ball marking

For a certain frame of image with the detected ping-pong balls, marking the coordinates and the radius of each circle in the image on the image by taking the upper left corner of the image as an origin and each pixel as a unit length; screening out the table tennis ball closest to the robot according to the physical rules of big and small, transmitting the x and y coordinates and the radius of the table tennis ball to a processor, and performing collection operation;

step 5, special treatment of table tennis identification and setting of relevant parameters

5.1 Table tennis ball shielding condition processing method

After Hough transformation is carried out, parameters such as the minimum distance between two circle centers, a judgment threshold value of whether an accumulation plane is a circle or not, the maximum value and the minimum value of the radius of a detected circle and the like need to be set;

5.2 parameter setting in operation of table tennis ball picking robot

In the example, the entrance of the ball picking device is 15cm, the pixel of the camera is 640 x 480, the irradiation range is 70 degrees, and the irradiation range is larger than the entrance range of the ball picking device; the ball picking range is 20cm to 70cm in front of the robot, the picture irradiated by the real straight line in the camera is trapezoidal, and in order to enable the table tennis ball which is 70cm farthest to be successfully collected, after a plurality of experiments, the collection is finally determined to be carried out when the center of the table tennis ball is positioned between 160 pixels and 480 pixels on the x axis. If the table tennis balls are identified outside the collectable range, the robot is driven to perform corresponding posture adjustment, so that the centers of the table tennis balls enter the collectable range and are collected.

5.3 Table tennis identification optimization

The picture is optionally further processed to remove background effects such as corrosion, swelling, etc.

The erosion and expansion operations performed on the gray scale map can be expressed by the following equations (4) and (5), respectively:

fob＝min{f_x-b(x),x∈D(b)} (4)

fgb＝max{f_x+b(x),x∈D(b)} (5)

wherein, D (b) the template region of the structural element b;

the purpose of improving the accuracy rate of identifying the table tennis is realized by using opening and closing operations; the operation is a process of firstly corroding and then expanding the image, and is used for eliminating small black points, separating objects at fine points and smoothing the boundary of a larger object without obviously changing the area of the object; the closed operation is used for eliminating small black holes in the process of expanding the image firstly and then corroding the image.

2. The ping-pong ball identification method based on the Hough circle transformation technology as claimed in claim 1, wherein: in the step 5.1, a black rubber field of 2m by 2m is selected when parameters are adjusted; the width of the ball picking device inlet is 15cm, the maximum resolution of the visual sensor is 640 x 480, the irradiation width is 70 degrees, the irradiation length is 50cm, the distance range for identifying the table tennis is 20cm-70cm. in front of the robot, the maximum radius of the table tennis which can be identified is 80 pixels, and the minimum radius of the table tennis is 15 pixels; setting maximum and minimum pixel values of the radius of the detected circle to 80 and 30; the judgment threshold values of whether the accumulation plane is round or not are respectively set to be 1, and the accumulation plane is in a more conventional moderate state; when two table tennis balls are overlapped too much and the captured table tennis balls cannot be identified by the camera, the minimum distance between two circle centers is set to be 120 pixels according to the radius of the table tennis balls which is usually 80 pixels.

3. The ping-pong ball identification method based on the Hough circle transformation technology as claimed in claim 1, wherein: in the step 5.2, the position of the table tennis ball entering the ball picking device is as central as possible, so that the situation that the table tennis ball cannot be picked due to being pushed away from the robot when touching the left edge and the right edge of the ball picking device is avoided, the ball picking action is better finished, and the ball picking efficiency is improved; the entrance of the ball picking device is 15cm, the pixel of the camera is 640 x 480, the irradiation range is 70 degrees, and the irradiation range is larger than the entrance range of the ball picking device; the ball picking range is 20cm to 70cm in front of the robot, the picture irradiated by the real straight line in the camera is trapezoidal, and the center of a table tennis ball is collected when the center of the table tennis ball is positioned between 160 pixels and 480 pixels on the x axis, so that the table tennis ball at the farthest position of 70cm can be successfully collected; if the table tennis balls are identified outside the collectable range, the robot is driven to perform corresponding posture adjustment, so that the centers of the table tennis balls enter the collectable range and are collected.

Images