JP3417503B2 - Article position determination method and article position determination device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention picks up a plurality of articles which are arranged close to each other by an image pickup means, and determines the edge position on the picked-up image on the side where the article to be judged is adjacent to the article. The present invention relates to an article position determination method and an article position determination device for determining based on binarized image information obtained by binarizing image information captured by the image capturing unit based on brightness.

[0002]

2. Description of the Related Art When a plurality of articles arranged close to each other are imaged by an image pickup means, at the edge position on the side where the article to be judged is adjacent, the adjacent articles imaged in the vicinity of the article to be judged. Since the change in brightness is small due to the reflected light of the image, the image of the part to be accurately determined on the basis of the binarized image information at the edge position on the side where the adjacent article of the determination target article exists. It is necessary to accurately binarize the information based on an appropriate threshold value.

Therefore, in the conventional article position determination method, in consideration of the fact that the brightness of the determination target article changes depending on the distance from the light source, for example, the image information of the slit light projected on the article is based on the brightness. In the case of making a determination based on the binarized image information obtained by the binarization processing, the density distribution M of the slit light on the captured image indicates the density value of the entire captured image within a certain range as shown in FIG. Regions N1, N2
The threshold values M1, M2, which are appropriate for binarizing the image information of the areas N1 and N4, are divided for each of the areas N1 to N4.
M3 and M4 are set, and the image information of the entire captured image is binarized for each of the divided regions N1 to 4 based on the thresholds M1 to 4 set for that region. Based on the obtained binary image information of the entire captured image,
The edge position on the side where the article adjacent to the article to be determined on the captured image is present is determined.

[0004]

According to the above prior art,
The finer the entire captured image is divided into a number of regions, the more accurately the image information of the entire captured image can be binarized, and the edge position on the side where the adjacent article of the determination target article exists is binarized image information. However, in this case, it is necessary to set an appropriate threshold value for each of a large number of areas that are finely divided, which has a disadvantage that the threshold value setting procedure becomes complicated.

The present invention has been made in view of the above-mentioned circumstances, and by devising a picked-up image region in which a threshold value for binarizing image information is devised, the judgment can be made by a simple threshold value setting procedure. The image information of the edge position on the side where the adjacent article of the target article exists is accurately determined based on an appropriate threshold value.
The purpose is to enable value processing.

[0006]

According to a first characteristic configuration of the present invention for achieving the above object, a plurality of articles arranged close to each other are imaged by an image pickup means, and an article to be judged on a picked-up image. Is an article position determination method for determining the edge position on the side where adjacent articles exist on the basis of the binarized image information obtained by binarizing the image information captured by the image capturing means based on the brightness. Then, the appearance range on the captured image of the edge position on the side where the article adjacent to the determination target article is present is predicted, a threshold is set based on the image information of the predicted appearance range, and based on the set threshold. Then, the image information of the appearance range is binarized, and the edge position is determined based on the binarized image information obtained by the binarization.

According to a second characteristic configuration of the present invention, in the article position determination method according to the first characteristic configuration, the appearance range is a binary value of an edge position on the captured image on the side where no adjacent article exists. The point is that prediction is performed based on the converted image information.

A third characteristic configuration of the present invention is the article position determining method according to the first or second characteristic configuration, wherein the appearance range is
The point is that prediction is performed based on the binarized image information of the slit light projected onto the determination target article by the light projecting means on the captured image.

According to a fourth characteristic configuration of the present invention, a plurality of articles arranged close to each other are imaged by an image pickup means, and the edge position on the picked-up image on the side where the adjacent article of the article to be judged is present. An article position determination device that determines, based on binarized image information obtained by binarizing image information captured by the image capturing means based on brightness, on the side where an article adjacent to a determination target article exists. Prediction means for predicting the appearance range of the edge position of the image on the captured image, threshold setting means for setting a threshold value based on the image information of the predicted appearance range, and an image of the appearance range based on the set threshold value An image processing means for binarizing the information and a judging means for judging the edge position based on the binarized image information obtained by the binarizing processing are provided.

A fifth characteristic configuration of the present invention is that in the article position determination device according to the fourth characteristic configuration, the threshold value setting means is configured to set the threshold value by a discriminant analysis method.

A sixth characteristic configuration of the present invention is the article position determining apparatus according to the fourth or fifth characteristic configuration, wherein the predicting means is
It is configured to predict the appearance range based on the binarized image information of the edge position on the side of the captured image where the adjacent article does not exist.

According to a seventh characteristic configuration of the present invention, in the article position determination device according to the fourth, fifth or sixth characteristic configuration, the predicting unit determines the appearance range by a light projecting unit on a captured image. The point is that the prediction is performed based on the binarized image information of the slit light projected on the article.

[0013]

The operation of the first characteristic structure is as follows. Predict the appearance range on the captured image of the edge position on the side where the judgment target article is adjacent to the article, set a threshold value based on the image information of the predicted appearance range, and based on the set threshold value Binarize the image information of the appearance range,
Based on the binarized image information obtained by the binarization process, the edge position on the side where the article to be determined is adjacent to the article is determined.

The operation of the second characteristic configuration is as follows. The appearance range on the captured image of the edge position of the determination target article on the side where the adjacent article exists is used as the binarized image information of the edge position of the determination target article on the side where the adjacent article does not exist. Make a prediction based on

The operation of the third characteristic structure is as follows. The appearance range on the captured image of the edge position on the side of the determination target article on which the adjacent article exists is based on the binarized image information of the slit light projected onto the determination target article by the light projecting unit on the captured image. Predict.

The operation of the fourth characteristic structure is as follows. The predicting means predicts the appearance range on the captured image of the edge position on the side where the article adjacent to the determination target article exists,
The threshold value setting means sets a threshold value based on the image information of the predicted appearance range, and the image processing means binarizes the image information of the appearance range based on the set threshold value.
The recognition means determines the edge position on the side where the adjacent article of the article to be determined exists based on the binarized image information obtained by the binarization process.

The operation of the fifth characteristic structure is as follows. The threshold value setting means sets a threshold value by a discriminant analysis method. That is, when the set of the brightness (density value) of each pixel in the image information of the predicted appearance range is divided into two classes by the threshold value, the threshold value that gives the best separation between the two classes is obtained. .

The operation of the sixth characteristic structure is as follows. The predicting unit determines the appearance range on the captured image of the edge position on the side where the adjacent article of the determination target article exists on the captured image as a binary value of the edge position on the captured image on the side where the adjacent article does not exist. Predict based on the converted image information.

The operation of the seventh characteristic structure is as follows. The predicting means binarizes the appearance range of the edge position on the side where the article adjacent to the determination target article is present on the captured image, of the slit light projected onto the determination target article by the light projecting means on the captured image. Predict based on image information.

[0020]

According to the article position determination method of the first characteristic structure, the entire captured image is not finely divided into a large number of areas and the threshold value is set for each area, but the articles adjacent to the determination target article are The appearance range on the captured image of the edge position on the existing side is predicted, and a threshold value is set based on the image information of the predicted appearance range of the image information imaged by the imaging unit, and the set threshold value. Since the image information of the appearance range is binarized based on the
The image information of the edge position on the side where the article adjacent to the determination target article is present can be binarized with high accuracy based on an appropriate threshold value.

According to the article position determination method of the second characteristic configuration, in addition to the effect of the first characteristic configuration, the appearance range on the captured image of the edge position on the side where the article to be determined is adjacent to the article, Since the prediction is performed based on the binarized image information of the edge position on the side where there is no adjacent article of the article to be determined, in which the change in brightness on the captured image is large and the determination is easy, the appearance range is narrowed. It is possible to accurately predict in a limited state and easily set a more appropriate threshold value.

According to the article position determination method of the third characteristic configuration, in addition to the effect of the first or second characteristic configuration, the appearance of the edge position of the determination target article on the side where the adjacent article exists on the captured image. Since the range is predicted based on the binarized image information of the slit light projected onto the determination target article by the light projecting means on the captured image, the appearance range is limited to a narrow range along the slit light and the accuracy is improved. It is possible to predict well and easily set a more appropriate threshold.

According to the article position determination device of the fourth characteristic configuration, the entire captured image is not finely divided into a number of areas and the threshold value is set for each area, but the articles adjacent to the determination target article are present. Predicting the appearance range of the side edge position on the captured image, setting a threshold value based on the image information of the predicted appearance range of the image information imaged by the imaging means, and based on the set threshold value Then, the image information of the appearance range is binarized, so a simple threshold setting procedure
The image information of the edge position on the side where the article adjacent to the determination target article exists can be binarized with high accuracy based on an appropriate threshold, and the threshold setting operation can be automated.

According to the article position determination device of the fifth characteristic configuration, in addition to the effect of the fourth characteristic configuration, the threshold value is set by the discriminant analysis method, so that the image information of the predicted appearance range is binarized. You can set an appropriate threshold with.

According to the article position determination device of the sixth characteristic configuration, in addition to the effect of the fourth or fifth characteristic configuration, the edge position of the determination target article on the side where the adjacent article is present appears on the captured image. The range is predicted based on the binarized image information of the edge position on the side where there is no adjacent article of the article to be determined, which is easy to determine due to a large change in brightness on the captured image. A more appropriate threshold value can be easily set by accurately predicting in a state of being limited to a narrow range.

According to the article position determination device of the seventh characteristic constitution, in addition to the effect of the fourth, fifth or sixth characteristic constitution,
The appearance range on the captured image of the edge position on the side of the determination target article on which the adjacent article exists is based on the binarized image information of the slit light projected onto the determination target article by the light projecting unit on the captured image. Since the appearance range is limited to a narrow range along the slit light, the appearance range can be accurately predicted and a more appropriate threshold value can be easily set.

[0027]

1 shows an article transfer apparatus A for transferring a plurality of rectangular parallelepiped articles G to a conveyor 2 one by one, and this article transfer apparatus A is provided with an article position determination apparatus B according to the present invention. An article position detecting device C is provided, and each of the articles G is
The articles are aligned in a state where a space is provided between adjacent articles in the horizontal direction, and are placed in multiple stages on the pallet 1 set at a predetermined position.

The article transfer device A includes a base frame 3
With respect to the vertical axis [alpha], the swivel frame 4 is pivotally supported about the vertical axis [alpha], the upper part of the swivel frame 4 is pivotally supported about the horizontal axis [phi] 1 around the first arm 5, and (1) An arm 5 is provided with a second arm 6 pivotally supported about a horizontal axis φ2 at the tip of the arm 5, and the tip of the second arm 6 is rotatable about an axis β of the vertical axis. The tool 7 is hung up.

The article supporter 7 is supported so that it can be vertically moved and horizontally moved while being kept horizontal regardless of the swing of the first arm 5 and the second arm 6, and the article G on the pallet 1 is supported. A plurality of suction pads for suction-supporting the upper surface of the transfer target article GT, which is the article to be transferred, and a sandwiching section for sandwiching the article G suction-supported by the suction pad are provided.

As shown in FIG. 2, the article position detecting device C has two laser light sources as light projecting means for projecting three slit lights L1, L2, L3 onto the rectangular article upper surface J. Of the article upper surface J and the image capturing section 9 and the image capturing section 9 having a CCD camera as an image capturing means for capturing the slit light L1, L2, L3 projected on the article upper surface J. A distance sensor 10 for detecting a distance, and an imaging unit 9
And a controller 11 for controlling the operation of the article transfer device A based on the image information captured by the above. The light projecting units 8a and 8b, the imaging unit 9, and the distance sensor 10 are provided on the article support 7 side. Has been.

Each of the light projecting portions 8a and 8b is for projecting an apparent slit light on the article upper surface J by swinging a laser beam at a high speed along a plane inclined with respect to the horizontal. The light unit 8a projects one first slit light L1 along the width W direction of the article upper surface J in a state of protruding from the transfer target article GT, and the light projecting unit 8b projects the first slit light L1. The two second and third slit lights L2 and L3 parallel to each other in the orthogonal direction are configured to be projected in a state of protruding from the transfer target article GT.
Each of b is provided so that it can be switched between a light projecting state and a non-light projecting state.

As shown in FIG. 3, the controller 11 displays the first image information from the entire image captured by the image capturing section 9.
~ Image information of a region corresponding to the third slit light L1, L2, L3 is extracted, and the image information is extracted based on the brightness to be 2
Next to the transfer target article GT on the captured image, based on the binarized image information of the thinned first to third slit lights L1, L2, L3 The position detection unit 13 that detects the outer edge position P on the side where the matching article G does not exist, and the specific ranges F1 and F of the captured image
Based on the binarized image information obtained by binarizing the image information of 2 and F3 based on the brightness, the side of the side where the article G adjacent to the transfer target article (determination target article) GT exists on the captured image. A position determination unit B as an article position determination device that determines the inner edge position Q, and displacements Δx and Δy with respect to the reference posture G ₀ of the transfer target article GT whose inner edge position Q is determined by the position determination unit B. And a displacement detector 14 for detecting the rotation angle ΔΘ and the position detector 13 based on the detection information.
The displacements Δx and Δy detected by the displacement detector 14 and the rotation angle Δ
The operation of the article transfer device A is controlled based on Θ to transfer the transfer target article GT onto the conveyor 2.

The reference posture G ₀ is, as shown in FIG.
On the image captured by the image capturing unit 9, the width W direction is parallel to the x-axis direction on the captured image, the depth V direction is parallel to the y-axis direction on the captured image, and the center position of the article upper surface J is the captured image. The light projection units 8a and 8b and the imaging unit 9 are set at positions corresponding to the upper center position CP, and the center position CP on the captured image is set in the state where the first slit light L1 is projected onto the article upper surface J. At a predetermined position which is symmetrical with respect to the center position CP on the captured image in a state where the second slit light L2 and the third slit light L3 are projected onto the article upper surface J while passing through and imaged in parallel with the x-axis direction. It is arranged so as to be imaged parallel to the y-axis direction.

In the arrangement pattern of the articles G placed on the pallet 1, a peculiar pattern code is determined according to the size of the article G, and the pattern code and the width W and depth V of the article G are defined. When input to the controller 11, the controller 11 of the article transfer device A causes the articles G on the pallet 1 to be transferred onto the conveyor 2 from the uppermost one in a predetermined order corresponding to the pattern code. Control operation.

The position determination section B serves as a prediction means for predicting the appearance ranges F1, F2, F3 of the inner edge position Q on the captured image based on the detection information of the outer edge position P by the position detection section 13. Based on the prediction unit 15 and the predicted image information of the appearance ranges F1, F2, F3.
A threshold value setting unit 16 as a threshold value setting unit for setting 1, R2 and R3, and image processing for binarizing the image information of the appearance ranges F1, F2 and F3 based on the set threshold values R1 to R3. Image processing unit 17 as means, and 2
A determination unit 18 as a determination unit that determines the inner edge position Q based on the binarized image information obtained by the binarization process is provided.

The control operation of the controller 11 will be described with reference to the flowchart of FIG. First, the transfer target article GT among the articles G on the pallet 1 is determined based on the input pattern code, and the transfer target article G is determined.
The article support 7 is moved to a predetermined position corresponding to T, and the height of the article support 7 is adjusted so that the distance detected by the distance sensor 10 becomes a preset distance (step 1), and the projection unit The light projecting unit 8 is operated by operating 10 and the imaging unit 11.
a and 8b are articles G on the pallet 1 in a non-illuminated state,
The light projecting units 8a and 8b image each of the articles G on the pallet 1 in the light projecting state from a constant height (step 2), and the imaged image information is input to the extracting unit 12.

The extraction unit 12 has the image information of the picked-up image shown in FIG. 6A obtained by picking up the image in the non-illuminated state and the image information obtained in FIG. The image information of the captured image shown is subtracted, and the first to third images shown in FIG.
The image information of the area corresponding to the slit lights L1, L2, L3 is extracted (step 3), and this image information is binarized based on the threshold value S which is set to be constant in advance.
Binarized image information of the image in which the first to third slit lights L1, L2, L3 shown in (d) are thinned is obtained (steps 4 and 5).

In the binarized image information binarized based on the preset threshold value S, the first to third
Since the slit lights L1, L2, L3 are projected onto the article upper surface J from an oblique direction, the first to third slit lights L1, L2, L3 are parallel to each other at the outer edge position P of the transfer target article GT. Although the line is discontinuous at the position deviated to, the first to third slit lights L1, L2, L3 are continuously thin at the inner edge position Q of the transfer target article GT.

The position detector 13 detects the first slit light L1 which is parallel to the x-axis direction and passes through the center position CP ₀ on the picked-up image in the image information obtained by the extractor 12, and the picked-up image. Image information of the second and third slit lights L2, L3 parallel to the y-axis direction is extracted at predetermined positions symmetrical with respect to the upper center position CP ₀ , and as shown in FIG. Binarized image information of an image in which only the slit lights L1, L2, L3 projected on the upper surface J are thinned is extracted (step 6), and further, as shown in FIG. Point P1 having the maximum coordinate value of light L1 in the x-axis direction and point P having the minimum coordinate value
The points P5, P6 and the points P2, P3 having the maximum coordinate values in the y-axis direction of the fourth and second and third slit lights L2, L3 are selected (step 7), and the pallet 1, the conveyor 2 and the article transfer are selected. From the relative postures of the mounting devices A, as shown in FIG. 7G, the point P1 having the maximum coordinate value of the first slit light L1 and the coordinate value of the second slit light L2 having the minimum coordinate value among these points are the minimum. The point P2 and the point P3 having the smallest coordinate value of the third slit light L3 are detected as reference points indicating the outer edge position P of the transfer target article GT (step 8), and the detected reference points P1, P2, P
The coordinate value of 3 is output to the displacement detection unit 14 and the prediction unit 15 of the position determination unit B as detection information.

As shown in FIG. 7C, the predicting unit 15 is based on the detection information, and a minute appearance range in which the inner edge position Q is considered to appear on the captured image of FIG. 6C. F1, F2 and F3 are pre-input articles G
Of the estimated appearance range F1, F2, F3 in the captured image of FIG. 7C, the threshold value setting unit 16 predicts the calculated information based on the width W and the depth V (step 9). Based on the above, threshold values R1, R2, R3 are individually set for each of the appearance ranges F1, F2, F3 by the discriminant analysis method (step 10), and the image processing unit 17 determines the appearance ranges F1, F2, F3. The image information is binarized based on the set thresholds R1, R2, R3 (step 11), and the first to third slit lights L1, L2, L3 are thin lines within the appearance ranges F1, F2, F3. Figure 7
The binarized image information of the image shown in (i) is output to the determination unit 18. By the way, the discriminant analysis method uses the predicted appearance range F
For each of F1, F2, and F3 image information, when a set of brightness (density value) of each pixel is divided into two classes by a threshold value, a threshold value that gives the best separation between the two classes is obtained. Become. Specifically, the variance of the average value of the brightness of each pixel in each of the two classes (interclass variance)
And a threshold that maximizes the ratio of the variance of the brightness of each pixel (intra-class variance) in each class.

From the binarized image information obtained by the image processing unit 18, the judging unit 18 interrupts the first to third slit lights L1, L2, L3 within the appearance ranges F1, F2, F3. That is, the determination points Q1 and Q indicating the inner edge position Q
2, it is determined whether or not Q3 exists (step 1
2), as shown in FIG.
3 is determined to exist, the transfer target article GT
It is confirmed that the posture of is not the posture rotated by 90 ° from the reference posture G ₀ .

The displacement detection unit 14 uses the transfer target article GT.
If it is confirmed that the posture of No. is not the posture rotated by 90 degrees from the reference posture G ₀ , the reference points P1 and P shown in FIG.
2, P3 and the position coordinates of the determination points Q1, Q2, Q3, the displacement Δx of the transfer target article GT with respect to the reference posture G ₀ ,
The Δy and the rotation angle ΔΘ are detected (step 13).

Next, the detected displacements Δx and Δy and the rotation angle Δ
After correcting the position of the article support tool 7 with respect to the transfer target article GT based on Θ, the article support tool 7 is lowered, and the transfer target article GT is held by the suction pad and the sandwiching section. The article support tool 7 that holds the transfer target article GT is moved onto the conveyor 2 and lowered onto the conveyor 2, and the suction pad and the sandwiching section are switched to the holding release state.
One transfer operation is completed (step 14). The above operation is repeated until the transfer is completed.

[Other Embodiments] 1. The appearance range of the edge position on the side where the adjacent article of the determination target article is present on the captured image may be set in advance, for example, as a range surrounded by an L shape along the edge position. 2. The determination may be made based on the binarized image information obtained by binarizing the image information of the edge of the article itself. 3. The threshold value setting means may be configured to set the threshold value not only by the discriminant analysis method but also by, for example, the mode method.

It should be noted that although reference numerals are given in the claims for convenience of comparison with the drawings, the present invention is not limited to the configurations of the accompanying drawings by the entry.

[Brief description of drawings]

FIG. 1 is a side view of an article transfer device.

FIG. 2 is a schematic perspective view of an article position detection device.

FIG. 3 is a block diagram of an article position detection device.

FIG. 4 is an explanatory diagram of a reference posture.

FIG. 5: Flow chart

FIG. 6 is an explanatory diagram of image processing.

FIG. 7 is an explanatory diagram of image processing.

FIG. 8 is an explanatory diagram of a conventional technique.

[Explanation of symbols]

8a, 8b Projection means 9 Imaging means 15 Prediction means 16 threshold setting means 17 Image processing means 18 Judgment means F1, F2, F3 Appearance range G goods Goods subject to GT judgment L1, L2, L3 slit light P edge position Q edge position R1, R2, R3 threshold

Continuation of the front page (56) Reference JP-A-6-241719 (JP, A) JP-A-5-127722 (JP, A) JP-A-7-57080 (JP, A) JP-A-62-232504 (JP , A) JP 62-32582 (JP, A) JP 62-289701 (JP, A) (58) Fields investigated (Int.Cl. ⁷ , DB name) B25J 3/00-3/04 B25J 9/10-9/22 B25J 13/00-13/08 B25J 19/02-19/06 G01B 11/00-11/24 B23Q 17/24 G06T 7/00

Claims (7)

(57) [Claims] 1. A side in which a plurality of articles (G) arranged in close proximity to each other are imaged by an image pickup means (9) and an article (G) adjacent to a judgment target article (GT) is present on a captured image. Is an article position determining method for determining the edge position (Q) of the image information based on the binarized image information obtained by binarizing the image information captured by the image capturing means (9) based on the brightness. Appearance range (F) on the captured image of the edge position (Q) on the side where the article (G) adjacent to the determination target article (GT) exists
1, F2, F3), thresholds (R1, R2, R3) are set based on the image information of the predicted appearance range (F1, F2, F3), and the set thresholds (R1, R2, R3) are set. ), The image information of the appearance range (F1, F2, F3) is binarized, and the edge position (Q) is determined based on the binarized image information obtained by the binarization. Article position determination method for determination. 2. The appearance range (F1, F2, F3)
The article position determination method according to claim 1, wherein prediction is performed based on binarized image information of an edge position (P) on a side where the article (G) adjacent to the determination target article (GT) does not exist on the captured image. 3. The appearance range (F1, F2, F3)
The slit light (L1, L2, L) projected onto the determination target article (GT) by the light projecting means (8a, 8b) on the captured image.
3. The article position determination method according to claim 1, wherein the prediction is performed based on the binarized image information of 3). 4. A side on which a plurality of articles (G) arranged close to each other are picked up by an image pickup means (9) and an article (G) adjacent to a judgment target article (GT) is present on a picked-up image. Is an article position determination device that determines the edge position (Q) of the image information based on the binarized image information obtained by binarizing the image information captured by the image capturing means (9) based on the brightness. Appearance range (F) on the captured image of the edge position (Q) on the side where the article (G) adjacent to the determination target article (GT) exists
1, F2, F3) and a threshold setting means (16) for setting thresholds (R1, R2, R3) based on the predicted image information of the appearance range (F1, F2, F3). ), And image processing means (17) for binarizing the image information of the appearance range (F1, F2, F3) based on the set thresholds (R1, R2, R3), and the binarizing process. An article position determination device provided with a determination means (18) for determining the edge position (Q) based on the binarized image information obtained as described above. 5. The article position determination device according to claim 4, wherein the threshold value setting means (16) is configured to set the threshold values (R1, R2, R3) by a discriminant analysis method. 6. The predicting means (15) defines the appearance range (F1, F2, F3) as an edge position on the captured image on the side where the article (G) adjacent to the determination target article (GT) does not exist. The article position determination device according to claim 4 or 5, which is configured to make a prediction based on the binarized image information of (P). 7. The slit in which the predicting means (15) projects the appearance range (F1, F2, F3) onto the determination target article (GT) by the projecting means (8a, 8b) on the captured image. 7. The prediction is configured based on the binarized image information of light (L1, L2, L3).
The article position determination device described.