JPH09189513A - Method and apparatus for measuring center of gravity of marker - Google Patents

Abstract

(57) [Abstract] (Correction) [PROBLEMS] In a target center of gravity measuring method for recognizing a three-dimensional position of an article by using a target, a target having a marker shape that is easy to detect and the center of gravity is easy to detect and measure. Provide a way. SOLUTION: A window 71 having an appropriate size is set in a two-dimensional image 70, a figure appearing in the window 71 is binarized according to the lightness, and a figure 72 left after being arranged by expansion / compression processing and compression / expansion processing, 73 and 74 are classified and labeled, the information of each labeled graphic is taken into the arithmetic device, and circumscribed rectangles 75, 76 and 77 are obtained,
A rhombus is created by connecting the midpoints of the respective sides of the circumscribed rectangle obtained, and the original figure is overlaid as a template. Based on the degree of agreement between the two, it is determined whether the target figure is a rhombus or not. The center of gravity is determined and the center of the field of view is moved to the center of gravity position so that the device can capture and measure the object.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention detects a target marker attached to an article by using image processing technology and measures the position or dimension of the article based on the position of the center of gravity of the marker.
The present invention relates to a marker centroid measuring method for automatic tracking, and particularly to improvement of a shape of a target marker.

[0002]

2. Description of the Related Art When constructing a large structure, before constructing a structural frame or formwork manufactured at a factory at a construction site or assembling at a construction site, temporarily assemble it in the factory where it is manufactured to ensure accuracy in completion. I'm confirming. Structural skeletons, etc., which have been assembled in the factory and confirmed to be within the allowable size range, are disassembled once, transported to the site, and reassembled. This procedure cannot be omitted because it is necessary to confirm the quality by the factory side where it can be reworked and re-arranged, but since assembly and disassembly puts pressure on the cost and construction period, a method to omit it was desired.

By the way, if the surface dimensions of a predetermined part such as a structural body can be accurately measured, the shapes of the assembled parts can be accurately obtained with the aid of a computer using these measurement results, and the structure can be designed as designed. Whether or not it can be grasped with sufficient accuracy. The applicant of the present application has already disclosed in Japanese Patent Application No. 39198, 1995, for example, an apparatus and method for automatically measuring a predetermined measurement portion of a long plate girder or a box girder constituting a bridge to obtain a three-dimensional coordinate value. Is disclosed.

This method uses a CCD camera for image capturing with a zoom lens and an infrared pulse display type optical wave range finder, and sequentially images a target or a measurement target which is detachably attached to each part of the cross beam by the CCD camera according to a program. Image processing is performed to determine the direction of the target by directing it toward the center of the target, and the light wave range finder accurately measures the distance to the target to measure the three-dimensional position of the target. From this data, the surface shape of the girder can be accurately measured. An error from the design value can be calculated from the three-dimensional coordinate value of the predetermined portion of the measured member thus obtained, and the shape after assembly can be estimated to determine whether or not it falls within the design allowable range. Therefore, unlike the conventional method, the result can be determined without actually assembling, so that the process of assembling and disassembling in the factory can be omitted to reduce the manufacturing cost and shorten the construction period.

In order to automatically perform the above-mentioned measurement, it is necessary to accurately extract the measurement target in order and automatically aim at the center position of the measurement target. What kind of measurement target is used? Is a problem. In the method disclosed in Japanese Patent Application No. 39198 of 1995, a red ring-shaped display portion for capturing a color image on a white display plate surface as a measurement target, and a square-shaped display portion inside the ring-shaped display portion are used. It was composed of a retroreflective plate portion. The measurement using a color image with the marker in red is usually intended to prevent erroneous detection of the measurement target by adding a color that does not easily appear in the factory.

A color image of a member to be measured including a measurement target is captured by a CCD camera, image processing is performed only for red within an image processing range depending on the distance to the measurement target, and the position of the center of gravity thereof is obtained. With this, an accurate distance is measured by aiming so that the optical axis of the light wave range finder coincides with the center of the measurement target.

The apparatus for accurately obtaining the three-dimensional coordinate value of the surface portion of the object to be measured is not only for avoiding the disassembly and disassembly for the pre-inspection of the constituent members of the building as described above, but also for the machine / apparatus in the factory. In order to enable automatic transfer, automatic access, automatic processing, etc. by the machine, the machine / device can automatically recognize the target of the work and approach the target or grasp and process it. It can also be used when making it possible. In this case as well, the shapes of objects targeted by machines / devices are diverse, and in order to automatically recognize them, it is necessary to have an extremely high level of image recognition ability. There is used a method of attaching to an object and recognizing the position, size, and orientation of the article by relying on the characteristics of the measurement target.

In general, a measurement target is used to
In the method of recognizing a dimensional position, in order to recognize a measurement target by image processing and automatically track an article, firstly, detection of the measurement target and secondly, measurement of the center of gravity of the measurement target are required. Conventionally, a method has been often used in which a simple circular marker or a square marker is used as a measurement target, an image captured by a CCD camera or the like is subjected to image processing to extract a target marker, and the barycentric position of the extracted marker is calculated. . In the method of extracting a marker by such image processing, it becomes difficult to detect when something other than the measurement target enters the field of view, so that unnecessary objects are not captured in the background, and advanced image processing technology is used. However, since there are considerable restrictions, the applicable cases were limited.

When a target other than the measurement target is included in the visual field, a method of detecting the marker by a pattern matching method using a marker having a shape easily distinguishable from the target is also used. When the pattern matching method is used, the shape of the marker attached to the pattern measurement point is preferably a circle that is easy to match and relatively easy to calculate the center of gravity.

However, in the factory, there are many circular holes such as bolt holes, round bars, and pipe sections. Therefore, erroneous detection of the marker will occur frequently. If the shape of the marker is made complicated in order to prevent erroneous detection, the burden of image processing will increase. Further, if the marker shape is complicated, the shape recognition may fail and the detection itself may not be performed when the distance to the measurement target or the viewing angle changes.

[0011]

Therefore, the problem to be solved by the present invention is that it is unlikely to be confused with various shapes in a factory, is not affected by the distance and direction, and is easy to detect and to easily measure the center of gravity. It is an object to provide a measurement target having a marker shape and a detection / measurement method thereof.

[0012]

In order to solve the above-mentioned problems, the marker centroid measuring method of the present invention represents a parallelogrammatic marker in which two diagonals are substantially vertical and horizontal, that is, a substantially rhombic marker. Attach a measurement target to the item,
The position of the article is measured by picking up an image including this, extracting the marker represented on the measurement target by an image processing technique, and obtaining the position of the center of gravity of the marker. As a method of attaching the measurement target to the article, various methods such as a method of directly coating the article itself or a member specially provided for the article, a method of fixing with a magnet, a method of attaching with an adhesive can be used.

Further, according to the second marker centroid measuring method of the present invention, a circumscribing rectangle which is placed substantially horizontally with respect to a figure in an image photographed by the image pickup device is calculated, and a midpoint of each side of the circumscribing rectangle is taken. It is characterized in that the degree of matching between the quadrangle connecting these midpoints and the preceding figure is calculated and the one with the best match is used as the target marker.

The second marker centroid measuring method of the present invention is further characterized in that the centroid position of the marker is calculated from the horizontal most frequent position and the most vertical position of the pixels belonging to the image of the target marker. May be

Further, in the marker centroid measuring method according to the third aspect of the present invention, a window is set in the image taken by the image pickup device, the image in the window is binarized based on the brightness, and a binarized image is obtained. The binarized image is shaped by the expansion / compression process and / or the compression / expansion process, the figure is extracted and labeled, and the circumscribed rectangle placed approximately horizontally on the labeled figure is calculated, and the midpoint of each side of the circumscribed rectangle is calculated. After determining that the quadrangle as the apex and the labeled figure best match the target marker, the marker is determined from the horizontal most frequent position and the most vertical position of the pixels belonging to the image of the target marker. It is characterized in that the position of the center of gravity is calculated.

In the second and third marker centroid measuring methods, it is preferable that a measurement target representing a substantially diamond-shaped marker is attached to the article and an image including the measurement target is acquired. .

In order to solve the above-mentioned problems, a marker centroid measuring device of the present invention comprises an image pickup device, a monitor, an image input / output device, an image processing device and an arithmetic device, and the image pickup device picks up an image including a marker. An analog output is performed, and the image input / output device inputs the analog output of the image pickup device and stores it as a digitized two-dimensional image signal. The two-dimensional image signal is sent to the image processing device and the arithmetic device and processed in the image processing device. The image processing apparatus inputs and stores the stored image signal, the image processing apparatus inputs the two-dimensional image signal from the image input / output apparatus, performs image processing, extracts a figure in the image, labels and stores the image signal, and stores the stored image signal. Output to the image input / output device and arithmetic unit,
The arithmetic device processes the labeled image signal to generate a circumscribing rectangle with one side horizontal for each figure in the image, and calculates a quadrangle connecting the midpoints of the sides of the circumscribing rectangle to calculate the image signal. The target target is determined based on the degree of coincidence with the represented marker, and the monitor inputs the video signal stored in the image input / output device to display the image.

Further, the arithmetic unit further calculates a frequency distribution in the horizontal axis direction and the vertical axis direction for the signal state of the image signal representing the marker determined to be the target target, and determines the center of gravity of the marker with the position having the highest frequency as a reference. It may be calculated and moved so that the marker centroid position is located at the center of the image.

In the marker center-of-gravity measuring apparatus, it is preferable that a measurement target representing a diamond-shaped marker is attached to an article and an image including the measurement target is acquired. It is more preferable that the marker center-of-gravity measuring device further includes an illuminating device such that the optical axis is close to the optical axis of the imaging device, and the diamond-shaped marker has a retroreflective function.
With this configuration, the image including the measurement target can be acquired by the imaging device while irradiating the marker with the illumination device.

Conventionally, the shapes that are often used as markers are simple circles and squares, but these are very generally used in the factory such as the outer shape of workpieces, hole shapes, pipe cross sections, or shelves, drawers, tool boxes, etc. Since the shape is visible, when used as a measurement target, it is likely to appear in the background of the image and cause erroneous detection.

However, even though the shape is similarly simple, the rhombus is an uncommon shape. Therefore, if a measurement target using a rhombus marker is used, it is difficult for a similar shape to appear in the background, and the measurement target is The probability of false detection is significantly reduced.

Further, if a measurement target representing a rhombus marker is used, pattern matching is easy, and a similar shape of the rhombus is determined as a template.
Accurate determination is possible without being affected by the relative position change with the measurement target.

In particular, by taking circumscribing rectangles placed horizontally for the figures on the screen and connecting the midpoints of the sides of the rectangle to connect them, it is possible to easily obtain a rhombus to be a template. By comparing the figure that is the source of the circumscribed rectangle with this template rhombus and calculating the degree of matching, it is possible to easily determine whether or not the original figure is the rhombus.
If the figure is a rhombus, it is highly possible that the figure is not a thing existing at the factory site from the beginning, but is an object artificially attached to the article as a measurement target.

Further, it is easy to find the position of the center of gravity of a diamond marker figure obtained by binarizing an image with appropriate brightness as a threshold value. That is, when the value of the diamond marker portion is added to the horizontal direction and projected on the vertical axis, the place having the maximum value represents the vertical position of the center of gravity of the diamond marker,
The position of the maximum value when projected on the horizontal axis in addition to the vertical direction represents the horizontal position of the center of gravity of the diamond marker. Therefore, the barycentric position of the diamond marker can be easily given by the vertical coordinate of the maximum value and the horizontal coordinate of the maximum value.

However, if there is a measurement error, the horizontal coordinates of the barycentric position are calculated by adding the horizontal coordinate values of all the pixels in the target graphic and then dividing by the number of pixels to obtain an average value. By taking the average value of the vertical coordinate values of each pixel as the value and setting it as the vertical coordinate value of the position of the center of gravity, the influence of the error can be mitigated. The above method can be applied regardless of whether the marker is a circle or a square, but if it is a diamond, the accuracy of detecting the center of gravity is significantly improved.

[0026]

BEST MODE FOR CARRYING OUT THE INVENTION The marker centroid measuring method and apparatus of the present invention will be described in detail with reference to Examples. FIG. 1 is a drawing explaining an algorithm in the marker centroid measuring method of the present invention. The image is binarized using an appropriate threshold value for brightness, and then subjected to appropriate image processing such as compression / expansion processing, and labeling is performed for each group of figures. Then, the circumscribed rectangle is calculated for each independent figure. The sides of this rectangle are formed by horizontal and vertical lines.

In FIG. 1A, 1 is an image of a rhombic marker labeled after binarization. 2 is marker 1
Is a circumscribed rectangle. The midpoints of the sides of the circumscribed rectangle 2 are taken to connect them to form a quadrangle 3. This quadrilateral 3 is a diamond and is used as a template. In the marker 1, each pixel in a portion inside the template quadrangle 3 is attached with 1 and pixel in an outside portion is attached with 0. The area of the portion marked with 0, that is, the number of pixels, is compared with the area of the portion marked with 1, that is, the number of pixels. For example, if the diamond-shaped marker 1 has an exact diamond shape and two diagonals are placed so as to be exactly horizontal and vertical, the template quadrangle 2 and the diamond-shaped marker 1 completely match each other, so that they are included in the image. 0 does not appear. Therefore, it can be seen that the measurement target having the diamond-shaped marker 1 is the target.

When the measurement target or the camera is slightly deviated from the correct posture, the area of 0 appears slightly, but it is smaller than the area of 1. Therefore, when the ratio of the region of 0 to the region of 1 does not exceed the appropriately determined threshold value, it is determined as the measurement target, and a more practical determination can be made.

Further, in order to be able to deal with various situations, a portion of the labeled figure that extends inside the circumscribed rectangle 2 and outside the template quadrilateral 3 and the inside of the template quadrilateral 3. A value obtained by normalizing the area including the area in which the labeled figure does not exist by the area of the circumscribed rectangle 2 can be used as an index of the degree of matching between the figure and the diamond. With the index calculated in this way, it is possible to make a correct decision because the relative degree of matching is required even if the figure image is small or the orientation is displaced due to a large distance to the measurement target. Is.

For example, as shown in FIG. 1B, the circumscribed rectangle 2 is formed in the same manner as above even when the target figure is the triangle 4, but the midpoint of each side of the circumscribed rectangle 2 of the target figure. It can be seen that the target figure is not a rhombus and is therefore not a measurement target because the portion of the template quadrilateral 3 formed by connecting the two is relatively large.

Also, as shown in FIG. 1C, when the target figure is a slightly distorted square 5, the area in the external region of the template quadrilateral 3 becomes large and the target figure becomes the measurement target. It turns out that not.

Further, as shown in FIG. 1D, even when the target figure is a circle 6, a portion of the template quadrilateral 3 in the outer region exceeds a predetermined threshold with respect to the area of the circumscribed rectangle 2. Therefore, it can be seen that the target figure is not the measurement target. The relationship between the part in the outer area and the part in the inner area of the template quadrilateral 3 is not so much influenced by the positional relationship between the measurement target and the camera.

FIG. 2 is a diagram for explaining how the image of the measurement target changes depending on the positions of the measurement target and the camera. FIG. 2A is an image when the measurement target is at a short distance. Image 7 of the measurement target is a CCD
It is acquired as a diamond-shaped figure enlarged by the camera. However, the circumscribed rectangle 2 of the measurement target image 7 is taken, and the midpoints of the sides of the rectangle 2 are connected to form the template quadrangle 3.
When this is formed, it can be seen that this template quadrangle 3 coincides with the measurement target image 7 and has almost no region of 0 described above, and is a target rhombic marker without any difficulty.

On the other hand, FIG. 2 (b) is an image when the measurement target is at a distance. The image 8 of the measurement target has a small diamond shape. This is also the measurement target image 8
When the circumscribed rectangle 2 is taken and the midpoints of the sides are connected to form a template quadrangle 3, the template quadrangle 3 coincides with the measurement target image 8, and it can be seen that it is a target rhomboid marker. Thus, there is no difference in the determination depending on the distance between the measurement target and the camera.

FIG. 2 (c) is an image when the measurement target is positioned diagonally with respect to the camera.
The image 9 of the measurement target is acquired by a CCD camera as a vertically elongated rhombic figure. However, when the circumscribed rectangle 2 of the measurement target image 9 is taken and the midpoints of the sides of the rectangle 2 are connected to form the template quadrangle 3, the template quadrangle 3 matches the measurement target image 7, so what It can be seen that the target is a diamond marker without difficulty.

On the other hand, FIG. 2 (d) is an image when the measurement target is positioned obliquely with respect to the camera. The image 10 of the measurement target is a horizontally long rhombus figure. Also, if the circumscribed rectangle 2 of the measurement target image 10 is taken and the midpoint of each side is connected to form a template quadrilateral 3, this template quadrilateral 3 matches the measurement target image 10 and is therefore a target rhombus marker. I know there is. In this way, the correct determination can be made even if the camera does not face the front of the measurement target.

FIG. 3 is a drawing for explaining the advantages of the method for measuring the position of the center of gravity of a marker in the method for measuring the center of gravity of a marker according to the present invention. FIG. 3A is a view showing the frequency distribution when the values are integrated in the vertical axis direction with respect to a rhombic figure in which pixels have a value of 1 when binarized. When the orientation of the diamond-shaped figure is correctly oriented vertically, the frequency distribution 11 of pixel values has a sharp peak P.
Becomes a triangle with. The center of gravity of the diamond-shaped figure exists at the coordinate position on the horizontal axis of this peak.

If the same operation is performed in the horizontal axis direction, the peak position of the obtained frequency distribution is the vertical coordinate position of the center of gravity of the rhombus, so that the intersection of the peak positions, that is, the vertical coordinate and horizontal coordinate of the center of gravity. The position represented by can be determined as the position of the center of gravity. Actually, in consideration of various noises appearing in the image, the average value obtained by adding all the coordinate values of each pixel having a value of 1 after binarization as described above and dividing by the number of pixels is the above coordinate value. Used as. The above averaging has the advantage that the statistical noise is canceled out, and the effect of sudden noise is eliminated, and the obtained value becomes closer to the true value.

On the other hand, when the target marker is a square having horizontal sides, the shape of the frequency distribution of pixels with a value of 1 is a quadrangle 12 with a flat upper side, as shown in FIG. Since it does not appear, the method of finding the center of gravity from the peak position of the frequency distribution as described above cannot be applied. Also,
In the case of a circular shape, as shown in FIG. 3 (c), the shape becomes a semi-elliptical circle 13 having a gentle arc, and the accuracy of the detection position cannot be improved by analyzing such a distribution shape to determine the position of the center of gravity.

FIG. 4 is a diagram showing how the barycentric position of the marker is measured using the above principle. The frequency distribution 14 having the peak Px is obtained by integrating the pixel values in the vertical axis direction for the graphic of the marker 1 in which the pixel value is 1 and the pixel value is 1. When the same operation is performed in the horizontal axis direction, the frequency distribution 15 having the peak Py is obtained, and the coordinate O (Px,
It can be seen that Py) is the coordinates of the barycentric position of the marker 1. The fact that the position of the center of gravity can be accurately determined by the simple method as described above is an advantage that occurs when the diamond-shaped marker is used.

As described above, according to the marker center-of-gravity measuring method of the present invention using the diamond-shaped marker, the number of confusing figures in the environment can be reduced and accurate determination can be made with a simple method, thereby reducing the erroneous detection of the marker and reducing the center position. It is used in the factory such as automatic tracking of the measurement target because the detection accuracy of the marker center of gravity is improved because it detects by the method that a sharp peak appears, and there is no need to change the processing method even if the positional relationship between the marker and the camera changes. It is extremely effective in doing this.

The method and apparatus for measuring the center of gravity of the marker of the present invention will be described more specifically below with reference to examples.

[0043]

FIG. 5 is a block diagram showing the configuration of an embodiment of the marker centroid measuring apparatus of the present invention. FIG. 6 is a diagram for explaining the function of the positioning device of the imaging device used in the marker centroid measuring device. FIG. 7 is a drawing for explaining stepwise an image processing method implemented by the marker centroid measuring device. In the figure, 21 is a CCD camera as an imaging device, 22 is a monitor, 30 is an image input / output device, 40 is an image processing device, 50 is an arithmetic device, and 60 is a positioning device for the CCD camera 21. The image input / output device 30 has an analog input unit 31, an analog output unit 32, a storage unit 33, a digital input unit 34, and a digital output unit 35.
The image processing device 40 also includes a digital output unit 41, a digital input unit 42, a storage unit 43, and a calculation unit 44. Further, the arithmetic unit 50 includes an arithmetic unit 51 and a storage unit 52.

According to the marker centroid measuring apparatus of this embodiment,
A measurement target having a diamond-shaped mark is previously attached to an object on which the mechanical device is to act, and an image acquired by the CCD camera 21 is captured via the image input / output device 30, and the image processing device 40 and the arithmetic device are included. By the action of 50, the existence of the target object is recognized by extracting the diamond mark of the measurement target included in the image, and the orientation of the target object can be known by measuring the center of gravity of the extracted diamond mark. If the position of the center of gravity of the diamond-shaped mark 1 thus obtained is set to the center of the visual field of the camera 21, the orientation of the image pickup device 21 and the mechanical device itself can be automatically adjusted to the object through the positioning device 60. You can also

First, the two-dimensional image obtained by the CCD camera 21 for each pixel is sent to the analog input section 31 of the image input / output device 30, converted into a digital signal, and stored in the storage section 33 as a two-dimensional image signal. Is stored in. Monitor 22
The image information stored as a digital signal in the storage unit 33 of the image input / output device 30 can be converted into an analog signal by the analog output unit 32 and displayed. Therefore, CC
The two-dimensional image acquired by the D camera 21 and stored in the storage unit 33 can be displayed on the monitor 22 via the analog output unit 32. In this way, the operator
Through 2, it is possible to observe a raw image taken by the CCD camera 21.

The two-dimensional image signal stored in the writing section 33 of the image input / output apparatus 30 is sent to the image processing apparatus 40 via the digital output section 35 of the image input / output apparatus 30. The image processing device 40 receives the two-dimensional image signal via the digital input unit 42, temporarily stores the two-dimensional image signal in the storage unit 43, reads a necessary portion, and performs image processing.

FIG. 6A is a diagram showing an image in the first stage of image processing. As shown in FIG. 6A, the first step of image processing is to set a window 71 of an appropriate size in the two-dimensional image 70 acquired by the CCD camera 21. By limiting the image processing to the inside of the window 71, the required arithmetic processing resources can be of a proper scale, and the unnecessary image noise at the peripheral portion of the image can be prevented.

When the arrangement of the measurement targets to be measured is determined and the measurement order is determined, the measuring device knows the distance to the measurement target 1 in advance, and the size of the measurement target 1 and the distance to the measurement target. Accordingly, the size of the image of the measurement target on the screen can be predicted, and thus the appropriate size of the window 71 can be determined based on the size of the predicted measurement target image.
When there are a plurality of measurement targets in the field of view, it is preferable to perform processing so that the window 71 includes only one target measurement target. For this purpose, it is also necessary to properly arrange the measurement target.

FIG. 6B is a drawing showing an image in the second stage of image processing. In the second stage of the image processing, as shown in FIG. 6B, the images in the window 71 are organized by a relatively simple image processing method to narrow down the targets to be subjected to the complicated processing. 2 according to the brightness first
Value. The marker is formed of a reflection mark using a micro-corner cube reflector or a small glass sphere, and reflects the incident illumination light in the incident direction without much attenuation. Therefore, by providing a lighting device near the CCD camera,
When a portion including the marker is imaged while illuminating from a position close to the optical axis of the CCD camera, the reflection mark appears brighter than other portions in the image. C under such lighting
By binarizing the image acquired by the CD camera using an appropriate threshold value, the target marker can be clearly separated from the image of an article with low brightness existing in the surroundings.

However, when an object with high brightness is present in the surroundings or strong reflected light is generated at the corners of the structure or the like, spot-like or linear spots other than the target marker remain in the binarized image. It becomes noise. Further, if the brightness of the reflection mark is uneven, a part of the figure may be missing due to the binarization process. The missing portion existing in the figure can be filled and restored by expansion and compression processing. That is, when a figure is expanded by a certain width, a small recess existing in a hole or a contour inside the figure is filled, and if it is compressed by the same amount after that, the figure becomes a figure close to the original shape in which the chipped portion is filled.
On the other hand, small light spots and thin line-shaped figures that remain even after binarization can be removed by compression and expansion processing. That is, if a small light spot is eliminated by a compression process that reduces a certain width and then the same width is expanded, the original state is restored except for small light spots and lines, and optical noise is eventually erased.

Thus, the figure 7 remaining in the window 71
The independence of figures is determined based on the binary information of 2, 73, and 74, and the figures are labeled for each figure.
The storage unit 43 of the image processing apparatus 40 as the dimensional label data.
Image memory.

FIG. 6C is a diagram showing an image in the third stage of image processing. In the third stage of the image processing, as shown in FIG. 6C, the arithmetic unit 50 fetches the information of each labeled graphic from the storage unit 43 and circumscribes a rectangle 75 of each graphic 72, 73, 74. 76, 77
Ask for. A circumscribed rectangle consists of horizontal and vertical sides. If the CCD camera 21 is installed horizontally, the horizontal axis direction and the vertical axis direction of the two-dimensional label data correspond to the horizontal direction and the vertical direction as they are. Therefore, the circumscribed rectangle is the minimum and maximum values in the horizontal axis direction for the same label,
Since the figure is surrounded by the minimum value and the maximum value in the vertical axis direction, it can be obtained by a simple calculation.

Next, a quadrangle connecting the midpoints of the sides of the circumscribed rectangle obtained above is created. Since this quadrangle is made by connecting the midpoints of the sides of the rectangle, it has a diamond shape. If you overlap the original figure using this quadrangle as a template, if the figure is a rhombus whose axis is vertical, both will match without much
It can be easily determined. If it is not a rhombus, a portion that protrudes into the external region of the quadrilateral used as the template occurs. It is possible to determine whether or not the target graphic is a rhombus based on the area of the portion protruding to the external region. When there are a plurality of figures 72, 73, 74, the figure 72 having a smaller protruding portion is determined as the measurement target.

However, for example, if the original figure is large, the area of the protruding portion becomes relatively large, so it may not be appropriate to immediately judge the degree to which the figure approximates a rhombus from the absolute value. Therefore, in order to prevent the blurring of the judgment due to the size and orientation of the figure, calculate the area of the part that protrudes in the external area of the template quadrilateral and the area that overlaps the internal area, and compare the two Can be determined to be more likely to be the measurement target.

FIG. 6D is a diagram showing an image in the fourth stage of image processing. In the fourth stage of the image processing, as shown in FIG. 6D, the center of gravity of the figure 72 determined as the measurement target is obtained, and the positioning device 60 is driven to move the center of the visual field of the camera to the center of gravity position. , Machines and devices should be able to capture and work with objects.

The center of gravity of the diamond-shaped target mark 72 is the abscissa of the peak position of the frequency distribution 78 obtained by accumulating the values in the vertical axis direction and projecting them on the horizontal axis for the binarized data of the shaped shape. The coordinate value (Px, Py) is obtained from Px and the ordinate Py of the peak position of the frequency distribution 79 obtained by integrating the values in the horizontal axis direction and projecting them on the vertical axis.
Therefore, the optical axis of the CCD camera 21 is adjusted to adjust the coordinate value (P
By setting (x, Py) at the center of the image 80, the posture of the machine / apparatus corresponds to the article to which the measurement target is attached.

FIG. 7 is a diagram showing a configuration for aligning the optical axis of the image pickup device 21 with the center position of the diamond-shaped marker obtained by the arithmetic device 50. The image pickup device such as the CCD camera 21 can be rotated around the two axes of the vertical axis and the horizontal axis by the positioning device 60. By this rotation, the horizontal shift amount and the vertical shift amount of the marker image in the visual field are compensated, the visual field center is adjusted to the marker center, and the optical axis is directed in the direction of the marker 1. It represents the direction component of the marker.

FIG. 8 is a flow chart showing a control method of the positioning device 60. When the device starts operating (S1), C
The image processing device 40 and the arithmetic device 50 cooperate with each other to perform image processing on the image acquired by the CD camera 21 and stored in the image input / output device 30, and the center of gravity of the marker is detected based on the result (S2). Calculate the shift amount and the vertical shift amount (S
3) If any of these values is larger than a predetermined threshold value (S4), the horizontal axis or the vertical axis is driven by an amount proportional to the amount of deviation to compensate (S5), and then the process returns to S2 again to display an image. input. If both the horizontal shift amount and the vertical shift amount are smaller than the prescribed threshold value, the image pickup device 21 has correctly oriented its optical axis to the center of the marker, and the operation is ended (S6).

The CCD camera 21 may have a zoom function and can photograph a long-distance object in a large size. Further, it may be possible to avoid the difficulty of imaging due to the measurement target position by configuring the entire imaging device so that it can be moved vertically or horizontally. Further, when the light wave range finder is arranged coaxially with the CCD camera 21, the optical axis of the light wave range finder is automatically aligned with the center of the marker by the above adjustment, so that an accurate distance can be measured. The three-dimensional coordinates of the marker 1 can be obtained.

It should be noted that the image at each stage in these image processing operations is performed from the storage section 43 of the image processing apparatus 40 via the digital output section 41 and the digital input section 34 of the image input / output apparatus 30, or the arithmetic section 51 of the arithmetic unit 50. Since the data is directly stored in the storage unit 33 of the image input / output device 30 by directly writing from, the operator can observe it through the monitor 22 by the analog output unit 32. Further, in the above description, for the sake of simplification, it is described that each device corresponds to an independent device or component, but even if a composite element or device such as a microcomputer is used, or it is achieved by one component for convenience of description. It goes without saying that achieving the function described above with a plurality of components and circuits is merely a design convenience.

[0061]

As described above, according to the marker centroid measuring method of the present invention, it is possible to accurately detect the marker centroid with a simple method using a marker having a simple shape in which a confusing shape does not easily appear in the environment. , Even in cluttered factories and sites, automatic tracking of objects can be performed accurately and easily,
It is extremely effective in automating machines and devices.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating the principle of a marker centroid measuring method of the present invention.

FIG. 2 is a diagram for explaining a diamond marker detection situation by the marker centroid measuring method of the present invention.

FIG. 3 is a diagram for explaining a gravity center position measurement situation by the marker gravity center measurement method of the present invention.

FIG. 4 is a diagram illustrating a procedure for measuring the center of gravity position by the method for measuring the center of gravity of a marker according to the present invention.

FIG. 5 is a block diagram showing an embodiment of a marker centroid measuring device of the present invention.

6 is an explanatory diagram showing marker tracking in the embodiment of FIG.

FIG. 7 is an explanatory diagram showing a positioning device used in an embodiment of the present invention.

FIG. 8 is a flow chart showing a control procedure for the positioning apparatus of this embodiment.

[Explanation of symbols]

 1 diamond-shaped marker 2 circumscribed rectangle 3 template quadrilateral 4 triangular marker 5 square marker 6 circular marker 7, 8, 9, 10 marker image 11, 12, 13, 14, 15 frequency distribution 70 two-dimensional image 71 window 72, 73, 74 Figure 75,76,77 Circumscribed rectangle 78,79 Frequency distribution 80 image

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Yasumitsu Kurosaki 118, Futtsuka, Noda, Chiba Prefecture Kawasaki Heavy Industries, Ltd., Noda factory (72) Inventor Kiyoshi Udagawa, 118, Futtsuka, Noda, Chiba, Kawasaki Heavy Industries, Ltd.

Claims (9)

[Claims] 1. A marker centroid measuring method for measuring a position of an article by imaging a measurement target attached to an article, extracting a marker represented by the measurement target by an image processing technique, and obtaining a marker centroid position. A method for measuring the center of gravity of a marker, characterized in that the shape is a substantially rhombus. 2. A circumscribing rectangle placed substantially horizontally with respect to a figure in an image photographed by an image pickup device is calculated, and the midpoints of the sides of the circumscribing rectangle are taken to form a quadrangle connecting these midpoints and the figure. A method for measuring the center of gravity of a marker, which comprises calculating a degree of matching and using the best matching target marker. 3. The marker centroid measuring method according to claim 2, further comprising: calculating the centroid position of the marker from the horizontal most frequent position and the most vertical position of the pixels belonging to the image of the target marker. A method for measuring the center of gravity of a marker, characterized in that 4. A window is set for an image captured by an image pickup device, the image in the window is binarized based on the brightness to obtain a binarized image, and the binarized image is expanded / compressed and / or compressed / expanded. The shape is extracted by processing, the figure is extracted and labeled, and a circumscribed rectangle placed approximately horizontally on the labeled figure is calculated, and the quadrangle having the midpoint of each side of the circumscribed rectangle as the vertex and the figure are the most A marker centroid characterized by determining a match as a target marker and calculating the barycentric position of the marker from the most frequent position in the horizontal direction and the most frequent position in the vertical direction of the pixels belonging to the image of the target marker. Measuring method. 5. The marker centroid measuring method according to claim 2, wherein a measurement target representing a diamond-shaped marker is attached to the article, and an image including the measurement target is acquired. A method for measuring the center of gravity of a marker, characterized in that 6. An image pickup device, a monitor, an image input / output device, an image processing device, and a calculation device, wherein the image pickup device picks up an image including a measurement target and outputs the image in an analog manner. An apparatus inputs an analog output of the image pickup apparatus, stores it as a digitized two-dimensional image signal, sends the two-dimensional image signal to the image processing apparatus and the arithmetic unit, and is processed by the image processing apparatus. An image signal is input and stored, and the image processing device inputs a two-dimensional image signal from the image input / output device, performs image processing, extracts a figure in the image, labels and stores the image signal, and the stored image signal Is output to the image input / output device and the arithmetic unit, and the arithmetic unit processes the labeled image signal to generate a circumscribed rectangle with one horizontal side for each figure in the image, and the circumscribed rectangle A quadrangle connecting the midpoints of the sides is calculated, the target target is determined based on the degree of coincidence with the marker represented by the image signal, and the monitor inputs the video signal stored in the image input / output device. An apparatus for measuring the center of gravity of a marker, which is characterized by displaying an image as a display. 7. The marker centroid measuring device according to claim 6, wherein the arithmetic unit further has a frequency distribution in a horizontal axis direction and a vertical axis direction with respect to a signal state of the image signal representing the marker determined to be the target target. Is calculated and the barycenter of the marker is calculated with the position having the highest frequency as a reference, and the marker barycenter position is moved so as to come to the center of the image. 8. The marker centroid measuring device according to claim 6 or 7, wherein a measurement target representing a substantially rhombic marker is attached to an article, and an image including the measurement target is acquired and processed by the imaging device. An apparatus for measuring the center of gravity of a marker. 9. The marker centroid measuring device according to claim 8, further comprising an illuminating device so that an optical axis is close to an optical axis of the imaging device, and the rhombic marker has a retroreflection function, A marker centroid measuring device, wherein an image including the measurement target is acquired by the imaging device while illuminating the marker with an illuminating device.