CN205466320U - Intelligent machine hand based on many camera lenses - Google Patents

Abstract

The utility model discloses an intelligent machine hand based on many camera lenses, include many joints multifunctional machinery hand, be used for to gather to wait to assemble the CCD camera of PCB board image, biological contact device and computer, the CCD camera install many joints multifunctional machinery on hand, biological contact device installs the finger tip at many functions of joint manipulator, CCD camera, biological contact device and many joints multifunctional machinery hand and computer link. The utility model discloses on the basis of two meshes location, biological contact device has added in the manipulator, can carry out the accurate positioning to the assembly target area, improved the rate of accuracy of special -shaped assembly, the utilization be the sensitive material's among the biological contact device pressure - electric output conversion relationship and the manipulator position and the mathematic model of pressure, the utility model discloses a high accuracy and the real -time of dysmorphism components and parts are assembled, have improved the production efficiency of electron assembly industry.

Description

An intelligent manipulator based on multi-lens

technical field

The utility model relates to the field of electronic assembly, in particular to an intelligent manipulator based on multiple lenses.

Background technique

At present, in the mechanical and electronic assembly industry, the installation of special-shaped components on the PCB often relies on manual assembly by workers on the assembly line. Manpower assembly mainly relies on human eyes and experience, and cannot guarantee production efficiency after a long time of work. Moreover, manual assembly is subjective and cannot maintain the same quality of each product. Therefore, automation technology is introduced into the electronic assembly industry imminent. Replacing human assembly with automation technology can effectively improve production efficiency, save human resources, and ensure assembly quality.

Relevant enterprises have used manipulators instead of human hands to work on the assembly line, but because the manipulators do not have human eyes for positioning, and unlike workers who can rely on the tactile experience of the hands to prompt assembly, the assembly accuracy of manipulators is not as good as human assembly. Production efficiency cannot be improved.

Utility model content

In order to overcome the shortcomings and deficiencies of the prior art, the utility model provides an intelligent manipulator based on multi-lens.

The utility model adopts the following technical solutions:

A multi-lens-based intelligent manipulator includes a multi-joint multi-functional manipulator, a CCD camera for collecting images of PCB boards to be assembled, a biological contact device and a computer, the CCD camera is installed on the multi-joint multi-functional manipulator, and the biological touch The point device is installed on the fingertip of the multi-joint multifunctional manipulator, and the CCD camera, the biological contact device and the multi-joint multifunctional manipulator are connected with the computer.

There are specifically two CCD cameras, which are respectively installed on the left side and the right side of the forearm of the multi-joint multifunctional manipulator.

The biological contact device includes a patch made of sensitive materials and a signal measurement circuit for measuring the deformation of the patch and outputting an electrical signal. The patch is covered on the fingertip of the manipulator, and the patch is connected with the signal measurement circuit The circuit is connected, and the signal measurement circuit is connected with the computer, and the signal measurement circuit is built in the multi-joint multifunctional manipulator.

The shape of the patch is finger cuff type.

The sensitive material is specifically conductive rubber.

The models and parameters of the two CCD cameras are identical, the coordinate systems of the two CCD cameras are coplanar, and the coordinate axes are placed in parallel.

A method for positioning and assembling an intelligent manipulator, comprising the steps of:

S1 The PCB board to be assembled moves to the front of the manipulator, and two CCD cameras are turned on to obtain two images of the PCB board to be assembled, which are left and right images respectively;

The S2 computer matches the obtained two PCB board images with the pre-input PCB template in the computer, and determines the target area to be assembled of the special-shaped parts in the two images;

S3 uses the binocular positioning algorithm to measure the distance between the target area to be assembled and the tip of the manipulator, and the computer controls the manipulator to move to the target area to be assembled to complete the preliminary positioning;

S4 uses the biological contact device at the tip of the robotic finger to contact the target area after the initial positioning in S3, and outputs the deformation electrical signal generated by the patch to the computer through the signal measurement circuit, and the computer fine-tunes the position to achieve precise positioning. assembly.

Said S2 is specifically: filter and denoise the images of the PCB board to be assembled acquired by two CCD cameras, and then use the edge detection operator to extract the edge part of the PCB board, remove the background part, and then use piecewise linear transformation to enhance the image , improve the contrast of the image, and finally use the template matching method based on the gray value to determine the target area to be assembled.

The S3 uses the binocular positioning algorithm to measure the distance between the target area to be assembled and the tip of the manipulator, and the computer controls the manipulator to move to the target area to be assembled to complete the preliminary positioning, specifically:

S3.1 Use Zhang Zhengyou's calibration algorithm to solve the internal parameter matrix and external parameter matrix of the CCD camera, and then perform binocular stereo calibration to determine the relative positional relationship between the two cameras. The relative positional relationship includes the rotation matrix R and the translation vector T;

S3.2 Use the template matching-based gray level cross-correlation matching method to complete the pixel point matching of the two images;

S3.3 Use the binocular positioning algorithm, that is, the imaging principle of the binocular ranging system, to measure the distance l between the target area to be assembled and the tip of the manipulator;

$\begin{array}{cc}\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; B</span>}\mathrm{<span\; class="notranslate">\; f</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}& \mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}\mathrm{<span\; class="notranslate">\; e</span>}\mathrm{<span\; class="notranslate">\; f</span>}\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; g</span>}\mathrm{<span\; class="notranslate">\; h</span>}\mathrm{<span\; class="notranslate">\; t</span>}}\end{array}$

Among them, the distance between the projection centers of the left and right cameras is the baseline distance B. When the target point A passes through the binocular ranging system composed of two left and right cameras with parallel optical axes, it will be imaged on the A1 point on the left CCD image surface and the right CCD respectively. Point A2 on the image plane, its positions on the left and right image planes are x _left and x _right respectively, the focal length of the two cameras is f, and x is the target point A imaged on the left and right CCD image planes by binocular cameras Poor location.

The S4 is specifically: after the preliminary positioning of the biological contact device at the tip of the robot finger, it contacts the PCB board to be assembled, the sensitive material will be deformed and the pressure will be changed, and the output electrical signal will be changed. The mathematical model of pressure and position adjusts the position of the manipulator, so as to achieve the purpose of accurately positioning and assembling special-shaped parts.

The beneficial effects of the utility model:

(1) The utility model can complete the automatic and precise positioning of the area to be assembled on the PCB without manual intervention, and complete the assembly of special-shaped parts;

(2) The utility model adopts two CCD cameras to respectively collect an image of a PCB board for processing, which meets the rapidity and real-time requirements of industrial electronic assembly;

(3) The utility model adds a biological tactile device in the manipulator, which can accurately locate the assembly target area and improve the accuracy of the assembly of special-shaped parts;

(4) The utility model uses the binocular positioning algorithm in the digital image processing technology in the positioning of the manipulator, which can effectively measure the distance between the manipulator and the target assembly area;

(5) The utility model uses the method of template matching to determine the target area for assembling special-shaped parts, which improves the accuracy of subsequent positioning work.

Description of drawings

Fig. 1 is the structural representation of the intelligent manipulator of the present utility model;

Fig. 2 is a structural schematic diagram of the biological contact device of the present utility model;

Fig. 3 a is the searched figure of the template matching method of the present utility model;

Fig. 3b is a schematic diagram of a matching template of the present invention;

Fig. 4 is a work flow diagram of the utility model.

detailed description

The utility model will be described in further detail below in conjunction with the embodiments and accompanying drawings, but the implementation of the utility model is not limited thereto.

Example

As shown in Figure 1, a kind of intelligent manipulator based on multi-lens, comprises a multi-joint multifunctional manipulator 1, is used for collecting the CCD camera 2 of PCB board image to be assembled, biological contact device 3 and computer, and described CCD camera is specific There are two cameras, which are respectively installed on the left and right sides of the forearm of the manipulator. The specific position of the camera can be installed on different parts of the manipulator according to the actual situation. The biological contact device can convert the position information of the area to be assembled into an electrical signal and feed it back to the computer. . The number of biological contacts can be adjusted according to actual needs. The computer is responsible for processing the collected images and feedback electrical signals and sending out control signals to instruct the manipulator to complete the final positioning and assembly of special-shaped parts. The manipulator has a built-in control circuit of the manipulator, and the control circuit is connected to the computer.

The models and parameters of the two CCD cameras used in the utility model are consistent, the optical axes are basically kept parallel, the coordinate systems of the binocular cameras are coplanar and the coordinate axes are placed in parallel, the left and right images collected synchronously have the same size and proportion, and the grayscale information of the images keep relatively intact,

As shown in Figure 2, the biological contact device includes a patch 5 made of sensitive materials and a signal measurement circuit 4 for measuring the deformation of the finger cuff and outputting an electrical signal. The patch 5 is covered on the fingertip of the manipulator. Above, the patch is connected to the signal measurement circuit, the signal measurement circuit 4 is connected to the computer, and the signal measurement circuit is built in the multi-joint multifunctional manipulator. In this embodiment, the patch adopts a finger cot type, and three patches are used to be placed on the fingertips of the manipulator.

The sensitive material has pressure-electrical output characteristics, and can convert received pressure into electrical signal output. In this embodiment, the shape of the patch is a fingertip, and the sensitive material used is conductive rubber. The biological contact device made of this sensitive material is installed on the fingertip of the manipulator like a fingertip. When the fingertip touches the PCB When the board is pressed, the pressure on the sensitive material on the biological contact device due to deformation will change, resulting in a change in the output electrical signal. Through the signal measurement circuit, the output electrical signal of the biological contact device changed due to deformation can be measured and sent to the computer as the output signal, and the computer can obtain the current position of the manipulator according to the pre-obtained mathematical model, and output the corresponding control The signal is adjusted.

Figure 3a is the searched image, and Figure 3b is the matching template. Let the matching template be superimposed on the searched image for translation, and the searched image covered by the template is a subgraph. Compare the contents of the subgraph and the matching template. If The largest similarity measure between the two means that the content of the two is consistent, and the subgraph at this time is the matching area to be found.

As shown in Figure 4, the positioning and assembly method implemented by this manipulator includes the following steps:

S1 acquires the image of the PCB board to be assembled.

The PCB board to be assembled is transported to the front of the manipulator through the assembly line, the brightness of the light source is adjusted, and the images of the marked PCB board are obtained by shooting with two CCD cameras on the left and right respectively. The CCD camera installed on the left becomes the left CCD camera, and the image is called the left image. , the image captured by the right CCD camera is called the right image.

S2 template matching finds the target region to be assembled.

There is a lot of noise in the PCB board image collected from the industrial site, so it is necessary to filter and denoise the image first. The image collected by the camera includes the PCB board to be assembled, which is called the foreground. At the same time, the image also includes the background part. To identify and analyze the target area, it must first be extracted from the background, so the image must be processed Image segmentation processing. Since the target (PCB board) is a regular geometric shape, an edge detection operator can be used to extract the edge part of the PCB board to extract the background part.

In order to improve the recognition of the image, highlight and strengthen the details and edges of the image, which is beneficial to the template matching of the target area to be assembled, it is necessary to enhance the image of the PCB board image, improve the contrast of the image, and convert the original gray value interval The range is converted according to a certain mapping relationship, thereby achieving the effect of enhancing the contrast between the background image and the target image. The utility model adopts a piecewise linear transformation enhancement method. Suppose the grayscale function of the original image is f(r,c), the grayscale range is [0,M _f ], the transformed image function is expressed as g(r,c), and the grayscale range is [0,M _g ] , the transformation formula can be expressed as:

$\left\{\begin{array}{cc}\frac{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; g</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; d</span>}}{{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; b</span>}}<span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; \&rsqb;</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; d</span>}& \mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span>{\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; f</span>}}\\ \frac{\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; c</span>}}{\mathrm{<span\; class="notranslate">\; b</span>}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; a</span>}}<span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; \&rsqb;</span><span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; c</span>}& \mathrm{<span\; class="notranslate">\; a</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; <</span>\mathrm{<span\; class="notranslate">\; b</span>}\\ \frac{\mathrm{<span\; class="notranslate">\; c</span>}}{\mathrm{<span\; class="notranslate">\; a</span>}}\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span>& \mathrm{<span\; class="notranslate">\; 0</span>}<span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&le;</span>\mathrm{<span\; class="notranslate">\; a</span>}\end{array}\right.$

Template matching is the process of finding the template image in the image to be searched by calculating the similarity measure between the template image and the image to be searched. The process of template matching can be described as: first calculate the similarity measure between the template image and the image to be searched pixel by pixel, and then find the largest similarity measure region as the matching position. The principle is shown in Figure 3a and Figure 3b.

After image enhancement is performed on the PCB board image, since the distribution of gray values in each area of the PCB board image is uniform and fixed, the utility model adopts a template matching method based on gray values. The template matching method based on gray value takes the gray value of the whole image as a similarity measure, and uses the defined search strategy to search for qualified regions in the image to be searched in order from top to bottom and from left to right. Set a search window with a specified size, and perform search comparison in the search window.

The position of the object in the image to be searched can be described by translation. The template is represented by an image t(r,c), where the region of interest is designated as T. Template matching is to translate the region of interest T of the template in a certain order in the image to be matched, and then calculate the relationship between the region and the template in the image to be matched. The similarity measure s for the region of interest. The similarity measure is described by the following formula:

s(r,c)=s{t(u,v),f(r+u,c+v);(u,v)∈T}

Where s(r,c) represents the similarity measure calculated based on the gray value, t(u,v) represents the gray value of each point in the template, f(r+u,c+v) represents the region of interest of the template moved to The grayscale value of the image's current position.

The easiest way to obtain the similarity measure is to calculate the sum of absolute values (SAD) or the sum of squares (SSD) of all differences in gray values between two images. SAD and SSD can be expressed by the following two formulas respectively:

$\mathrm{<span\; class="notranslate">\; the\; s</span>}\mathrm{<span\; class="notranslate">\; a</span>}\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; no</span>}}\underset{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; T</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; |</span>$

$\mathrm{<span\; class="notranslate">\; the\; s</span>}\mathrm{<span\; class="notranslate">\; the\; s</span>}\mathrm{<span\; class="notranslate">\; d</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; no</span>}}\underset{<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&Element;</span>\mathrm{<span\; class="notranslate">\; T</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}{<span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; r</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; c</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&rsqb;</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}$

Wherein, n represents the number of pixels in the region of interest of the template, that is, n=|T|. For SAD and SSD, the greater the value of the similarity measure, the greater the difference between the image to be searched and the template. The target area to be assembled can be determined by using the template matching method based on the gray value.

S3 uses the binocular positioning algorithm to measure the distance between the target area to be assembled and the tip of the manipulator, and the computer controls the manipulator to move to the target area to be assembled to complete the preliminary positioning;

S3.1 Calibrate the binocular camera.

Since this paper uses two cameras with identical parameters to form a binocular stereo system, it is proposed to calibrate the two cameras first, and then solve their internal and external parameters. The conversion relationship between the image coordinate system and the world coordinate system is:

$\begin{array}{c}\mathrm{<span\; class="notranslate">\; the\; s</span>}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; v</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{cccc}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}& {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}& \mathrm{<span\; class="notranslate">\; 0</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}\end{array}\right]\left[\begin{array}{cc}{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 3</span>}}& {\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; 0</span>}}^{\mathrm{<span\; class="notranslate">\; T</span>}}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\\ <span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; A</span>}\mathrm{<span\; class="notranslate">\; m</span>}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; h</span>}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ {\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; w</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\end{array}$

A is the internal parameter matrix of the camera, M is determined by the position and direction of the camera relative to the world coordinate system, and has nothing to do with the pinhole camera model of the internal parameter map of the camera. M is called the external parameter matrix of the camera, and the calibration algorithm of the camera is used to solve the camera Intrinsic parameter matrix and extrinsic parameter matrix.

After completing the calibration of a single camera and obtaining the internal and external parameters of a single camera, perform binocular stereo calibration again to calculate the relative position between the left and right cameras, that is, the external parameters, that is, the relative positional relationship between the left and right cameras, including the rotation matrix R and translation vector T.

Assuming that the three-dimensional world coordinates of point P on the checkerboard calibration board are X _w , and two cameras are used to collect images at the same time, the coordinates of point P in the coordinate system of the left and right cameras are X _L and X _R respectively, and the external parameter matrices of the left and right cameras are respectively ( R _L ,t _L ) and (R _R ,t _R ), according to the conversion relationship between the world coordinate system and the camera coordinate system:

$\left\{\begin{array}{c}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; W</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; L</span>}}\\ {\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; R</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; W</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; R</span>}}\end{array}\right.$

According to the above formula, the transformation relationship from the left camera to the right camera can be obtained:

X _R ＝ RX _L +T

in

R＝R _R R _L ^-1 , T＝t _R -Rt _L

Then, according to the respective external parameters of the left and right cameras, the relationship parameters of the stereo cameras in the binocular vision system, that is, the positional relationship of the stereo cameras, can be obtained, and the calibration of the binocular cameras is thus completed.

S3.2 Use the template matching-based gray level cross-correlation matching method to complete the pixel point matching of the two images;

The models and parameters of the left and right cameras used in the utility model are consistent, the optical axes are basically kept parallel, the coordinate systems of the binocular cameras are coplanar and the coordinate axes are placed in parallel, the left and right images collected synchronously have the same size and proportion, and the gray information of the images remains It is relatively complete, and the image of the target area to be assembled has been determined in S2. Therefore, the utility model adopts the gray-scale cross-correlation matching method based on the template matching to complete the high-precision matching of the target image area.

The normalized cross-correlation matching algorithm judges whether the match is based on the cross-correlation function established between the template image and the search sub-image on the image to be matched. The expression of the cross-correlation function is as follows:

$\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>{\mathrm{<span\; class="notranslate">\; S</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}{\sqrt{\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; T</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; \&lsqb;</span>{\mathrm{<span\; class="notranslate">\; S</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; \&rsqb;</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}}$

$\mathrm{<span\; class="notranslate">\; N</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; S</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; S</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; )</span>}{{\sqrt{\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; \&lsqb;</span>\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; \&rsqb;</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}<span\; class="notranslate">\; \&lsqb;</span>{\mathrm{<span\; class="notranslate">\; S</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span>\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; S</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; \&rsqb;</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

$\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; T</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>$

$\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{{\mathrm{<span\; class="notranslate">\; S</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; 1</span>}}{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; N</span>}}\underset{\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; m</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}\underset{\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{<span\; class="notranslate">\; \&Sigma;</span>}}{\mathrm{<span\; class="notranslate">\; S</span>}}^{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; j</span>}}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; no</span>}<span\; class="notranslate">\; )</span>$

In the above formula, the template image is T(m,n), and the size of the template image is M×N, is the average gray value of all pixels on T(m,n); the search image area with (i,j) as the upper left corner pixel on the reference image is S ^i,j (m,n), is the average of the gray values of all pixels in the search image. Image matching is to match the upper left pixel of the target image area, that is, the template. The value range of the cross-correlation function value N(i,j) is 0≤N(i,j)≤1, and its value depends on the search image with (i,j) as the upper left corner pixel on the reference image How well the region matches the template image. The larger the value of the cross-correlation function corresponding to a certain pixel point, the higher the matching degree of the pixel point is, and the pixel point with the largest cross-correlation function value is selected as the most matching pixel point.

After completing the matching of the left and right image pixels, as shown in the figure, according to the imaging principle of the binocular ranging system, the left and right CCD cameras of the same model are placed in parallel on the same plane, and the distance between the projection centers of the left and right cameras is the baseline distance B .

When the target point A passes through the binocular ranging system composed of two left and right cameras with parallel optical axes, it will be imaged on the A1 point on the left CCD image plane and the A2 point on the right CCD image plane respectively, and its position on the left and right image planes are x _left and x _right respectively. It is known that the focal length of the two cameras is f, and the measured distance l can be deduced according to the triangle similarity principle:

$\begin{array}{cc}\mathrm{<span\; class="notranslate">\; l</span>}<span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; B</span>}\mathrm{<span\; class="notranslate">\; f</span>}}{\mathrm{<span\; class="notranslate">\; x</span>}}& \mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}\mathrm{<span\; class="notranslate">\; e</span>}\mathrm{<span\; class="notranslate">\; f</span>}\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}\mathrm{<span\; class="notranslate">\; i</span>}\mathrm{<span\; class="notranslate">\; g</span>}\mathrm{<span\; class="notranslate">\; h</span>}\mathrm{<span\; class="notranslate">\; t</span>}}\end{array}$

x is the position difference between the imaging points of point A on the left and right CCD image planes respectively imaged by binocular cameras, which is also called binocular parallax. It can be seen that under the ideal state that the optical axes of the binocular cameras are strictly parallel, and the image of the target object is acquired at the same time, the corresponding position of the same target in the left and right CCD images is determined by the image matching algorithm, and the binocular parallax x is calculated. Baseline size, the target distance can be obtained through the above formula, so far the initial positioning of the target area to be assembled is completed.

S4 uses the biological contact device at the tip of the robotic finger to contact the target area after the initial positioning in S3, and outputs the deformation electrical signal generated by the patch to the computer through the signal measurement circuit, and the computer fine-tunes the position to achieve precise positioning. assembly.

When the manipulator is assembling, it will be in contact with the PCB board, and the fingertips of the manipulator will be under pressure. When the special-shaped components are in accurate contact with the area to be assembled and when the special-shaped components are in contact with other areas of the PCB board, the pressure is different. Therefore, a mathematical model of pressure and position can be established, and the manipulator can be adjusted by the pressure of the fingertips. position, so as to complete the precise positioning of the target area to be assembled.

The utility model adopts a sensitive material to design a biological contact device. The sensitive material has pressure-electrical output characteristics and can convert the received pressure into an electrical signal output. The biological contact device made of this sensitive material is installed on the fingertip of the manipulator like a finger cot. When the fingertip touches the PCB board, the pressure on the conductive material on the biological contact device will change due to deformation. This results in a change in the output electrical signal. There is a signal measurement circuit between the sensitive material and the tip of the manipulator, which can measure the output electrical signal of the biological contact device due to deformation and send it to the computer as the output signal. The computer can obtain the current manipulator according to the pre-obtained mathematical model. position, and output the corresponding control signal for adjustment.

The manipulator completed the initial positioning of the area to be assembled in step S3, moved the manipulator to this area, and fine-tuned the position of the manipulator according to the output signal of the biological contact device on the fingertip after the manipulator came into contact with the PCB board, and finally achieved accurate Locate and complete the assembly of special-shaped parts.

The above-mentioned embodiment is a preferred implementation mode of the present utility model, but the implementation mode of the present utility model is not limited by the described embodiment, and any other changes, modifications, modifications, Substitution, combination, and simplification should all be equivalent replacement methods, and are all included in the protection scope of the present utility model.

Claims (6)

1. a puma manipulator based on many camera lenses, it is characterized in that, including multi-joint multifunction manipulator, for gathering the ccd video camera of pcb board image to be assembled, biological contact making device and computer, described ccd video camera is arranged on multi-joint multi-functional mechanical on hand, biological contact making device is arranged on the finger tip of multi-joint multifunction manipulator, and described ccd video camera, biological contact making device and multi-joint multifunction manipulator are connected with computer.

Puma manipulator the most according to claim 1, it is characterised in that described ccd video camera is specially two, is separately mounted to left side and the right side of multi-joint multifunction manipulator forearm.

Puma manipulator the most according to claim 1, it is characterized in that, described biological contact making device includes the paster made with sensitive material and for measuring paster deformation and exporting the circuitry for signal measurement of the signal of telecommunication, described paster covers on the finger tip of mechanical hand, described paster is connected with circuitry for signal measurement, circuitry for signal measurement is connected with computer, and described circuitry for signal measurement is built in inside multi-joint multifunction manipulator.

Puma manipulator the most according to claim 3, it is characterised in that described paster be shaped as finger cot type.

Puma manipulator the most according to claim 3, it is characterised in that described sensitive material is specially conductive rubber.

Puma manipulator the most according to claim 2, it is characterised in that model and the parameter of said two ccd video camera are identical, and the coordinate system of two ccd video cameras is coplanar, and each coordinate axes is placed in parallel.