CN107911687B - Robot teleoperation auxiliary system based on binocular stereo vision - Google Patents
Robot teleoperation auxiliary system based on binocular stereo vision Download PDFInfo
- Publication number
- CN107911687B CN107911687B CN201711302665.8A CN201711302665A CN107911687B CN 107911687 B CN107911687 B CN 107911687B CN 201711302665 A CN201711302665 A CN 201711302665A CN 107911687 B CN107911687 B CN 107911687B
- Authority
- CN
- China
- Prior art keywords
- image data
- camera
- processor
- measurement
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Length Measuring Devices By Optical Means (AREA)
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Image Analysis (AREA)
Abstract
本发明公开了一种基于双目立体视觉的机器人遥操作辅助方法，包括如下步骤：图像数据输出步骤：通过测量相机将数字图像数据输出至图像测量处理器，且同时通过监视相机将数字图像数据输出至立体显示处理器；图像数据处理步骤：所述图像测量处理器接收所述测量相机的图像数据后进行计算，计算出指定目标的相对位姿，并传输至立体显示处理器和机器人主控系统，所述立体显示处理器接收所述监视相机的图像数据进行极线校正，并将监视相机的图像数据传输至所述立体显示设备；图像数据显示步骤：所述立体显示设备将所述立体显示处理器输出的图像数据进行显示，供操作者观看。本发明测量精度高、视场大、容易搜索目标。
The invention discloses a method for assisting robot teleoperation based on binocular stereo vision. output to the stereoscopic display processor; image data processing step: the image measurement processor calculates after receiving the image data of the measurement camera, calculates the relative pose of the specified target, and transmits it to the stereoscopic display processor and the robot master system, the stereoscopic display processor receives the image data of the surveillance camera to perform epipolar correction, and transmits the image data of the surveillance camera to the stereoscopic display device; the image data display step: the stereoscopic display device converts the stereoscopic display device to the stereoscopic display device. The image data output by the display processor is displayed for the operator to watch. The invention has high measurement precision, large field of view and easy target search.
Description
Technical Field
The invention relates to binocular stereo vision measurement and display technology, in particular to a robot teleoperation auxiliary method and system based on binocular stereo vision.
Background
The teleoperation robot can assist an operator to complete various tasks, and particularly needs to replace manual operation under the severe environment which is not suitable for field operation of the operator. An important part of teleoperation is the perception of the environment and the situation of the target. The vision sensor is the key for the robot to sense the environment, and a general two-dimensional image sensor cannot sense the depth information of the environment, needs to be used in combination with other sensors such as laser ranging and the like, and is complex in structure. Two-dimensional image information fed back by a general robot vision sensor cannot completely describe a three-dimensional environment, and a general teleoperation interaction medium cannot express real three-dimensional information, so that an operator cannot experience the experience of being personally on the scene. The existing binocular stereo vision system has single function, and if the field of view is required to be large, the precision is insufficient; however, the measurement accuracy is required to be high, the field of view is small, and the target is difficult to search.
Disclosure of Invention
The invention aims to overcome the defects of the prior art and provides a robot teleoperation auxiliary method and system based on binocular stereo vision.
In order to achieve the purpose, the invention adopts the following technical scheme:
the robot teleoperation auxiliary method based on binocular stereo vision comprises the following steps:
an image data output step: outputting the digital image data to an image measurement processor through a measurement camera, and simultaneously outputting the digital image data to a stereoscopic display processor through a monitoring camera;
an image data processing step: the image measurement processor receives the image data of the measurement camera, calculates the relative pose of the designated target, and transmits the relative pose to the stereo display processor and the robot master control system, and the stereo display processor receives the image data of the monitoring camera, performs polar line correction, and transmits the image data of the monitoring camera to the stereo display equipment;
an image data display step: and the stereo display equipment displays the image data output by the stereo display processor.
The image data output step includes:
the measuring cameras comprise a left measuring camera and a right measuring camera, and the left monitoring camera and the right monitoring camera are arranged between the left measuring camera and the right measuring camera;
digital image data is output to an image measurement processor through the left and right measurement cameras, and simultaneously digital image data is output to a stereoscopic display processor through the left and right monitoring cameras.
The image data output step includes:
the measuring camera outputs digital image data to the image measuring processor according to a sensor driving time sequence output by the image measuring processor; the monitoring camera outputs digital image data to the stereoscopic display processor according to a sensor driving timing sequence output by the stereoscopic display processor.
The stereoscopic display equipment comprises a stereoscopic display helmet and a stereoscopic display television, and image data output by the stereoscopic display processor is displayed on the stereoscopic display helmet or the stereoscopic display television according to actual requirements.
The display helmet synthesizes the images of the left monitoring camera and the right monitoring camera into an image with a size of 1920 multiplied by 1080, and the image is sent to an LED liquid crystal display screen in the display helmet through an HDMI interface to be displayed.
The image data processing step comprises:
the image measurement processor performs double-thread calculation on the digital image data output by the left measurement camera and the right measurement camera through the processor, and sets the left image sensor and the right image sensor to output image data according to 4 frames/second and the resolution of 1920 multiplied by 1200;
storing the digital image data in a synchronous state random access memory for use in attitude measurement;
the image measurement processor carries out edge extraction, edge matching, three-dimensional reconstruction of target point cloud, space plane fitting and pose measurement on the received image data of the measurement camera;
transmitting the image polar line correction linked list to the stereo display processor through an asynchronous serial port;
and transmitting the calculated pose information to the stereo display processor through an asynchronous serial port for an operator to watch.
In order to achieve the purpose, the invention also adopts the following technical scheme:
provided is a robot teleoperation auxiliary system based on binocular stereo vision, including:
an image data output unit: outputting the digital image data to an image measurement processor through a measurement camera, and simultaneously outputting the digital image data to a stereoscopic display processor through a monitoring camera;
an image data processing unit: the image measurement processor receives the image data of the measurement camera, calculates the relative pose of the designated target, and transmits the relative pose to the stereo display processor and the robot master control system, and the stereo display processor receives the image data of the monitoring camera, performs polar line correction, and transmits the image data of the monitoring camera to the stereo display equipment;
an image data display unit: and the stereo display equipment displays the image data output by the stereo display processor for an operator to watch.
The image data output unit:
the measuring cameras comprise a left measuring camera and a right measuring camera, and the left monitoring camera and the right monitoring camera are arranged between the left measuring camera and the right measuring camera;
digital image data is output to an image measurement processor through the left and right measurement cameras, and simultaneously digital image data is output to a stereoscopic display processor through the left and right monitoring cameras.
The image data output unit:
the measuring camera outputs digital image data to the image measuring processor according to a sensor driving time sequence output by the image measuring processor; the monitoring camera outputs digital image data to the stereoscopic display processor according to a sensor driving timing sequence output by the stereoscopic display processor.
The stereoscopic display equipment comprises a stereoscopic display helmet and a stereoscopic display television, and image data output by the stereoscopic display processor is displayed on the stereoscopic display helmet or the stereoscopic display television according to actual requirements for an operator to watch.
The display helmet synthesizes the images of the left monitoring camera and the right monitoring camera into an image with a size of 1920 multiplied by 1080, and the image is sent to an LED liquid crystal display screen in the display helmet through an HDMI interface to be displayed.
In the image data processing unit:
the image measurement processor performs double-thread calculation on the digital image data output by the left measurement camera and the right measurement camera through the processor, and sets the left image sensor and the right image sensor to output image data according to 4 frames/second and the resolution of 1920 multiplied by 1200;
storing the digital image data in a synchronous state random access memory for use in attitude measurement;
the image measurement processor carries out edge extraction, edge matching, three-dimensional reconstruction of target point cloud, space plane fitting and pose measurement on the received image data of the measurement camera;
transmitting the image polar line correction linked list to the stereo display processor through an asynchronous serial port;
and transmitting the calculated pose information to the stereo display processor through an asynchronous serial port for an operator to watch.
The invention has the beneficial effects that: the invention discloses a robot teleoperation auxiliary method and device based on binocular stereo vision, which are used for carrying out high-precision adjustment and calibration on a measuring camera and a monitoring camera according to a binocular camera erection scheme designed by combining measurement and monitoring functions, correcting and displaying binocular images by adopting a stereo helmet, manually marking interested targets and carrying out pose measurement and calculation. The invention can carry out real-time large-view-field immersion type monitoring, the operator has strong personally on-the-scene feeling, and meanwhile, the gesture measurement and calculation are carried out on the interested target, the actual measurement precision is superior to 0.95mm and 0.58 degrees, and the invention can effectively assist the operator to carry out teleoperation on the robot.
Drawings
Fig. 1 is a flow chart of the robot teleoperation assisting method based on binocular stereo vision according to the present invention.
Fig. 2 is a schematic view showing the coverage of the field of view in the horizontal direction at different horizontal angles.
Fig. 3 shows an optical design of a measuring camera.
Fig. 4 is a schematic view of the optical design of the monitoring camera.
Fig. 5 is a block diagram of the robot teleoperation auxiliary system based on binocular stereo vision according to the present invention.
Fig. 6 is a schematic block circuit diagram of the image measurement processor.
Fig. 7 is a block diagram showing an algorithm flow of the image measurement processor.
FIG. 8a shows a target depth map generated by the left measurement camera.
FIG. 8b shows the target depth map generated by the right measurement camera.
Fig. 9 is a diagram showing the coordinates of the object pose.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, an embodiment of the present invention provides a robot teleoperation assisting method based on binocular stereo vision, including the following steps:
and step 3 is an image data display step: and the stereo display equipment displays the image data output by the stereo display processor for an operator to watch.
In the robot teleoperation auxiliary method based on binocular stereo vision, the robot teleoperation auxiliary method is practically applied to be composed of four cameras which are divided into two groups, the middle two cameras are monitoring cameras, the outer two cameras are measuring cameras, and a cube mirror for adjusting is installed at the top of each camera. In addition, the coordinate system of the imaging surface of the camera and the coordinate system of a prism of the camera are adjusted to be parallel through the auto-collimation theodolite, a reference prism is placed between the two monitoring cameras, the optical axes of the monitoring cameras are ensured to be parallel by adjusting the angle between the prism of the camera and the reference prism, and the included angle between the optical axis of the camera and the horizontal plane is measured to be 79 degrees. The internal and external parameter calibration of the four cameras uses a checkerboard calibration board, and the internal and external parameter calibration of the monitoring camera and the measuring camera is completed by using a Zhang Zhengyou plane calibration method.
In one embodiment, in the image data output step:
the measuring cameras comprise a left measuring camera and a right measuring camera, and the left monitoring camera and the right monitoring camera are arranged between the left measuring camera and the right measuring camera;
digital image data is output to an image measurement processor through the left and right measurement cameras, and simultaneously digital image data is output to a stereoscopic display processor through the left and right monitoring cameras.
As shown in fig. 2, in order to ensure the measurement accuracy and the working range of the measuring camera, when the focal length of the optical system of the measuring camera is designed to be 12mm, the field angle is 42 ° × 27 °, the base line B is fixed to be 183mm due to the volume limitation of the system, and the measurement distance is 350mm to 800mm, and the analytical relationship between the working range d1 d2 of the measuring camera and the arrangement angle α and the base line B of the camera is established, so that the arrangement of the binocular camera is determined.
As shown in fig. 3, when the horizontal included angle is 79 °, 254.4mm corresponds to a horizontal field angle of 40 ° when the horizontal included angle is measured from the optical axis of the camera to the horizontal, and 465.1mm corresponds to a horizontal field angle of 32 ° when the horizontal included angle is measured from the optical axis of the camera to the horizontal when the horizontal included angle is 350mm, and 465.1mm corresponds to a horizontal field angle of 32 ° when the horizontal included angle is 800 mm. Therefore, the optical axes of the two measuring cameras are symmetrically arranged at an included angle of 79 degrees with the horizontal.
As shown in fig. 4, when the horizontal included angle is different, the coverage of the horizontal viewing field is 254.4mm at a distance of 350mm, which corresponds to a horizontal angle of view of 40 °, and is 465.1mm at a distance of 800mm, which corresponds to a horizontal angle of view of 32 °, when the optical axes of the left and right surveillance cameras and the horizontal included angle are 79 °. Therefore, the optical axes of the two cameras are symmetrically arranged at an included angle of 79 degrees with the horizontal.
In one embodiment, the image data output step comprises:
the measuring camera outputs digital image data to the image measuring processor according to a sensor driving time sequence output by the image measuring processor; the monitoring camera outputs digital image data to the stereoscopic display processor according to a sensor driving timing sequence output by the stereoscopic display processor. The electronic part of the monitoring camera is designed the same as the electronic part of the measuring camera, and is composed of an on-the-eye VITA area array CMOS image sensor and a peripheral circuit, and digital image data is output according to a driving time sequence provided by an image measuring processor and transmitted to the image measuring processor, wherein a schematic circuit block diagram of the image measuring processor is shown in FIG. 6.
In one embodiment, the stereoscopic display device comprises a stereoscopic display helmet and a stereoscopic display television, and image data output by the stereoscopic display processor is displayed on the stereoscopic display helmet or the stereoscopic display television according to actual needs for an operator to watch.
In one embodiment, the display helmet combines the left and right surveillance camera images into a 1920 × 1080 size image that is sent to the LED lcd display in the display helmet via an HDMI interface for display.
In one embodiment, the image data processing step:
the image measurement processor performs double-thread calculation on the digital image data output by the left measurement camera and the right measurement camera through the processor, and sets the left image sensor and the right image sensor to output image data according to 4 frames/second and the resolution of 1920 multiplied by 1200;
storing the digital image data in a synchronous state random access memory for use in attitude measurement;
the image measurement processor carries out edge extraction, edge matching, three-dimensional reconstruction of target point cloud, space plane fitting and pose measurement on the received image data of the measurement camera;
transmitting the image polar line correction linked list to the stereo display processor through an asynchronous serial port;
and transmitting the calculated pose information to the stereo display processor through an asynchronous serial port for an operator to watch.
It should be noted that an edge is one of the most basic features of an image, which can greatly reduce the information to be processed but retain the shape information of the object in the image. The invention extracts the image edge as the main characteristic to complete the posture measurement of the three-dimensional object. As shown in fig. 7, the image measurement processor mainly performs edge extraction, edge matching, three-dimensional reconstruction, spatial plane fitting, and pose measurement of the left and right camera images.
It should be noted that, as shown in fig. 8a and 8b, two sets of target depth maps are generated by a binocular measuring camera, an algorithm automatically extracts a saliency plane of a target, performs three-dimensional reconstruction and spatial plane fitting, and calculates a pose of the target. The target pose coordinates are shown in fig. 9, and the target significant plane equation, the position coordinates and the normal angle side of the target position coordinates shown in fig. 9 refer to the following table;
| objective significant plane equation | Position coordinates | Normal angle | |
| Position 1 | -0.00192144x+0.00111288y+0.0212844z＝1 | (3.39，-5.572，47.43) | (84.84，87.01，5.95) |
| |
0.000629517x+0.000639321y+0.0208609z＝1 | (1.60，-5.926，48.16) | (88.27，88.24，3.00) |
In order to achieve the purpose, the invention also adopts the following technical scheme:
as shown in fig. 5, there is provided a robot teleoperation assistance system based on binocular stereo vision, including:
an image data output unit: outputting the digital image data to an image measurement processor through a measurement camera, and simultaneously outputting the digital image data to a stereoscopic display processor through a monitoring camera;
an image data processing unit: the image measurement processor receives the image data of the measurement camera, calculates the relative pose of the designated target, and transmits the relative pose to the stereo display processor and the robot master control system, and the stereo display processor receives the image data of the monitoring camera, performs polar line correction, and transmits the image data of the monitoring camera to the stereo display equipment;
an image data display unit: and the stereo display equipment displays the image data output by the stereo display processor for an operator to watch.
In the robot teleoperation auxiliary method based on binocular stereo vision, the robot teleoperation auxiliary method is practically applied to be composed of four cameras which are divided into two groups, the middle two cameras are monitoring cameras, the outer two cameras are measuring cameras, and a cube mirror for adjusting is installed at the top of each camera. In addition, the coordinate system of the imaging surface of the camera and the coordinate system of a prism of the camera are adjusted to be parallel through the auto-collimation theodolite, a reference prism is placed between the two monitoring cameras, the optical axes of the monitoring cameras are ensured to be parallel by adjusting the angle between the prism of the camera and the reference prism, and the included angle between the optical axis of the camera and the horizontal plane is measured to be 79 degrees. The internal and external parameter calibration of the four cameras uses a checkerboard calibration board, and the internal and external parameter calibration of the monitoring camera and the measuring camera is completed by using a Zhang Zhengyou plane calibration method.
In one embodiment, in the image data output unit:
the measuring cameras comprise a left measuring camera and a right measuring camera, and the left monitoring camera and the right monitoring camera are arranged between the left measuring camera and the right measuring camera;
digital image data is output to an image measurement processor through the left and right measurement cameras, and simultaneously digital image data is output to a stereoscopic display processor through the left and right monitoring cameras.
As shown in fig. 2, in order to ensure the measurement accuracy and the working range of the measuring camera, when the focal length of the optical system of the measuring camera is designed to be 12mm, the field angle is 42 ° × 27 °, the base line B is fixed to be 183mm due to the volume limitation of the system, and the measurement distance is 350mm to 800mm, and the analytical relationship between the working range d1 d2 of the measuring camera and the arrangement angle α and the base line B of the camera is established, so that the arrangement of the binocular camera is determined.
As shown in fig. 3, when the horizontal angle is different, the coverage of the horizontal viewing field is 254.4mm corresponding to the horizontal angle of view 40 ° at a distance of 350mm, and 465.1mm corresponding to the horizontal angle of view 32 ° at a distance of 800mm, when the camera optical axis forms a 79 ° angle with the horizontal. Therefore, the optical axes of the two measuring cameras are symmetrically arranged at an included angle of 79 degrees with the horizontal.
As shown in fig. 4, when the horizontal included angle is different, the coverage of the horizontal viewing field is 254.4mm at a distance of 350mm, which corresponds to a horizontal angle of view of 40 °, and is 465.1mm at a distance of 800mm, which corresponds to a horizontal angle of view of 32 °, when the optical axes of the left and right surveillance cameras and the horizontal included angle are 79 °. Therefore, the optical axes of the two cameras are symmetrically arranged at an included angle of 79 degrees with the horizontal.
In one embodiment, in the image data output unit:
the measuring camera outputs digital image data to the image measuring processor according to a sensor driving time sequence output by the image measuring processor; the monitoring camera outputs digital image data to the stereoscopic display processor according to a sensor driving timing sequence output by the stereoscopic display processor. The electronic part of the monitoring camera is designed the same as the electronic part of the measuring camera, and consists of a VITA area array CMOS image sensor and a peripheral circuit in the Anson, and digital image data is output according to a driving time sequence provided by the image measuring processor and transmitted to the image measuring processor.
In one embodiment, the stereoscopic display device comprises a stereoscopic display helmet and a stereoscopic display television, and image data output by the stereoscopic display processor is displayed on the stereoscopic display helmet or the stereoscopic display television according to actual needs for an operator to watch.
In one embodiment, the display helmet combines the left and right surveillance camera images into a 1920 × 1080 size image that is sent to the LED lcd display in the display helmet via an HDMI interface for display.
In one embodiment, the image data processing unit operates as follows:
the image measurement processor performs double-thread calculation on the digital image data output by the left measurement camera and the right measurement camera through the processor, and sets the left image sensor and the right image sensor to output image data according to 4 frames/second and the resolution of 1920 multiplied by 1200;
storing the digital image data in a synchronous state random access memory for use in attitude measurement;
the image measurement processor carries out edge extraction, edge matching, three-dimensional reconstruction of target point cloud, space plane fitting and pose measurement on the received image data of the measurement camera;
transmitting the image polar line correction linked list to the stereo display processor through an asynchronous serial port;
and transmitting the calculated pose information to the stereo display processor through an asynchronous serial port for an operator to watch.
It should be noted that an edge is one of the most basic features of an image, which can greatly reduce the information to be processed but retain the shape information of the object in the image. The invention extracts the image edge as the main characteristic to complete the posture measurement of the three-dimensional object. As shown in fig. 7, the image measurement processor mainly performs edge extraction, edge matching, three-dimensional reconstruction, spatial plane fitting, and pose measurement of the left and right camera images.
It should be noted that, as shown in fig. 8a and 8b, two sets of target depth maps are generated by a binocular measuring camera, an algorithm automatically extracts a saliency plane of a target, performs three-dimensional reconstruction and spatial plane fitting, and calculates a pose of the target. The target pose coordinates are shown in fig. 9, and the target significant plane equation, the position coordinates and the normal angle side of the target position coordinates shown in fig. 9 refer to the following table;
| objective significant plane equation | Position coordinates | Normal angle | |
| Position 1 | -0.00192144x+0.00111288y+0.0212844z＝1 | (3.39，-5.572，47.43) | (84.84，87.01，5.95) |
| |
0.000629517x+0.000639321y+0.0208609z＝1 | (1.60，-5.926，48.16) | (88.27，88.24，3.00) |
The above-described embodiments of the present invention should not be construed as limiting the scope of the present invention. Any other corresponding changes and modifications made according to the technical idea of the present invention should be included in the protection scope of the claims of the present invention.
Claims (8)
1. A robot teleoperation auxiliary method based on binocular stereo vision is characterized by comprising the following steps:
an image data output step: outputting the digital image data to an image measurement processor through the left and right measurement cameras, and simultaneously outputting the digital image data to a stereoscopic display processor through the left and right monitoring cameras;
the left monitoring camera and the right monitoring camera are arranged between the left measuring camera and the right measuring camera, and the imaging plane coordinate systems of the four cameras and the prism coordinate systems of the four cameras are adjusted to be parallel through the auto-collimation theodolite; a reference prism is arranged between the left monitoring camera and the right monitoring camera, the optical axes of the two monitoring cameras are ensured to be parallel by adjusting the angle between the camera prism and the reference prism, and the included angle between the optical axes of the two measuring cameras and the horizontal plane is 79 degrees;
an image data processing step: the image measurement processor receives the image data of the measurement camera, calculates the relative pose of the designated target, and transmits the relative pose to the stereo display processor and the robot master control system;
the stereoscopic display processor receives the image data of the monitoring camera to perform polar line correction and transmits the image data of the monitoring camera to the stereoscopic display equipment;
an image data display step: and the stereo display equipment displays the image data output by the stereo display processor.
2. The robot teleoperation assisting method based on binocular stereo vision according to claim 1, wherein in the image data outputting step:
the measuring camera outputs digital image data to the image measuring processor according to a sensor driving time sequence output by the image measuring processor; the monitoring camera outputs digital image data to the stereoscopic display processor according to a sensor driving timing sequence output by the stereoscopic display processor.
3. The binocular stereo-vision based robot teleoperation assisting method of claim 1, wherein the stereoscopic display device comprises a stereoscopic display helmet and a stereoscopic display television for an operator to watch.
4. The binocular stereo-vision-based robot teleoperation assisting method of claim 3, wherein the display helmet synthesizes the left and right surveillance camera images into a 1920 x 1080 size image, and the image is sent to an LED liquid crystal display in the display helmet through an HDMI interface for display.
5. The binocular stereo vision based robot teleoperation assisting method of claim 1, wherein in the image data processing step:
the image measurement processor performs double-thread calculation on the digital image data output by the left measurement camera and the right measurement camera through the processor, and sets the left image sensor and the right image sensor to output image data according to 4 frames/second and the resolution of 1920 multiplied by 1200;
storing the digital image data in a synchronous dynamic random access memory for use in attitude measurement;
the image measurement processor carries out edge extraction, edge matching, three-dimensional reconstruction of target point cloud, space plane fitting and pose measurement on the received image data of the measurement camera;
transmitting the image polar line correction linked list to the stereo display processor through an asynchronous serial port;
and transmitting the calculated pose information to the stereo display processor through an asynchronous serial port for an operator to watch.
6. A robot teleoperation auxiliary system based on binocular stereo vision is characterized by comprising:
an image data output unit:
outputting the digital image data to an image measurement processor through the left and right measurement cameras, and simultaneously outputting the digital image data to a stereoscopic display processor through the left and right monitoring cameras;
the left monitoring camera and the right monitoring camera are arranged between the left measuring camera and the right measuring camera, and the imaging plane coordinate systems of the four cameras and the prism coordinate systems of the four cameras are adjusted to be parallel through the auto-collimation theodolite; a reference prism is arranged between the left monitoring camera and the right monitoring camera, the optical axes of the two monitoring cameras are ensured to be parallel by adjusting the angle between the camera prism and the reference prism, and the included angle between the optical axes of the two measuring cameras and the horizontal plane is 79 degrees;
an image data processing unit: the image measurement processor receives the image data of the measurement camera, calculates the relative pose of the designated target, and transmits the relative pose to the stereo display processor and the robot master control system, and the stereo display processor receives the image data of the monitoring camera, performs polar line correction, and transmits the image data of the monitoring camera to the stereo display equipment;
an image data display unit: and the stereo display equipment displays the image data output by the stereo display processor.
7. The binocular stereo-vision-based robot teleoperation assistance system of claim 6, wherein in the image data output unit:
the measuring camera outputs digital image data to the image measuring processor according to a sensor driving time sequence output by the image measuring processor; the monitoring camera outputs digital image data to the stereoscopic display processor according to a sensor driving timing sequence output by the stereoscopic display processor.
8. The binocular stereo-vision based robot teleoperation assistance system of claim 6, wherein in the image data processing unit:
the image measurement processor performs double-thread calculation on the digital image data output by the left measurement camera and the right measurement camera through the processor, and sets the left image sensor and the right image sensor to output image data according to 4 frames/second and the resolution of 1920 multiplied by 1200;
storing the digital image data in a synchronous dynamic random access memory for use in attitude measurement;
the image measurement processor carries out edge extraction, edge matching, three-dimensional reconstruction of target point cloud, space plane fitting and pose measurement on the received image data of the measurement camera;
transmitting the image polar line correction linked list to the stereo display processor through an asynchronous serial port;
and transmitting the calculated pose information to the stereo display processor through an asynchronous serial port for an operator to watch.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711302665.8A CN107911687B (en) | 2017-12-11 | 2017-12-11 | Robot teleoperation auxiliary system based on binocular stereo vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711302665.8A CN107911687B (en) | 2017-12-11 | 2017-12-11 | Robot teleoperation auxiliary system based on binocular stereo vision |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107911687A CN107911687A (en) | 2018-04-13 |
| CN107911687B true CN107911687B (en) | 2020-04-10 |
Family
ID=61853999
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711302665.8A Expired - Fee Related CN107911687B (en) | 2017-12-11 | 2017-12-11 | Robot teleoperation auxiliary system based on binocular stereo vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107911687B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115963917B (en) * | 2022-12-22 | 2024-04-16 | 北京百度网讯科技有限公司 | Visual data processing apparatus and visual data processing method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101794349A (en) * | 2010-02-09 | 2010-08-04 | 北京邮电大学 | Experimental system and method for augmented reality of teleoperation of robot |
| CN102902271A (en) * | 2012-10-23 | 2013-01-30 | 上海大学 | Binocular vision-based robot target identifying and gripping system and method |
Family Cites Families (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| TW201403545A (en) * | 2012-07-13 | 2014-01-16 | Vivotek Inc | A synthetic virtual perspective image processing systems and method |
| CN103231389A (en) * | 2013-04-13 | 2013-08-07 | 李享 | Object identification method based on robot binocular three-dimensional vision |
| CN103413313B (en) * | 2013-08-19 | 2016-08-10 | 国家电网公司 | The binocular vision navigation system of electrically-based robot and method |
| CN106003036A (en) * | 2016-06-16 | 2016-10-12 | 哈尔滨工程大学 | Object grabbing and placing system based on binocular vision guidance |
-
2017
- 2017-12-11 CN CN201711302665.8A patent/CN107911687B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101794349A (en) * | 2010-02-09 | 2010-08-04 | 北京邮电大学 | Experimental system and method for augmented reality of teleoperation of robot |
| CN102902271A (en) * | 2012-10-23 | 2013-01-30 | 上海大学 | Binocular vision-based robot target identifying and gripping system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107911687A (en) | 2018-04-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11223820B2 (en) | Augmented reality displays with active alignment and corresponding methods | |
| US10869024B2 (en) | Augmented reality displays with active alignment and corresponding methods | |
| CN106651794B (en) | A Projection Speckle Correction Method Based on Virtual Camera | |
| CN111243002A (en) | Monocular laser speckle projection system calibration and depth estimation method applied to high-precision three-dimensional measurement | |
| US20160295194A1 (en) | Stereoscopic vision system generatng stereoscopic images with a monoscopic endoscope and an external adapter lens and method using the same to generate stereoscopic images | |
| WO2018086348A1 (en) | Binocular stereo vision system and depth measurement method | |
| US8897502B2 (en) | Calibration for stereoscopic capture system | |
| CN107316319B (en) | Rigid body tracking method, device and system | |
| CN104075688A (en) | Distance measurement method of binocular stereoscopic gazing monitoring system | |
| TW201100760A (en) | Distance measuring apparatus having dual stereo cameras | |
| CN106898022A (en) | A kind of hand-held quick three-dimensional scanning system and method | |
| TW202203645A (en) | Method, processing device, and display system for information display | |
| CN112837207A (en) | Panoramic depth measuring method, four-eye fisheye camera and binocular fisheye camera | |
| CN111854636A (en) | Multi-camera array three-dimensional detection system and method | |
| CN113034615A (en) | Equipment calibration method for multi-source data fusion and related device | |
| CN107911687B (en) | Robot teleoperation auxiliary system based on binocular stereo vision | |
| WO2021093804A1 (en) | Omnidirectional stereo vision camera configuration system and camera configuration method | |
| TWM594322U (en) | Camera configuration system with omnidirectional stereo vision | |
| CN104567812A (en) | Method and device for measuring spatial position | |
| CN114353667A (en) | Ground target measurement method based on AR and UAV monocular vision and its application | |
| CN109644259A (en) | 3-dimensional image preprocess method, device and wear display equipment | |
| CN116309854A (en) | Method, device, equipment, system and storage medium for calibrating augmented reality equipment | |
| CN111935472A (en) | Real-time three-dimensional panoramic video monitoring system and video monitoring method thereof | |
| CN111918050A (en) | Shooting device and image shooting method for three-dimensional panoramic image | |
| CN109214984B (en) | Image acquisition method and device, autonomous positioning navigation system and computing equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20200410 |