JP4464912B2 - Robot control apparatus and autonomous mobile robot - Google Patents

Description

The present invention relates to a robot control apparatus and an autonomous mobile robot that generate a movable route while an autonomous mobile robot recognizes a movable range by autonomous movement . The present invention does not particularly lay a magnetic tape or a reflective tape partially on the floor surface as a guide path. For example, an autonomous mobile robot is provided with an array antenna, a person is provided with a transmitter, and the like. A robot control device that detects a direction angle in time series, moves a robot in accordance with the movement of a person, teaches a basic path by walking and follows a basic path, and generates a movable path while recognizing a movable range And an autonomous mobile robot .

  In the prior art, detailed manual map information is indispensable regarding the route teaching and position detection control method of the autonomous mobile robot. For example, in Patent Document 1, guides such as guide lines are made unnecessary by controlling a moving body based on map information and a storage unit for a moving route, and positional information from sensors provided on the front and side of a vehicle body. ing.

  However, in mobile robot route teaching in a home environment, it is not practical to let a person directly edit and teach position data. A conventional technique is provided in a horizontal direction on a storage unit that stores map information of a moving object that moves on a floor, a first distance sensor provided on the front surface of the moving object, and a side surface of the moving object. A plurality of second distance sensors, a signal processing circuit that performs signal processing on the outputs of the first distance sensor and the second distance sensor, and an output signal of the signal processing circuit. The amount of deviation during traveling is obtained based on the detection distance of the distance sensor, the vehicle body angle is obtained, the corner portion is detected based on the detection distances of the first distance sensor and the second distance sensor, and the storage portion A position detecting unit that detects the position of the moving body based on the map information stored in the control unit, and a control unit that controls the moving direction of the moving body based on the detection result of the position detecting unit.

  The prior art is a method of detecting the position of the moving body based on stored map information and controlling the moving direction of the moving body based on the position detection result. There is no method in the prior art that does not use maps as teaching mediators.

  In robot path teaching and path generation in the prior art, position data is directly edited by numerical values or visual information to be taught by a person.

Patent 2825239 (Toshiba)

  However, in mobile robot route teaching in a home environment, it is not practical to let a person edit and teach position data directly. The application of techniques such as chasing human instructions sequentially is a problem.

Therefore, the object of the present invention is not to let the person directly edit and teach the position data, and after walking the basic route and teaching the copying route, the range in which the robot can move by its autonomous movement is set. An object of the present invention is to provide a robot control device and an autonomous mobile robot that generate a movable route while recognizing.

  In order to achieve the above object, the present invention is configured as follows.

According to a first aspect of the present invention is mounted on a movable robot, and, after detection knowledge the human as a moving body existing in front of the robot, the human movement detection section for detecting a movement of the person When,
A driving device that is mounted on the robot and moves the robot in accordance with the movement of the person detected by the human movement detection unit during route teaching;
A robot movement amount detection unit for detecting a movement amount of the robot moved by the driving device;
A first route teaching data conversion unit that accumulates movement amount data representing the movement amount detected by the robot movement amount detection unit and converts the movement amount data into route teaching data;
It is mounted on the robot, and a peripheral object detection unit around the obstacles and the robot before Symbol robot detects the ceiling or wall position of the space to be moved,
When the robot autonomously moves by driving the driving device along the route teaching data converted by the first route teaching data conversion unit, the position of the obstacle detected by the surrounding object detection unit is A robot movable range calculator that calculates a robot movable range of the robot with respect to route teaching data;
A movement path generation unit that generates a movement path for autonomous movement of the robot based on the path teaching data and the movement range calculated by the robot movement range calculation unit;
The human movement detector is
A corresponding point position calculating and arranging unit for calculating and arranging the position of corresponding points to be detected according to the movement of the person around the robot in advance;
A time-series multiple image acquisition unit that acquires a plurality of images in time series;
Corresponding points arranged by the corresponding point position calculating and arranging unit are detected between the time-series images acquired by the time-series plural image acquiring unit, and between the detected images of the corresponding points are detected. A movement amount calculation unit for calculating the movement amount of
A moving body movement determination unit that determines whether the movement amount calculated by the movement amount calculation unit is a corresponding point that matches the movement of the person;
From the corresponding point group obtained by the mobile body movement determination unit, a mobile body region extraction unit that extracts a mobile body region that is a range where the person exists,
A distance image calculation unit that calculates a distance image of a distance image specific range around the robot that is a certain range around the robot;
A distance image specifying range moving unit that moves the distance image specifying range calculated by the distance image calculating unit so as to match the range of the moving object region extracted by the moving object region extracting unit;
A moving object region determination unit that determines a moving object region of the distance image after the movement by the distance image specific range moving unit;
A moving body position specifying unit that specifies the position of the person from the moving body area of the distance image obtained by the moving body area determination unit;
A distance calculating unit that calculates a distance from the robot to the person based on the position of the person specified on the distance image by the moving body position specifying unit;
The position of the corresponding point to be detected according to the movement of the person in advance is configured to include a corresponding point position calculation arrangement changing unit that changes each time according to the position of the person,
According to the movement route generated by the movement route generation unit, the person movement detection unit identifies the person and continuously detects the distance and direction between the person and the robot, thereby autonomously moving the robot. A robot controller that controls
According to the second aspect of the present invention, the surrounding object detection unit can acquire an omnidirectional image around the robot and an obstacle detection capable of detecting an obstacle around the robot. The robot control device according to the first aspect is provided for detecting the obstacle around the robot and a ceiling or wall position of a space in which the robot moves.
According to a third aspect of the present invention, there is provided an autonomous mobile robot comprising the robot control device according to the first or second aspect.

  As described above, according to the robot control apparatus of the present invention, it is possible to identify a moving body and continuously detect the distance (direction / direction) of the moving body.

  According to the robot control apparatus of the present invention, a panoramic image of the ceiling / wall surface is input by the omnidirectional camera attached to the robot as an example of the omnidirectional image input unit, and the ceiling / wall surface of the target point stored in advance by image input is stored. Calculate the position deviation information with the panoramic image, for example, add the path integrated value and position deviation information by the wheel attached encoder of the robot drive unit to the carriage motion equation and control the carriage position, and move the deviation from the target point while moving It is corrected and the indoor work is carried out by a work device such as a robot.

  As a result, it is possible to move the robot in accordance with various indoor scenes without requiring detailed map information or magnetic tape installed on the floor surface.

  According to the present invention, it is possible to correct misalignment during movement, and work at multiple points can be performed continuously in a short time. Further, if a panoramic image of the ceiling or wall surface of the target point is input and stored, designation of a fixed position such as a so-called landmark becomes unnecessary.

  Before continuing the description of the present invention, the same parts are denoted by the same reference numerals in the accompanying drawings.

  Hereinafter, various embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

(First embodiment)
As shown in FIG. 1A, the robot control apparatus according to the first embodiment of the present invention is mounted on a robot 1 that can travel on a substantially flat traveling floor surface 105, and the mobile robot 1 moves forward and backward. It controls the movement of. Specifically, the robot control device includes a driving device 10, a travel distance detection unit 20, a direction angle detection unit 30, a human movement detection unit 31, a robot movement amount detection unit 32, and robot basic path teaching data. The conversion unit 33, the robot basic route teaching data storage unit 34, the movable range calculation unit 35, the obstacle detection unit 36, the movement route generation unit 37, and the drive device 10 to the movement route generation unit 37 are operated. The control part 50 to be controlled is provided.

  The driving device 10 includes a left motor driving unit 11 that drives the left traveling motor 111 to move the mobile robot 1 to the right, and a right motor that drives the right traveling motor 121 to move the mobile robot 1 to the left. It is comprised so that the drive part 12 may be provided. The left traveling motor 111 and the right traveling motor 121 are respectively provided with rear drive wheels 100 shown in FIGS. 1B and 2. When the mobile robot 1 is moved to the right, the left motor drive unit 11 The left traveling motor 111 is rotated more than the rotation of the right traveling motor 121. Conversely, when the mobile robot 1 is moved to the left side, the right side motor drive unit 12 rotates the right side travel motor 121 more than the rotation of the left side travel motor 111. When the mobile robot 1 is moved forward or backward, the left side motor drive unit 11 and the right side motor drive unit 12 are synchronized, and the left side travel motor 111 and the right side travel motor 121 are similarly rotated forward or reverse. Yes. A pair of front auxiliary traveling wheels 101 are rotatably disposed on the front side of the robot 1 so as to be rotatable.

  Further, the travel distance detection unit 20 detects the travel distance of the mobile robot 1 that is moved by the driving device 10 and outputs travel distance data. As a specific configuration example of the travel distance detection unit 20, a pulse signal proportional to the rotation speed of the left drive wheel 100 driven by the control of the drive device 10, that is, the rotation speed of the left travel motor 111 is generated. Thus, it is proportional to the rotation speed of the left encoder 21 that detects the travel distance that the mobile robot 1 has moved to the right side, and the right drive wheel 100 that is driven by the control of the drive device 10, that is, the rotation speed of the right travel motor 121. A right encoder 22 that detects a travel distance that the mobile robot 1 has moved to the left side by generating a pulse signal, and a travel distance that the mobile robot 1 has moved to the right side and a travel distance that has moved to the left side; Based on the above, the travel distance of the mobile robot 1 is detected and travel distance data is output.

  The direction angle detection unit 30 detects a change in the traveling direction of the mobile robot 1 moved by the driving device 10 in the mobile robot 1 and outputs traveling direction data. For example, the number of rotations of the left driving wheel 100 from the left encoder 21 is integrated to obtain the amount of movement of the left driving wheel 100, and the number of rotations of the right driving wheel 100 from the right encoder 22 is integrated to calculate the amount of movement of the right driving wheel 100. Then, the travel direction change of the robot 1 may be calculated from the information of both movement amounts, and the travel direction data may be output.

  The human movement detection unit 31 is an example of an omnidirectional image input system that is fixed to the upper end of a support column 32b erected on the rear of the robot 1, for example, as shown in FIG. A person existing around the robot 1 is detected using image data obtained by imaging with the omnidirectional optical system 32a and optical flow calculation. Specifically, from the result of the optical flow calculation, it is continuously (in the radial direction of the all-around image) within a certain angle (30 ° to 100 °, the best is about 70 °) in the all-around image. If an object is present, it is determined that the person 102 is present, and the person 102 is detected, and stereo cameras 31a and 31b disposed at the center of the robot 1, for example, at the center angle are detected. It is aimed.

  Further, as shown in FIG. 11B and the like, the robot 1 moves in front of the robot 1 using image data obtained by imaging with the stereo camera systems 31a and 31b arranged at the front portion of the robot 1, for example. The person 102 is detected, the direction angle and distance of the person 102 existing in front of the robot 1 are detected, and the direction angle and distance data of the person 102 are output. Specifically, based on the person's area detected in the entire surrounding image, the gray value (distance value) of the image of the person part in the image data obtained by imaging with the stereo camera systems 31a and 31b is averaged. Then, the position data of the person 102 (distance data between the robot 1 and the person 102) is calculated, the direction angle and distance of the person 102 are detected, and the direction angle and distance data of the person 102 are output.

  Here, the omnidirectional optical system 32a is composed of, for example, an omnidirectional camera. An omnidirectional camera uses a reflective optical system, and is composed of a single camera installed upward and a composite reflection mirror installed on the camera, and is reflected by the single camera on the composite reflection mirror. It is possible to acquire an image of all surrounding directions. Here, the optical flow is, for example, that it is not possible to determine how the robot 1 is moving only by the difference between the captured image frames captured every predetermined time by the omnidirectional optical system 32a. In order to investigate how the frame moves, it is obtained what velocity vector each point in the frame has (see FIG. 13).

  The robot movement amount detection unit 32 monitors the direction angle and the distance detected by the human movement detection unit 31 in time series, and at the same time in time series above the robot 1 by the omnidirectional optical system 32a. An image of the entire panoramic view of the moving room is captured to obtain the panoramic view image data, and the robot 1 is moved by the driving device 10 in accordance with the movement locus (path) of the person (while reproducing the movement path of the person). Using the movement direction data and movement distance data of the robot 1 output from the direction angle detection unit 30 and the travel distance detection unit 20, a movement amount composed of the movement direction and movement distance of the robot 1 is detected, and movement amount data Is output to the robot basic route teaching data converter 33 and the like. The movement amount of the robot 1 is obtained by, for example, accumulating the rotation speed of the left driving wheel 100 from the left encoder 21 to obtain the movement amount of the left driving wheel 100 and adding the rotation speed of the right driving wheel 100 from the right encoder 22. The amount of movement of the right drive wheel 100 is taken as the amount of movement, and the amount of movement of the robot 1 may be calculated from information on both amounts of movement.

  The robot basic route teaching data conversion unit 33 sets the detected movement amount data (moving direction and moving distance of the robot 1 itself, panoramic view image data of the omnidirectional optical system 32a—the teaching result of FIG. 9B) in time series. Accumulated, converted into basic route teaching data, and output to the robot basic route teaching data storage unit 34 and the like.

  The robot basic route teaching data accumulation unit 34 accumulates the robot basic route teaching data output by the robot basic route teaching data conversion unit 33 and outputs it to the movable range calculation unit 35 and the like.

  The movable range calculation unit 35 is arranged, for example, on both sides of the front part of the robot 1 while autonomously moving the robot 1 based on the robot basic route teaching data stored in the robot basic route teaching data storage unit 34. Using the obstacle information calculated by detecting the position of the obstacle 103 by the obstacle detection unit 36 such as an ultrasonic sensor, the robot 1 moves to the basic route 104 without being obstructed by the obstacle 103 or the like. The data of the range (movable range) 104a that can be moved in the width direction, that is, the movable range data is calculated and output to the movement route generation unit 37 and the like.

  The movement route generation unit 37 generates and outputs an optimal movement route for the robot 1 from the movable range data output by the movable range calculation unit 35.

  Further, in FIG. 1A, the travel distance data detected by the travel distance detector 20 and the travel direction data detected by the direction angle detector 30 are input to the control unit 50 at predetermined time intervals. Then, the control unit 50 calculates the current position of the mobile robot 1 based on the travel distance data and the travel direction data input to the control unit 50, and the current position of the mobile robot 1 obtained from the calculation result is calculated. The drive unit 10 is driven and controlled by the control unit 50 based on the movement path output from the movement path generation unit 37, so that the mobile robot 1 can accurately reach the target point without departing from the movement path that is a normal path. A control unit 50 is configured as a central processing unit (CPU) that performs operation control so that the vehicle can run. Note that the control unit 50 also controls the operation of other parts.

  Hereinafter, the operation and effect of the positioning device and method of the mobile robot 1 configured as described above, and the control device and method thereof will be described.

  FIG. 3 shows a basic route direct teaching method of the mobile robot 1. In this method for directly teaching the basic route of the robot 1, the robot 1 moves along the basic route 104 when the robot 1 follows the person 102 that travels along the basic route. Accumulated in the basic route teaching data accumulating unit 34, based on the accumulated position information, the robot basic route teaching data converting unit 33 converts the basic route teaching data into the basic route teaching data of the robot 1 and accumulates the basic route teaching data. The data is accumulated in the unit 34.

  As a specific method of this basic route direct teaching method, while detecting the person 102 existing around the robot 1 using, for example, the omnidirectional image input system 32a of the movement detection unit 31 of the person 102, the robot 1 By driving the driving device 10, the robot 1 moves so as to follow the person 102 walking along the basic route 104. That is, the distance between the robot 1 and the person 102 is within an allowable range (for example, a distance range in which the person 102 does not come into contact with the robot 1 and the person 102 does not deviate from the image acquired by the omnidirectional image input system 32a. ) When the robot 1 is moving while the control unit 50 drives and controls the driving device 10 of the robot 1, the position information at the time of robot movement is accumulated and accumulated in the basic route teaching data accumulation unit 34. And the robot basic route teaching data. In FIG. 3, reference numeral 103 denotes an obstacle.

  FIG. 4 shows the movable range additional learning by the robot 1. The movable range additional learning means learning by adding a movable range, which is a movable range when avoiding the obstacle 103, in relation to the robot basic route teaching data.

  Based on the robot basic route teaching data by the person 102, for example, obstacle information such as the position and size of the obstacle detected by the obstacle detection unit 36 such as an ultrasonic sensor is used. When an object 103 is likely to be faced, the controller 50 is controlled by the control unit 50 each time, causing the robot 1 to avoid the obstacle 103 before the obstacle 103, and again causing the robot 1 to enter the basic path 104. Move autonomously along. Each time the robot 1 is likely to face the obstacle 103 or the like, the robot basic route teaching data is repeatedly expanded along the traveling floor surface 105 of the robot 1 each time. The final expanded plane obtained by final expansion is stored in the basic route teaching data storage unit 34 as the movable range 104a of the robot 1.

  FIG. 5 shows generation of a movable range route by the robot 1.

  The movement path generation unit 37 generates a movement path 106 that is optimal for the robot 1 from the movement range 104 a obtained by the movement range calculation unit 35. Basically, the central portion in the width direction of the plane of the movable range 104 a is generated as the movement path 106. Specifically, first, the width direction component 106a of the plane of the movable range 104a is extracted at predetermined intervals, and then the central portion of the width direction component 106a is connected to generate the movement path 106.

  Here, in the follow-up teaching, when only the movement direction information of the person 102 who is the teaching is used and the position of the person 102 is followed as it is, as shown in FIG. In order to avoid 107b, the route 107a is turned around 90 degrees. As a result, as shown in FIG. 6B, if the robot 1 operates to select the shortcut 107c in the vicinity of the corner, the robot 1 moves in the place 107b where the baby is present, and correct teaching is performed. There are cases where it is not possible.

  Therefore, in the first embodiment, by using the information on both the moving direction and the moving distance of the person 102, the system shown in FIG. 6C is used to control the route 107a that is the operator's route as the route 107d of the robot 1. Realize (see FIG. 6B).

  6C includes an operator relative position detection sensor 61 corresponding to the omnidirectional optical system 32a and stereo camera systems 31a and 31b, the travel distance detection unit 20, the direction angle detection unit 30, the control unit 50, and the like. The robot current position detection unit 62 configured as described above detects the position of the person 102 who is the operator with respect to the teaching path, and stores the path of the person 102 in the path database 64. Further, a teaching path is formed based on the current position of the robot 1 detected by the robot current position detecting unit 62, and the formed teaching path is stored in the teaching path database 63. Based on the current position of the robot 1 detected by the robot current position detection unit 62, the route of the person 102 stored in the route database 64, and the taught route stored in the taught route database 63, the follow-up to the person 102 is followed. A path is formed, and the control unit 50 controls the drive device 10 to move the robot 1 so that the robot 1 moves along the formed follow-up path.

  More specific movement of the robot 1 is as follows.

  (1) The robot 1 picks up and detects the travel position of the robot 1 and the current relative position of the operator 102 with the omnidirectional optical system 32a, generates a path traveled by the person 102 who is an operator, and routes the generated path. The data is stored in the database 64 (FIG. 6B).

  (2) The route of the operator 102 stored in the route database 64 is compared with the current route of the robot 1 to determine the traveling direction (moving direction) of the robot 1, and the determined traveling direction (moving direction) of the robot 1 ), The drive unit 10 is driven and controlled by the control unit 50, and the robot 1 follows the person 102.

  As described above, the robot 1 follows the person 102, accumulates position information when the robot moves, in the basic route teaching data storage unit 34, and moves the basic route teaching data based on the accumulated position information to a movable range calculation unit 35. The method in which the robot 1 autonomously moves along the basic route 104 by controlling the drive device 10 by the control unit 50 based on the generated basic route teaching data is referred to as “playback navigation”. .

  When the content of this playback type navigation is described again, when the robot 1 moves so as to follow a person, the robot 1 learns the basic route 104 on which the robot 1 can move safely, and the robot 1 moves autonomously. The robot 1 autonomously moves in playback along the learned basic route 104.

  As shown in the operation flow of the playback type navigation in FIG. 7, the operation of the playback type navigation is composed of two steps S71 and S72.

  The first step S71 is a step for teaching a person-following basic route. In this step S71, before the robot 1 autonomously moves, the person 102 teaches the robot 1 the basic route, and at the same time, surrounding landmarks and target points when the robot 1 moves are also stored from the omnidirectional camera 32a. The data is taken into the unit 34 and stored. Next, based on the taught route / position information stored in the basic route teaching data storage unit 34, the movable range calculation unit 35 of the robot 1 generates a basic route 104 composed of map information. The basic route 104 configured by the map information here is configured by odometry-based point and line information obtained from the driving device 10.

  The second step S72 is a playback type autonomous movement step. In this step S72, a safety ensuring technique (for example, the obstacle 103 and the robot 1 are not in contact with each other and the obstacle 103 is detected so as not to contact the obstacle 103 detected by the obstacle detector 36). The robot 1 autonomously moves while avoiding the obstacle 103 by using a technique such as the control unit 50 driving and controlling the driving device 10 so that the robot 1 moves along a path separated by a sufficiently safe distance. Based on the information of the basic route 104 that the person 102 taught to the robot 1 and stored in the basic route teaching data storage unit 34 before the robot 1 autonomously moves, the route information when the robot 1 moves (for example, the robot 1 The route change information that the obstacle 103 is newly avoided) is added each time the additional route information is newly generated by the movable range calculation unit 35 by the robot 1 avoiding the obstacle 103 or the like. The information is added to the map information stored in the basic route teaching data storage unit 34. In this way, while the robot 1 moves along the basic route 104, from the map information of points and lines (basic route 104 constituted by points and lines), the map information of planes (basic route constituted by points and lines). 104 is grown to a movable range (additional route information) 104a in the width direction (direction orthogonal to the robot movement direction) and stored in the basic route teaching data storage unit 34. .

  Hereinafter, the two steps S71 and S72 will be described in detail.

(Step S71, that is, person-following basic route teaching (person-following basic route learning))
FIG. 8A to FIG. 8C show the steps of the person following basic route teaching. Before the robot 1 moves autonomously, the person 102 teaches the basic route 104 to the robot 1, and at the same time, surrounding landmarks and target points when the robot 1 moves are also transferred from the omnidirectional camera 32 a to the basic route teaching data storage unit 34. Capture and memorize. In the teaching of the basic route 104, for example, the person 102 walks on the safe route 104P and moves the robot 1 to follow the person 102 so that the person 102 does not touch the desk 111, the shelf 112, the wall 113, or the like in the room 110. . That is, the control unit 50 drives the driving device 10 to move the robot 1 so that the person 102 is always located at a certain portion of the omnidirectional camera 32a. Robot odometry information (for example, information on the number of revolutions of the left driving wheel 100 from the left encoder 21 and information on the number of revolutions of the right driving wheel 100 from the right encoder 22) when the person 102 follows the person 102 is obtained. Then, the accumulated values are accumulated from the driving device 10 to the basic route teaching data accumulating unit 34 and stored as information on the respective movement amounts. Based on the teaching route / position information stored in the basic route teaching data storage unit 34, the movable range calculation unit 35 of the robot 1 generates a basic route 104 composed of map information. The basic route 104 composed of the map information here is composed of odometry-based point and line information.

  The person-following basic route teaching uses human position detection by the omnidirectional camera 32a. First, the person 102 is detected by extracting an image corresponding to the person from the image of the omnidirectional camera 32a as viewed from the robot 1, and the direction of the person 102 approaching the robot 1 is extracted from the image of the omnidirectional camera 32a. Next, the robot 1 follows the person 102 while detecting the direction of the person 102 viewed from the robot 1 and the distance between the robot 1 and the person 102 using the stereo cameras 31a and 31b. In other words, for example, the person 102 is always located within a predetermined area of the images obtained by the stereo cameras 31a and 31b, and the distance between the robot 1 and the person 102 is always within an allowable range (for example, the person 102 is the robot 1 The robot 1 follows the person 102 while controlling the driving device 10 of the robot 1 with the control unit 50 so that the person falls within the range of a distance that does not contact the user and the person 102 does not come off from the camera image. Furthermore, the allowable width (movable range 104a) of the basic route 104 is detected by the obstacle detection unit 36 such as an ultrasonic sensor when the basic route is taught.

  As shown in FIGS. 9A to 9B, the entire view of the ceiling 114 of the room 110 is taken into the storage unit 51 by the omnidirectional camera 32a in time series while learning the person-following basic route. Saved together with odometry information at the time of image capture.

(Step S72, ie, playback type autonomous movement)
FIG. 10 shows the autonomous movement of the playback robot 1. Using the safety ensuring technique, the robot 1 moves autonomously while avoiding the obstacle 103. The route information for each movement is based on the information on the basic route 104 that the person 102 taught to the robot 1 and stored in the basic route teaching data storage unit 34 before the autonomous movement of the robot 1. For the route information (for example, route change information indicating that the robot 1 has newly avoided the obstacle 103), additional route information is newly generated by the movable range calculation unit 35, for example, by the robot 1 avoiding the obstacle 103. Each time, additional route information is added to the map information stored in the basic route teaching data storage unit 34. In this way, while the robot 1 moves along the basic route 104, from the map information of points and lines (basic route 104 constituted by points and lines), the map information of planes (basic route constituted by points and lines). 104 is grown to a movable range (additional route information) 104a in the width direction (direction orthogonal to the robot movement direction) and stored in the basic route teaching data storage unit 34. .

  For the position correction of the robot 1 during autonomous movement, the ceiling whole view image and odometry information are used in a time series acquired at the same time when the person following basic route is taught. When the obstacle 103 is detected by the obstacle detection unit 36, a route is locally calculated by the movable range calculation unit 35 and generated by the movement route generation unit 37, and the robot is generated along the generated local route 104L. The driving device 10 is controlled by the control unit 50 so that 1 moves, and the detected obstacle 103 is avoided, and then the original basic route 104 is restored. When a plurality of routes are generated during the local route calculation and generation by the movable range calculation unit 35 and the movement route generation unit 37, the shortest route is selected by the movement route generation unit 37. The route information when the avoidance route is calculated and generated locally (route information related to the local route 104L) is added to the basic route (map) information in step S71 stored in the basic route teaching data storage unit 34. As a result, the robot 1 autonomously moves along the basic route 104, and from the map information of points and lines (basic route 104 composed of points and lines), the map information of planes (basic route 104 composed of points and lines). To the movable range in the width direction (direction orthogonal to the robot movement direction) (additional route information) 104a is added to the range and stored in the basic route teaching data storage unit 34. At this time, for example, in the case where the detected obstacle 103 is a person 103L such as a baby or an elderly person, it may be unexpectedly moved further to the surroundings, so the local route 104L is generated so as to make a large detour. Is desirable. Considering the case where the robot 1 transports liquid or the like, when the robot 1 moves in the vicinity of a precision device 103M such as a television, which is easily damaged if liquid or the like is accidentally spilled, the vicinity of the precision device 103M is slightly increased. It is desirable to create a local path 104M that curves and moves away.

  According to the first embodiment described above, the person 102 walks on the basic route 104 and follows the person 102 to move the basic route 104 so that the robot 1 teaches the basic route 104 to the robot 1. When the robot 1 autonomously moves along the basic route 104, the movable range for avoiding the obstacle 103 and the like is calculated, and the robot 1 can actually move autonomously by the basic route 104 and the movable range. A simple movement path 106 can be generated.

  For example, the robot 1 can be used to carry luggage at home. The robot is waiting at the entrance of the house, and when the operator returns home, The robot receives the parcel, and at the same time, the robot recognizes the person who placed the parcel as a target to follow, and the robot follows the person and carries the parcel. At this time, in the path of movement of the robot in the home, many obstacles often exist unlike the public place. However, the robot according to the first embodiment having means for avoiding an obstacle can avoid the obstacle by following the movement path of the person. This is because the route that the robot 1 of the first embodiment has taken as the travel route is the travel route that the operator 102 has passed immediately before, and thus there is a low probability that an obstacle exists. If an obstacle appears after the person 102 passes, the obstacle detection unit 36 mounted on the robot 1 can detect the obstacle and the robot 1 can avoid the detected obstacle. It is.

(Second Embodiment)
Next, in the robot control apparatus and method according to the second embodiment of the present invention, a moving body detection apparatus (corresponding to the human movement detection unit 31) and method for detecting a moving body will be described with reference to FIGS. 12A to 18D. Detailed description. The “moving body” referred to here refers to a person, an animal, or an automatically moving device (such as a robot capable of autonomous movement, an autonomous traveling cleaner). In addition, the moving body detection device 1200 is newly added to the robot control device according to the first embodiment (see FIG. 1A), but some of the components are the same as those of the robot control device according to the first embodiment. It can also be used as a component.

  The mobile object detection apparatus and method uses an optical flow.

  Before explaining the moving body detection apparatus and method, first, an explanation of the optical flow is shown in FIG.

  Normally, as shown in the corresponding points 201 in FIG. 13, the optical flow is arranged with corresponding points 201 for calculating the flow in the form of grid points on the entire screen to be processed, and each corresponding on a plurality of images that are continued in time series. Each point corresponding to the point 201 is detected, the amount of movement between the points corresponding to each corresponding point 201 is detected, and the flow for each corresponding point 201 is obtained. For this reason, the corresponding points 201 are arranged in lattice points at equal intervals on the entire screen, and a flow calculation is performed for each corresponding point 201, which results in a huge amount of calculation. In addition, the greater the number of corresponding points 201, the higher the possibility of noise flow detection and the easier it is to detect errors.

  Here, when using the omnidirectional optical system 32a, as shown in the omnidirectional camera image 302 of FIG. 16A and the corresponding point diagram of FIG. A flow other than the optical flow generated by the movement of the person and the object at the corresponding point 301 in the center portion of the image that becomes the region 302a and the corresponding point 301 in the region 302b at the four corners of the image that is outside the field of view of the omnidirectional camera. (Noise flow) is likely to occur.

  Therefore, in the moving body detection apparatus and method of the robot control apparatus and method according to the second embodiment of the present invention, the area where the corresponding points (flow calculation points) 301 are arranged is not evenly arranged in the entire screen area. The calculation amount and the calculation cost are reduced by limiting to a specific region of the moving body approaching the robot 1. An example of an image when an omnidirectional camera is used as an example of the omnidirectional optical system 32a is shown in FIG. 14A, and various arrangement examples of corresponding points with respect to the image are shown in FIGS. 14B to 14E.

  In the case of an omnidirectional camera image 302 captured using an omnidirectional camera (FIG. 14A), the person 102 appears on the outer circumference of the image 302. For this reason, the corresponding points 301 in FIG. 14B are arranged on the outer circumference of the image 302. As a specific example, in the case of an omnidirectional camera image having a size of 256 × 256 pixels, the omnidirectional camera image is arranged at an interval of about 16 pixels on the outer circumference of the omnidirectional camera image. Further, the foot of the person 102 appears on the inner circumference of the omnidirectional camera image 302, and an inappropriate noise flow tends to appear for position detection. Further, in the case of the omnidirectional camera image 302, the four corners of the omnidirectional camera image 302 are outside the significant image range, and the flow calculation using the corresponding points is unnecessary. Therefore, as shown in FIG. 14B, when the flow corresponding point 301 is arranged on the outer circumference of the omnidirectional camera image 302 and the flow corresponding point 301 is not arranged on the inner circumference, both the calculation cost and the noise countermeasures are provided. Is advantageous.

  Further, as shown in FIG. 14C, when detecting the person 102 only to the person 102 approaching the robot 1, the flow corresponding point 311 is arranged only on the outermost circumference of the omnidirectional camera image 302. Also good.

  Furthermore, as shown in FIG. 14D, when detecting both the approach of the person 102 and the approach of the obstacle 103, the flow corresponding points only on the outermost circumference and the innermost circumference of the omnidirectional camera image 302. 312 may be arranged.

  Further, as shown in FIG. 14E, for high-speed calculation, the flow correspondence points are arranged in a grid pattern at equal intervals in the x direction (horizontal direction in FIG. 14E) and the y direction (vertical direction in FIG. 14E) of the omnidirectional camera image 302. 313 may be arranged. As a specific example, in the case of an omnidirectional camera image having a size of 256 × 256 pixels, the omnidirectional camera image is arranged at an interval of 16 pixels in the x direction and 16 pixels in the y direction.

  As shown in FIG. 12A, a moving object detection apparatus 1200 for performing the moving object detection method of the second embodiment includes a flow calculation corresponding point position calculation arrangement unit 1201, a time-series image acquisition unit 1202, and a movement amount calculation unit. 1203, a moving object movement determination unit 1204, a moving object region extraction unit 1205, a distance image calculation unit 1206a, a distance image specific range moving unit 1206b, a moving object region determination unit 1206c, and a moving object position specification unit 1206d The distance calculation unit 1206e and the storage unit 51 are provided.

  The flow calculation corresponding point position calculating / arranging unit 1201 captures the omnidirectional camera image 302 shown in FIG. 14A captured by the omnidirectional camera, which is an example of the omnidirectional optical system 32a, into the storage unit 51 in time series. In the case of a sequence image, as shown in FIG. 14B, FIG. 14C, FIG. 14D, or FIG. 14E, the position of the significant corresponding point 301, 311, 312, or 313 is calculated on the circumference of the omnidirectional time-series image. . The flow calculation corresponding point position calculation / arrangement unit 1201 further includes a corresponding point position calculation / arrangement changing unit 1201a, and detects (flow calculation) corresponding point positions which are calculated and arranged in advance in accordance with the movement of the moving body, for example, the person 102. In accordance with the position of the moving object, for example, the person 102, the corresponding point position calculation / arrangement changing unit 1201a may change the position each time. In this way, it is further advantageous in terms of both computation cost and noise countermeasures. Further, the movement amount calculation unit 1203 calculates an optical flow for each of the corresponding points 301, 311, 312, or 313 of the flow calculation corresponding point position calculated by the flow calculation corresponding point position calculation arrangement unit 1201.

  The time-series image acquisition unit 1202 captures images at an appropriate time interval for optical flow calculation. For example, the omnidirectional camera image 302 captured by the omnidirectional camera is input and stored in the storage unit 51 by the time-series image acquisition unit 1202 every several hundred ms. When using the omnidirectional camera image 302 imaged by the omnidirectional camera, a moving body approaching the robot 1 can be detected within a range of 360 degrees around the robot 1. There is no need to move and rotate the moving object detection camera. In addition, even if a moving body (for example, a person who teaches a basic route) that is to be followed without time delay approaches the robot 1 from any direction, the moving body can be easily and reliably detected. It becomes possible.

  The movement amount calculation unit 1203 corresponds between the time-series images acquired by the time-series image acquisition unit 1202 for each of the corresponding points 301, 311, 312, or 313 calculated by the flow calculation corresponding point position calculation arrangement unit 1201. The positional relationship between the points 301, 311, 312, or 313 is detected. Usually, template matching is performed on the new time-series image of the time-series image using the corresponding point block consisting of the older corresponding points 301, 311, 312, or 313 of the time-series image as a template. The matching point information obtained by this template matching and the positional deviation information of the corresponding point block position of the older image are calculated as a flow. The degree of matching at the time of template matching is also calculated as flow calculation information.

  The moving body movement determination unit 1204 determines whether the calculated movement amount matches the movement of the moving body for each of the corresponding points 301, 311, 312, or 313. More specifically, the moving body movement determination unit determines whether the moving body has moved using position shift information of the corresponding point block position and flow calculation information. In this case, if the same determination criterion is used for all screens, for example, in the case of a person's foot and head as an example of a moving body, the possibility of detecting the person's foot as a flow increases. However, in practice, it is necessary to prioritize the human head and detect the flow. For this reason, in the second embodiment, a different criterion is used for each of the corresponding points 301, 311, 312, or 313 arranged by the flow calculation corresponding point position calculation arrangement unit 1201. As one example, as a flow determination standard for an omnidirectional camera image, the radial flow is set on the inner side of the omnidirectional camera image (specifically, the radius ¼ (lateral image size) from the center of the omnidirectional camera image). In addition, on the outside of the omnidirectional camera image (specifically, outside the radius ¼ (horizontal image size) from the center of the omnidirectional camera image), a bi-directional flow of radiation and concentric circles is left.

  The moving body region extraction unit 1205 extracts the moving body region by integrating the corresponding corresponding points. That is, the moving body region extraction unit 1205 integrates the flow corresponding points that have been determined to pass by the moving body movement determination unit 1204 that determines whether or not the calculated movement amount matches the movement of the moving body for each corresponding point. Extract as a region. As illustrated in FIG. 15A, a region surrounding the corresponding point group 401 determined to be accepted by the moving body movement determination unit 1204 is extracted by the moving body region extraction unit 1205. As shown in FIG. 15B, a trapezoidal region surrounded by reference numeral 402 is an example of the extracted moving body region (a human region in the case of a person as an example of a moving body). 15C uses the panorama development image 502 of the human area of the person 102, and extracts the head from the specific gray scale image (indicated by 503 in the vertical direction and 504 in the horizontal direction) of the human area panorama development image 502. It is explanatory drawing of the result. A region from which the head is extracted is indicated by 501.

  The distance image calculation unit 1206a calculates a distance image of a specific range around the robot 1 (distance image specific range). The distance image is an image that brightly represents an object closer to the robot 1. The specific range is set in advance as, for example, a range of ± 30 ° (determined by experience values; variable depending on the sensitivity of the sensor or the like) with respect to the front of the robot 1.

  The distance image specific range moving unit 1206b includes a range of the human area 402 so that the range of the distance image specific range matches the range of the human area 402 as an example of the moving body area extracted by the moving body area extracting unit 1205. The distance image specific range is moved in accordance with the movement of. Specifically, as shown in FIGS. 18A to 18D, the distance image specifying the angle between the line connecting the center of the human region 402 surrounding the corresponding point group 401 and the center of the image and the forward direction of the robot 1 in FIG. 18A. By obtaining the value by the range moving unit 1206b, the direction angle α of the person 102 indicated by reference numeral 404 in FIG. 18A (the forward direction of the robot 1 in FIG. 18A is set to a reference angle 0 ° of the direction angle) can be calculated. Corresponding to the calculated angle (direction angle) α, the distance image of the specific range around the robot 1 (distance image specific range) is moved so that the distance image specific range matches the range of the human region 402. . In order to move the distance image specifying range so as to match the range of the human region 402 in this way, specifically, the drive device 10 rotates both the drive wheels 100 in the reverse direction, and the robot 1 on the spot. Is rotated by the angle (direction angle) α so that the distance image specifying range matches the range of the extracted human region 402. Thereby, in front of the robot 1 that is within the range that can be imaged by the stereo camera systems 31a and 31b, which are examples of the distance image detection sensor constituting the part of the distance image calculation unit 1206a, in other words, the distance image specific range. The detected person 102 can be positioned.

  The moving body region determination unit 1206c determines a moving body region within the distance image specific range after being moved by the distance image calculation unit 1206a. For example, the image with the largest area among the grayscale images within the distance image specifying range is determined as the moving body region.

  The moving body position specifying unit 1206d specifies the position of the moving body, for example, the person 102 from the obtained distance image moving body area. As a specific example of specifying the position of the person 102, in FIG. 18C, the position of the person 102 is calculated by calculating the position of the center of gravity of the person area (the intersection 904 of the cross line in FIG. 18C) as the position and direction of the person 102. Can be specified.

  The distance calculation unit 1206e calculates the distance from the robot 1 to the person 102 based on the position of the moving object such as the person 102 on the distance image.

  Using the mobile body detection apparatus 1200 according to the above configuration to perform a mobile body detection operation will be described with reference to FIGS. 18A to 19.

  First, in step S191, a distance image is input to the distance image calculation unit 1206a. Specifically, the following operation is performed. In FIG. 18A, as a specific example of the distance image calculation unit 1206a, a distance image detection sensor that calculates a distance image (distance image specific range) around the robot 1 and a distance detected by the distance image detection sensor. A distance image calculation unit that calculates an image can be used. It is assumed that the person 102 is positioned in a specific range around the robot 1 by the moving body movement determination unit 1204, the direction angle 404 of the person 102, and the distance image specific range movement unit 1206b. If the stereo cameras 31a and 31b having parallel optical axes are used as an example of the distance image detection sensor, the corresponding points of the left and right stereo cameras 31a and 31b appear on the same scanning line, and high-speed corresponding point detection and three-dimensional distance calculation are performed by the distance image. This can be done by the calculation unit 1206a. A distance image which is a corresponding point detection result of the left and right stereo cameras 31a and 31b and a three-dimensional distance calculation result by the distance image calculation unit 1206a is shown by 902 in FIG. 18D. In the distance image 902, the stereo cameras 31 a and 31 b express a brighter object closer to the robot 1.

  Next, in step S192, an object close to the robot 1 (in other words, an area having a constant brightness (brightness exceeding a predetermined threshold) on the distance image 902) is detected as a human area. An image obtained by binarizing the detection result is the image 903 in FIG. 18D detected as a human region.

  Next, in step S193, the distance calculation unit 1206e corresponds to the human region of the image 903 detected as the human region in FIG. 18D (when the grayscale image is binarized with “0” and “1”, it corresponds to the region where a person is present). The distance image 902 is masked (an AND operation between images) in the “1” area), and the distance image of the human area is specified by the distance calculation unit 1206e. The distance of the person area (the density value (distance value) of the person area) is averaged by the distance calculation unit 1206e to obtain the position of the person 102 (distance between the robot 1 and the person 102).

  Next, in step S194, the gray value (distance value) obtained by averaging is substituted into the distance value-actual distance value conversion table of the distance calculation unit 1206e, and the distance L between the robot 1 and the person 102 is calculated. Is calculated by the distance calculation unit 1206e. As an example of the distance value-actual distance value conversion table, FIG. 17 shows an image 801 in which panoramic development images of omnidirectional camera images with different distances to the person 102 are arranged. When the distance is 100 cm, ± 2.5 cm / pixel if 50 cm / 10 pixel, when the distance is 200 cm, ± 5 cm / pixel, when the distance is 300 cm, ± 50 cm / 2.5 pixel, when the distance is 300 cm 10 cm / pixel.

  Next, in step S195, the center of gravity position of the human region in the image 903 is calculated by the moving body position specifying unit 1206d. The x-coordinate of the center of gravity position is substituted into the center-of-gravity x-coordinate-human direction conversion table of the moving body position specifying unit 1206d, and the human direction β is calculated by the moving body position specifying unit 1206d.

  Next, in step S196, the distance L between the robot 1 and the person 102 and the person's direction β are sent to the robot movement amount detection unit 32 of the control device according to the first embodiment to reproduce the movement path of the person 102. Further, a movement amount consisting of the movement direction and movement distance of the robot 1 is detected.

  Next, in step S197, the robot 1 is moved according to the detected movement amount including the movement direction and movement distance of the robot 1. Odometry data at the time of robot movement is stored in the robot basic path teaching data conversion unit 33.

  Thus, by repeating the calculations in steps S191 to S197, the person 102 can be specified, and the distance L between the robot 1 and the person 102 and the direction β of the person can be detected continuously.

  In addition, as a method for confirming the operation in which the person 102 teaches the follow-up running (see 38 in FIG. 1B), the knowledge that the person 102 who is closest and contacts the robot 1 is the person to be followed (target). Based on the above, it may be incorporated as a teaching subject identification unit 38 when a plurality of subjects are present (see FIG. 12B). Specific examples of the teaching subject identification unit 38 include pressing a switch installed at a specific position of the robot 1, generating a voice and estimating a sound source position by a voice input device such as a microphone. Alternatively, there is an example in which a person 102 possesses an ID resonance tag and the like, and the robot 1 is equipped with a reading device capable of reading the ID resonance tag to test a specific person. As a result, as shown in FIG. 12B, a robot follower can be specified and used as a trigger for starting the robot follower operation at the same time.

  As described above, according to the second embodiment, by detecting the distance and direction of the person 102 from the robot 1 with the above-described configuration, a correct route cannot be generated when the position of the person 102 is followed as it is in the follow-up running. The case can be avoided (see FIGS. 6A and 6B). This is a case where only the direction of the person 102 is detected. For example, when the path 107a of the person 102 becomes a right angle corner as shown in FIG. 6A, the robot 1 does not follow the person 102 at a right angle and generates a shortcut as the path 107c. There are things to do. In the second embodiment of the present invention, by obtaining the direction and distance of the person 102, the control of setting the path 107a of the person 102 at the right corner to the path 107d of the robot 1 is realized (see FIG. 6B). A procedure for generating such a path 107d will be described below.

  (1) The robot 1 sees its own traveling route and the current relative position of the operator 102, generates a route 107a of the operator 102, and stores this (FIG. 6B).

  (2) The stored route 107a of the operator 102 is compared with the current route 107d of the robot 1 to determine the traveling direction of the robot 1, and the robot 1 follows the operator 102. Can move along the path 107d of the robot 1 along the line.

  According to the configuration of the second embodiment, the mobile object, for example, the person 102 who teaches the route is specified, and the human 102 is continuously detected (for example, the distance between the robot 1 and the person 102 and the robot 1 Detection of the direction of the person 102 viewed from the above.

(Third embodiment)
Next, the robot positioning apparatus and method of the robot control apparatus and method according to the third embodiment of the present invention will be described in detail with reference to FIGS. The robot control device according to the following third embodiment is newly added to the robot control device according to the first embodiment, but some of the components are the configuration of the robot control device according to the first embodiment. It is also possible to combine elements.

  As shown in FIG. 20A, the robot positioning device of the robot control device according to the third embodiment is roughly configured by a drive device 10, a travel distance detection unit 20, a direction angle detection unit 30, and a positional deviation information calculation unit 40. The control unit 50 is provided. 20A, the members such as the movement detection unit 31 to the movement path generation unit 37, the teaching subject identification unit 38, and the moving body detection device 1200 illustrated in FIG. 1A in the robot control apparatus of the first embodiment are illustrated. Is omitted. FIG. 1A also shows the positional deviation information calculation unit 40 and the like in order to show the robot control apparatus according to the first to third embodiments.

  The drive device 10 of the robot control apparatus according to the third embodiment controls the forward and backward movement and the left / right movement of the robot 1 which is an example of an autonomous vehicle, and the robot control according to the first embodiment. This is the same as the driving device 10 of the apparatus. A more specific example of the robot 1 is an autonomous traveling type vacuum cleaner.

  Furthermore, the travel distance detection unit 20 of the robot control device according to the third embodiment is the same as the travel distance detection unit 20 of the robot control device according to the first embodiment, and is moved by the drive device 10. The travel distance of the robot 1 is detected.

  Further, the direction angle detection unit 30 detects a change in the traveling direction of the robot 1 moved by the driving device 10 in the same manner as the direction angle detection unit 30 of the robot control device according to the first embodiment. The direction angle detection unit 30 is a direction angle sensor such as a gyro sensor that detects a change in the traveling direction by detecting the rotation speed of the robot 1 according to a voltage level that changes when the robot 1 moved by the driving device 10 rotates. .

  The misregistration information calculation unit 40 detects target points to the ceiling 114 and the wall surface 113 that exist on the movement route (basic route 104) of the robot 1 that is moved by the driving device 10, and stores a keyboard or a touch panel in the storage unit 51 in advance. Misalignment information with respect to the target point input from the input / output unit 52, such as the misalignment information calculation unit 40. The misalignment information calculation unit 40 is a target to the ceiling 114 and the wall surface 113 existing in the movement path of the robot 1. An omnidirectional camera unit 41 attached to the robot 1 for detecting a point is provided. The input / output unit 52 also includes a display device such as a display for appropriately displaying necessary information such as a target point and for confirmation by a person.

  The robot-attached omnidirectional camera unit 41 of the positional deviation information calculation unit 40 detects a target point to the ceiling 114 and the wall surface 113 that exist on the movement path of the robot 1, and is attached to the robot-attached omnidirectional camera 411 (see FIG. 1B). The height of the omnidirectional camera processing unit 412 that inputs and processes the image of the omnidirectional camera 411, and the robot-attached omnidirectional camera 411 from the floor surface 105 on which the robot 1 travels is detected. The camera height detection unit 414 (for example, an upward ultrasonic sensor) and the omnidirectional camera 411 are arranged toward the ceiling 114 and the wall surface 113 so that the support 1b of the robot 1 has a motor or the like. The height of the omnidirectional camera 411 fixed to the bracket screwed to the ball screw by rotating the ball screw by driving can be adjusted (movable up and down). It is configured to include a spherical camera height adjustment portion 413 to place.

  In FIG. 20A, the control unit 50 is input with the travel distance data detected by the travel distance detection unit 20 and the travel direction data detected by the direction angle detection unit 30 at predetermined time intervals. 1 is calculated by the control unit 50, and positional shift information for the target point to the ceiling 114 and the wall surface 113 calculated by the positional shift information calculation unit 40 is input to the control unit 50, and according to the positional shift information result By controlling the driving device 10 by the control unit 50 and controlling the movement path of the robot 1, the center is controlled so that the robot 1 can accurately travel to the target point without deviating from the normal path (that is, the basic path). A processing unit CPU.

  Hereinafter, the operation and effect of the positioning apparatus and method for the robot 1 configured as described above, and the control apparatus and method thereof will be described. FIG. 21 is a control flowchart of the mobile robot positioning method according to the third embodiment.

  As shown in FIG. 20B, the omnidirectional camera processing unit 412 includes a conversion extraction unit 412a, a conversion extraction storage unit 412f, a first cross-correlation matching unit 412b, a rotation angle deviation amount conversion unit 412c, and a second mutual correlation. A correlation matching unit 412d and a positional deviation amount conversion unit 412e are configured.

  The conversion extraction unit 412a converts and extracts the panoramic image of the ceiling 114 and the wall surface 113 and the panoramic image of the ceiling 114 and the wall surface 113 from the image input by the omnidirectional camera 411 functioning as an example of an image input unit. (Step S2101).

  The conversion / extraction storage unit 412f inputs, in advance, a ceiling / wall surface full view central image and a ceiling / wall full view peripheral image from the conversion extraction unit 412a at a specified position, and converts, extracts, and stores the image (step S2102). .

  The first cross-correlation matching unit 412b includes a ceiling and wall whole view peripheral image input at the present time (when positioning operation is performed), and a ceiling and wall full view peripheral image of a specified position stored in advance in the conversion extraction storage unit 412f. And cross-correlation matching is performed (step S2103).

  The rotation angle deviation amount conversion unit 412c converts the lateral positional relationship (deviation amount) obtained by the first cross-correlation matching unit 412b into the rotation angle deviation amount (step S2104).

  The second cross-correlation matching unit 412d performs cross-correlation matching between the currently input center image of the ceiling and wall surface and the ceiling and wall surface central image of the designated position stored in advance in the conversion extraction storage unit 412f. (Step S2105).

  The positional deviation amount conversion unit 412e converts the vertical / horizontal positional relationship obtained by the matching by the second cross-correlation matching unit 412d into a positional deviation amount (step S2106).

  The omnidirectional camera processing unit 412 having such a configuration performs matching between the ceiling and wall whole view image, which is a reference for which the position and orientation are known, and the input ceiling and wall whole view image, and detects position and posture deviation (the amount of rotation angle deviation) The rotation angle deviation amount obtained by the conversion unit and the position deviation amount obtained by the position deviation amount conversion unit are detected), the self-position is recognized, and the rotation angle deviation amount and the position deviation amount are within the allowable ranges. By controlling the drive device 10 to be corrected so as to be corrected, the robot control device controls the robot 1 to move autonomously. This will be described in detail below. Here, the autonomous movement of the robot 1 refers to a person who is away from the robot 1 at a certain distance so that the distance between the robot 1 and a moving body such as the person 102 is within an allowable range. This means that the robot 1 moves following the moving body such as the person 102 along the path along which the moving body such as the person 102 moves so that the direction to the moving body such as 102 falls within the allowable range.

  First, an omnidirectional camera 411 that functions as an example of an omnidirectional image input unit and an omnidirectional camera 411 that functions as the omnidirectional image input unit are arranged so that the height can be adjusted toward the ceiling 114 and the wall surface 113. Conversion that converts and extracts the panoramic image of the ceiling 114 and the wall surface 113 and the panoramic image of the ceiling 114 and the wall surface 113 from the image input by the height adjustment unit 413 and the omnidirectional camera 411 functioning as the image input unit. And the extraction unit 412a are used to input, convert, extract, and store the reference image of the entire center of the ceiling 114 and the wall surface 113 and the image of the entire periphery of the ceiling 114 and the wall surface 113 at a specified position in advance. Reference numeral 601 in FIG. 25B denotes a reference image of the center of the ceiling and wall and a peripheral image of the entire view of the ceiling and wall, which are input, converted, extracted, and stored in advance according to the third embodiment (see FIG. 26). In the third embodiment, a PAL lens or a fisheye lens is used for the omnidirectional camera 411 that functions as an omnidirectional image input unit (see FIG. 22).

  Here, an omnidirectional camera 411, an omnidirectional camera height adjustment unit 413 that arranges the omnidirectional camera 411 toward the ceiling and the wall surface so as to be height-adjustable, and an image input from the omnidirectional camera 411 One example of the procedure using the conversion and extraction unit 412a for converting and extracting the whole wall peripheral view peripheral image is shown in the upper half of FIG. One example of an image converted and extracted by the above procedure is 401.

・・・・・・・ 式４０３

.... Formula 403

  Next, an omnidirectional camera 411, an omnidirectional camera height adjustment unit 413 that arranges the omnidirectional camera 411 so that the height of the omnidirectional camera 411 can be adjusted toward the ceiling and the wall surface, and an image input from the omnidirectional camera 411 One example of the procedure for using the conversion and extraction unit 412a for converting and extracting the whole wall center view image is shown in the lower half of FIG.

・・・・・・・ 式４０４

.... 404

・・・・・・・式４０５

.... Formula 405

  The contents of the expression 405 are shown in FIG. In FIG. 24, X, Y, and Z are the position coordinates of the object input by the omnidirectional camera 411, and x and y are the position coordinates of the converted object. Usually, since the ceiling 114 has a constant height, the value of Z is assumed to be a constant value on the ceiling 114 to simplify the calculation. In order to calculate the value of Z, the detection result by the object height detection unit (for example, the upward ultrasonic sensor) 422 and the detection result by the camera height detection unit 414 are used. This is shown in equation (4) in FIG. By this conversion, the peripheral distortion of the omnidirectional image input unit is removed, and the cross correlation matching calculation described below becomes possible.

  The conversion is derived to convert a polar coordinate image into a lattice image. By adding the term of formula (4), hemispherical / concentric circular strain change can be included. One example of the image extracted by the above procedure is 402 (FIG. 23).

  One example of the reference image of the center of the ceiling and wall surface, which is calculated in the above-described procedure and is converted, extracted, and stored in advance by inputting a designated position, is 612 in FIGS. 25C and 26. In addition, reference numeral 611 in FIG. 25B is an example of the reference ceiling and wall surface panoramic view peripheral image that has been input in advance, converted, extracted, and stored.

  At this time, an omnidirectional camera 411, an omnidirectional camera height adjustment unit 413 that arranges the omnidirectional camera 411 so as to be height-adjustable toward the ceiling and the wall surface, and an image input from the omnidirectional camera 411 Using the conversion / extraction unit 412a that converts and extracts the whole-wall-view peripheral image and the ceiling / wall-view central image, the reference ceiling / wall-view central image and the ceiling / wall-view peripheral portion at a predetermined position in advance. Input image, convert and extract. 602, 621, and 622 in FIGS. 25A and 26 are the ceiling and wall panoramic center image (622) and the ceiling that are currently input (the input image is indicated by 602), converted, extracted, and stored according to one example. And a wall surface full view peripheral image (621).

  Similar to the procedure for inputting, converting, extracting, and storing the central image of the ceiling and wall surface and the peripheral image of the ceiling and wall surface at the specified position in advance, the central image of the current ceiling and wall surface, and the ceiling and wall surface. The panoramic view peripheral image is also converted and extracted by the processing procedure of FIG.

  651 in FIG. 25C shows a situation where cross-correlation matching is performed using one example with the ceiling and wall surface whole view peripheral part image input at the present time and the ceiling and wall whole surface peripheral part peripheral image of the specified position stored in advance. A lateral displacement 631 between the reference ceiling and wall full view peripheral image 611 and the ceiling, wall full view peripheral image 621 converted, extracted, and stored indicates the posture (angle) deviation. The amount of lateral displacement can be converted into the rotation angle (posture) of the robot 1.

  An image 652 in FIG. 25C shows a situation where cross-correlation matching is performed using one example of the ceiling and wall surface panoramic center image input at the present time and the ceiling and wall surface panoramic central image of the specified position stored in advance. XY lateral displacement amounts 632 and 633 (Y-direction displacement amount 632 and X-direction displacement amount) between the ceiling and wall surface overall view center image 612 serving as a reference and the ceiling and wall surface overall view central image 622 converted, extracted, and stored. A quantity 633) indicates a positional deviation amount.

  From the two positional deviation amounts, the position and orientation deviation can be detected to recognize the self position. In addition, so-called landmarks are not used in the ceiling and wall surface full view peripheral image and the ceiling and wall full view central image.

  The reason why the ceiling is used as a reference in each of the above embodiments is that it is usually easy to assume that the ceiling has a small height and a constant height, and is easy to handle as a reference point. On the other hand, it is considered that the wall surface is difficult to handle as a reference point because it may be moved or newly furnished even if a moving object exists or furniture is arranged. .

  In the various omnidirectional camera images, the black circle shown at the center is the camera body.

  It is to be noted that, by appropriately combining any of the various embodiments, the effects possessed by them can be produced.

  Although the present invention has been fully described in connection with preferred embodiments with reference to the accompanying drawings, various variations and modifications will be apparent to those skilled in the art. Such changes and modifications are to be understood as being included therein, so long as they do not depart from the scope of the present invention according to the appended claims.

  The present invention relates to a robot control apparatus and method for generating a movable path while an autonomous mobile robot recognizes a movable range by autonomous movement. Here, the autonomous movement of the robot means that a certain distance is provided so that the distance between the robot and a moving body such as a person is within an allowable range, and the robot is moved to a moving body such as a person. This means that the robot moves following the moving body such as a person along the path along which the moving body such as a person moves so that the direction is within the allowable range. The present invention does not particularly lay a magnetic tape or a reflective tape partially on the floor surface as a guide path. For example, an autonomous mobile robot is provided with an array antenna, a person is provided with a transmitter, and the like. A robot control device that detects a direction angle in time series, moves a robot in accordance with the movement of a person, teaches a basic path by walking and follows a basic path, and generates a movable path while recognizing a movable range About. In the robot control apparatus and method according to one aspect of the present invention, the movement path range can be taught by the follow-up of the person and the robot autonomous movement. Furthermore, in the robot control apparatus and method according to another aspect of the present invention, it is also possible to identify the moving body and continue to detect the moving body (distance and direction). In the robot control apparatus and method according to still another aspect of the present invention, matching is performed between the ceiling and wall whole view image serving as a reference for which the position and orientation are known and the input ceiling and wall whole view image, and the position and posture deviation detection is performed. The position can also be recognized.

It is a control block diagram of the robot controller according to the first to third embodiments of the present invention. It is a front view of the robot controlled by the robot control apparatus by the said 1st Embodiment of this invention. It is a side view of the robot controlled by the robot control apparatus by the said 1st Embodiment of this invention. It is explanatory drawing for demonstrating the direct teaching of the basic path | route of the robot controlled by the robot control apparatus by the said 1st Embodiment of this invention. It is explanatory drawing for demonstrating learning of the movable range of the robot controlled by the robot control apparatus by the said 1st Embodiment of this invention. It is explanatory drawing for demonstrating the production | generation of the path | route within the movable range of the robot controlled by the robot control apparatus by the said 1st Embodiment of this invention. In the robot control apparatus according to the first embodiment of the present invention, it is an explanatory diagram for explaining a case where a person's direction angle is monitored in time series and a person's movement is detected from time series direction angle change data. In the robot control apparatus according to the first embodiment of the present invention, it is an explanatory diagram for explaining a case where a person's direction angle is monitored in time series and a person's movement is detected from time series direction angle change data. FIG. 5 is a block diagram for explaining a case in which a person's direction angle is monitored in time series and a person's movement is detected from time series direction angle change data in the robot control apparatus according to the first embodiment of the present invention. 4 is a flowchart showing an operation of playback navigation in the robot control apparatus according to the first embodiment of the present invention. It is explanatory drawing for demonstrating the operation | movement of a person following basic route teaching in the robot control apparatus by the said 1st Embodiment of this invention. It is a figure of the image imaged with the omnidirectional camera for demonstrating the operation | movement of a person following basic path | route teaching in the robot control apparatus by the said 1st Embodiment of this invention. In the robot control apparatus by the said 1st Embodiment of this invention, it is a figure explaining the state in which the robot for demonstrating operation | movement of a person following basic path | route teaching moves in a room. It is explanatory drawing for demonstrating the operation | movement of a person following basic route teaching in the robot control apparatus by the said 1st Embodiment of this invention. In the robot control apparatus according to the first embodiment of the present invention, in order to explain the operation of teaching a person-following basic route, it is a diagram of a time-series ceiling panoramic image of an omnidirectional optical system together with a state in which the robot moves in a room. . In the robot control apparatus by the said 1st Embodiment of this invention, in order to demonstrate playback-type autonomous movement, it is a figure of the state which a robot moves in a room, and the time series ceiling whole view image of an omnidirectional optical system. In the robot control apparatus by the said 1st Embodiment of this invention, it is a figure of the ceiling whole view image of the omnidirectional optical system in the case of a person detection. FIG. 6 is a diagram for explaining a case of following a person in the case of human detection in the robot control apparatus according to the first embodiment of the present invention. It is a control block diagram of the mobile robot controller according to the second embodiment of the present invention. It is a control block diagram of the mobile robot control apparatus by the modification of the said 2nd Embodiment of this invention. It is a figure which shows the optical flow which is a prior art. In the moving body detection method by the said 2nd Embodiment of this invention, it is a figure of the image imaged at the time of using an omnidirectional camera. In the moving body detection method according to the second embodiment of the present invention, there are various examples of corresponding point arrangements for captured images when an omnidirectional camera is used, and the moving body according to the second embodiment of the present invention. It is a figure of the state which limited the corresponding point (flow calculation point) by the detection method to the specific area | region of a person or a mobile body, and reduced calculation cost. In the moving body detection method according to the second embodiment of the present invention, there are various corresponding point arrangement examples for a captured image when an omnidirectional camera is used, and the moving body according to the second embodiment of the present invention It is a figure of the state which limited the corresponding point (flow calculation point) by the detection method to the specific area | region of a person or a mobile body, and reduced calculation cost. In the moving body detection method according to the second embodiment of the present invention, there are various examples of corresponding point arrangements for captured images when an omnidirectional camera is used, and the moving body according to the second embodiment of the present invention. It is a figure of the state which limited the corresponding point (flow calculation point) by the detection method to the specific area | region of a person or a mobile body, and reduced calculation cost. In the moving body detection method according to the second embodiment of the present invention, there are various examples of corresponding point arrangements for captured images when an omnidirectional camera is used, and the moving body according to the second embodiment of the present invention. It is a figure of the state which limited the corresponding point (flow calculation point) by the detection method to the specific area | region of a person or a mobile body, and reduced calculation cost. In the moving body detection method by the said 2nd Embodiment of this invention, it is explanatory drawing of the corresponding point group which carried out the pass determination in the image imaged with the omnidirectional camera. In the moving body detection method by the said 2nd Embodiment of this invention, it is a figure which shows the result of having extracted the human area | region from the corresponding point group which carried out the pass determination in the image imaged with the omnidirectional camera, respectively. In the moving object detection method according to the second embodiment of the present invention, the human head panorama development image generated from the image captured by the omnidirectional camera is used, and the human head is obtained from the specific grayscale projection of the human area panorama development image. It is explanatory drawing of the extracted result. In the moving object detection method according to the second embodiment of the present invention, when an omnidirectional optical system is used, corresponding points of a central portion that is a human foot area (close to the floor) in an image captured by an omnidirectional camera will be described. FIG. In the moving object detection method according to the second embodiment of the present invention, when an omnidirectional optical system is used, a flow other than the optical flow generated by the movement of a person and an object at corresponding points in the four corner regions outside the omnidirectional optical system. It is a figure for demonstrating that (noise flow) is easy to generate | occur | produce. In the moving object detection method according to the second embodiment of the present invention, the distance from the robot to the person in the panoramic unfolded image of the omnidirectional camera image in order to explain the method of calculating the distance to the person based on the person's characteristic position. It is a figure of the state which arranged the image where different. In the moving body detection method by the said 2nd Embodiment of this invention, in order to demonstrate a person's distance image detection based on the specified person area | region, it is a figure explaining the state which a robot moves in a room. In the moving body detection method by the said 2nd Embodiment of this invention, it is a figure of the image imaged with the omnidirectional camera in order to demonstrate a person's distance image detection based on the specified person area | region. In the moving body detection method by the said 2nd Embodiment of this invention, in order to demonstrate a person's distance image detection based on the specified person area, it is explanatory drawing in the case of specifying a moving body position from the moving body area | region of a distance image. . In the moving body detection method by the said 2nd Embodiment of this invention, in order to demonstrate distance image detection of a person based on the specified person area, it is explanatory drawing in the case of determining the moving body area | region of a distance image. It is a flowchart which shows the process of the moving body detection method by the said 2nd Embodiment of this invention. It is a control block diagram of a robot provided with a robot positioning device of the robot control device according to the third embodiment of the present invention. It is a block diagram of the omnidirectional camera processing part of the robot control apparatus by the said 3rd Embodiment of this invention. It is a flowchart of the robot positioning operation | movement in the robot control apparatus by the said 3rd Embodiment of this invention. In the robot positioning device of the robot control device according to the third embodiment of the present invention, an explanatory diagram for explaining the characteristics of an input image when a PAL type lens or a fisheye lens is used in one example of an omnidirectional image input unit. is there. In the robot positioning device of the robot control apparatus according to the third embodiment of the present invention, an omnidirectional image input unit and an omnidirectional camera height adjustment unit that arranges the image input unit so as to be height-adjustable toward a ceiling and a wall surface. FIG. 6 is an explanatory diagram of an operation procedure of a conversion and extraction unit that converts and extracts a ceiling and wall whole view peripheral image and a ceiling and wall whole view central image from an image input by the image input unit. In the robot positioning device of the robot control apparatus according to the third embodiment of the present invention, an omnidirectional image input unit, and a conversion and extraction unit that converts and extracts a ceiling and wall surface panoramic image from an image input by the image input unit. It is explanatory drawing of an operation | movement procedure. In the robot positioning device of the robot control apparatus according to the third embodiment of the present invention, the ceiling and wall full view peripheral image and the ceiling and wall full view central image input at the present time (when the positioning operation is performed), and the prestored designation It is the figure which showed the condition which cross-correlation-matches as one example with the ceiling and wall surface whole view peripheral part image of a position, and a ceiling and wall surface whole view center part image. In the robot positioning device of the robot control apparatus according to the third embodiment of the present invention, the ceiling and wall full view peripheral image and the ceiling and wall full view central image input at the present time (when the positioning operation is performed), and the prestored designation It is the figure which showed the condition which performs cross-correlation matching as one example with the ceiling and wall surface whole view peripheral part image of a position, and a ceiling and wall surface whole view center part image. In the robot positioning device of the robot control apparatus according to the third embodiment of the present invention, the ceiling and wall full view peripheral image and the ceiling and wall full view central image input at the present time (when the positioning operation is performed), and the prestored designation It is the figure which showed the condition which performs cross-correlation matching as one example with the ceiling and wall surface whole view peripheral part image of a position, and a ceiling and wall surface whole view center part image. In the robot positioning device of the robot control device according to the third embodiment of the present invention, with respect to FIGS. 25A, 25B, and 25C, a map is added when the robot performs playback autonomous movement based on the map of the basic route. It is explanatory drawing for demonstrating.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Robot, 10 ... Drive apparatus, 11 ... Left motor drive part, 12 ... Right motor drive part, 20 ... Travel distance detection part, 21 ... Left encoder, 22 ... Right encoder, 30 ... Direction angle detection part, 31 ... Human Movement detection unit, 31a, 31b ... stereo camera, 32 ... robot movement amount detection unit, 32a ... omnidirectional optical system, 32b ... post, 33 ... robot basic path teaching data conversion unit, 34 ... robot basic path teaching data storage unit, 35 ... movable range calculation unit, 36 ... obstacle detection unit, 37 ... movement route generation unit, 60 ... system, 50 ... control unit, 51 ... storage unit, 52 ... input / output unit, 100 ... rear drive wheel, 101 ... front auxiliary traveling wheel, 102 ... person, 103 ... obstacle, 104 ... basic route, 104a ... range of movement in the width direction with respect to the basic route, 105 ... traveling floor surface, 106 ... moving route, 1 6a ... component in the width direction, 107a ... route, 107b ... place where the baby is located, 107c ... shortcut, 107d ... route, 111 ... left side travel motor, 113 ... wall surface, 114 ... ceiling, 121 ... right side travel motor, 201 ... corresponding point 301 ... corresponding point in the center of the image 302 ... omnidirectional camera image 302a ... human foot (close to the floor) region 302b ... four corners of the image 311 ... flow corresponding point 312 ... corresponding point 313 ... Corresponding points, 401 ... Corresponding point group, 402 ... Human region, 404 ... Human direction angle α, 411 ... Omnidirectional camera, 412 ... Omnidirectional camera processing unit, 412a ... Conversion extraction unit, 412b ... First cross-correlation matching unit 412c: Rotation angle deviation amount conversion unit, 412d: Second cross-correlation matching unit, 412e ... Position deviation amount conversion unit, 412f ... Conversion extraction storage unit, 413 ... Omnidirectional camera Height adjusting unit, 414... Camera height detecting unit, 501... Head extracted area, 502... Human area panorama developed image, 503... Vertical specific grayscale image, 504. , 601... Central image of the ceiling and wall and a peripheral image of the ceiling and wall, 902. Distance image, 903. Image detected as a human area, 1200. Mobile detection device, 1201. Point position calculation / arrangement unit, 1201a ... corresponding point position calculation / allocation change unit, 1202 ... time-series image acquisition unit, 1203 ... movement amount calculation unit, 1204 ... moving body movement determination unit, 1205 ... moving body region extraction unit, 1206a ... distance Image calculation unit, 1206b ... distance image specific range moving unit, 1206c ... moving body region determination unit, 1206d ... moving body position specifying unit, 1206e ... distance calculation unit.

Claims (3)

Mounted on a movable robot, and, after detection knowledge the human as a moving body existing in front of the robot, and a human movement detection section for detecting a movement of the person,
A driving device that is mounted on the robot and moves the robot in accordance with the movement of the person detected by the human movement detection unit during route teaching;
A robot movement amount detection unit for detecting a movement amount of the robot moved by the driving device;
A first route teaching data conversion unit that accumulates movement amount data representing the movement amount detected by the robot movement amount detection unit and converts the movement amount data into route teaching data;
It is mounted on the robot, and a peripheral object detection unit around the obstacles and the robot before Symbol robot detects the ceiling or wall position of the space to be moved,
When the robot autonomously moves by driving the driving device along the route teaching data converted by the first route teaching data conversion unit, the position of the obstacle detected by the surrounding object detection unit is A robot movable range calculator that calculates a robot movable range of the robot with respect to route teaching data;
A movement path generation unit that generates a movement path for autonomous movement of the robot based on the path teaching data and the movement range calculated by the robot movement range calculation unit;
The human movement detector is
A corresponding point position calculating and arranging unit for calculating and arranging the position of corresponding points to be detected according to the movement of the person around the robot in advance;
A time-series multiple image acquisition unit that acquires a plurality of images in time series;
Corresponding points arranged by the corresponding point position calculating and arranging unit are detected between the time-series images acquired by the time-series plural image acquiring unit, and between the detected images of the corresponding points are detected. A movement amount calculation unit for calculating the movement amount of
A moving body movement determination unit that determines whether the movement amount calculated by the movement amount calculation unit is a corresponding point that matches the movement of the person;
From the corresponding point group obtained by the mobile body movement determination unit, a mobile body region extraction unit that extracts a mobile body region that is a range where the person exists,
A distance image calculation unit that calculates a distance image of a distance image specific range around the robot that is a certain range around the robot;
A distance image specifying range moving unit that moves the distance image specifying range calculated by the distance image calculating unit so as to match the range of the moving object region extracted by the moving object region extracting unit;
A moving object region determination unit that determines a moving object region of the distance image after the movement by the distance image specific range moving unit;
A moving body position specifying unit that specifies the position of the person from the moving body area of the distance image obtained by the moving body area determination unit;
A distance calculating unit that calculates a distance from the robot to the person based on the position of the person specified on the distance image by the moving body position specifying unit;
The position of the corresponding point to be detected according to the movement of the person in advance is configured to include a corresponding point position calculation arrangement changing unit that changes each time according to the position of the person,
According to the movement route generated by the movement route generation unit, the person movement detection unit identifies the person and continuously detects the distance and direction between the person and the robot, thereby autonomously moving the robot. A robot controller that controls   The surrounding object detection unit includes an omnidirectional image input system capable of acquiring omnidirectional images around the robot and an obstacle detection unit capable of detecting obstacles around the robot, The robot control device according to claim 1, wherein the robot control device detects a ceiling or a wall position of a space around which the obstacle or the robot moves. An autonomous mobile robot comprising the robot control device according to claim 1.

Images