JP7503426B2 - Map conversion system and map conversion program - Google Patents

Description

The present invention relates to obtaining a map for defining a route for a moving object.

At construction sites, technology is being developed to precisely estimate the position of a moving robot and automate construction work through the robot's operations. For example, technology is known for estimating the position of a robot that can estimate its own position even at a construction site (for example, Patent Document 1). With this technology, mapping information is obtained by extracting it from construction data of a measurable construction object, and the robot's own position within space is calculated based on the measurement data and the mapping information.

There is also known a technology for self-location recognition that can be easily applied even if the layout of the moving space is changed, has low introduction costs, and is fast and accurate (for example, Patent Document 2). There is also known a technology for a moving body to estimate its own position and autonomously control its movement to a specified destination, thereby stably obtaining the position of the moving body (for example, Patent Document 3).

Techniques related to localization of mobile devices using visual information are also known (e.g., Patent Literature 4). In this technique, in a computing system, each area description file represents a point cloud of spatial features detected by a first mobile device in the area. Feedback data also identifies whether one or more of the spatial features represented by the localization area description file are detected by a second mobile device while in the area.

As a route generation technique, a route generation technique is known that automatically generates an autonomous movement route that includes task information and executes route movement (for example, Patent Document 5). With this technique, a route is generated by searching for node elements and connectors and generating a list of connectors that the robot will pass through. Also, a technique is known that generates route data with relatively small error by improving Graph-based SLAM technology (for example, Patent Document 6).

JP 2018-164966 A JP 2010-66934 A JP 2019-125354 A JP 2018-519558 A JP 2006-259963 A JP 2018-84995 A

The technologies described in Patent Documents 1 to 6 above make it possible to estimate the self-position of a robot in real space and generate a route for the robot to work on. However, with some robots, even if they estimate their self-position, they may have problems, such as being unable to travel along a pre-planned route. This is because the movable range on a three-dimensional map varies depending on the type of robot, and so even if a route is generated based on a three-dimensional map that uniformly defines the movable range, the same self-position estimation and route cannot be applied uniformly to all robots. Furthermore, efficient routes differ depending on the individual robot.

Taking the above facts into consideration, the present invention aims to enable precise control of moving objects within a building.

To achieve the above objective, the map conversion system of the present invention obtains a two-dimensional or three-dimensional map that defines the possible routes of a moving object through a simulation using self-location estimation based on three-dimensional BIM data that represents a space in which the internal structure and attribute information of the structure have been processed in advance, and on specific individual information according to the type of moving object. This enables precise control of moving objects within a building.

The present invention has the effect of enabling precise control of moving objects within a building.

FIG. 1 is an image diagram of a process for acquiring an environmental map. FIG. 1 is an image of a robot management platform. 1 is a block diagram showing a configuration of a map conversion system according to an embodiment of the present invention. 10 is a diagram showing an example of individual information for each type of robot stored in a storage unit; FIG. 4 is a flowchart showing a map conversion process in the map conversion device according to the embodiment of the present invention. 4 is a flowchart showing an operation control process in the map conversion device according to the embodiment of the present invention. FIG. 1 is a diagram showing an example of an environmental map in which various obstacles are placed in a space. FIG. 13 is a diagram showing an example of a simulation of a movement path of a humanoid walking robot. FIG. 13 is a diagram showing an example of a simulation of a path of movement of a four-wheel drive robot. FIG. 13 is a diagram showing an example of a simulation of a drone's movement path.

[Embodiments of the present invention]
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A map conversion system according to an embodiment of the present invention will now be described with reference to the accompanying drawings.

In order to solve the above-mentioned problems, the present invention uses three-dimensional BIM (Building Information Modeling) data (hereinafter simply referred to as BIM data) and robot self-location estimation to obtain a map that specifies the path that the robot can move along depending on its body or posture. This enables precise control of the robot within a building. Here, the robot in this embodiment refers to a moving object including a work robot, a surveillance drone, etc. BIM data is drawing data of a three-dimensional solid model that represents a space in which the internal structure of a building and attribute information of the structure have been processed. In the field of architectural design, the use of BIM data is progressing, and when architectural design is performed, BIM data is created as a three-dimensional model of the building to be designed. Here, in the BIM data, it is unknown how the robot can move, and the movable range differs depending on the type of robot.

Therefore, in an embodiment of the present invention, an environmental map that specifies the robot's possible movement paths is obtained for each type of robot through a simulation using self-location estimation. The environmental map is obtained as either a two-dimensional map or a three-dimensional map. Even if it is a two-dimensional map, the possible movement range on the X-axis and Y-axis is specified, so control is possible. In other words, in this embodiment, an environmental map is obtained in which BIM data has been converted to specify the robot's possible movement paths, and is used to control the robot.

Figure 1 is an image diagram of the process of acquiring an environmental map. In this embodiment, as shown in Figure 1, a simulation is performed for each type of robot based on the BIM data, and an environmental map for each type of robot is acquired.

For self-location estimation in the simulation, SLAM (Simultaneous Localization and Mapping) is used. SLAM is a technology that uses the robot's sensors to recognize the surrounding environment and simultaneously estimate the robot's own position and orientation with high accuracy, and generates an environmental map while estimating the robot's own position. In SLAM, the robot recognizes the environment by combining multiple sensors. The sensors vary depending on the robot, but distance sensors, cameras, gyro sensors, magnetic sensors, acceleration sensors, radar sensors, etc. are used. In this embodiment, self-location estimation using SLAM is performed in a simulation environment. By providing BIM data and individual information of the robot to the simulation environment and performing a simulation, it is possible to generate self-location estimation results and an environmental map similar to those when the robot is actually run on a construction site. In this embodiment, SLAM is used in a simulation environment rather than a real environment because it allows for faster operation emulation.

The need to create an environmental map for each type of robot in this way will be explained. For example, suppose a planar two-dimensional map is acquired from BIM data in which areas where structures exist are simply defined as unmovable spaces. A two-dimensional map acquired in this way cannot be said to be a map that correctly reflects the robot's movable range. This is because it does not take into account the height direction of the structure, protrusions, etc., and collisions between the robot and the map. Even if a route is generated based on such a two-dimensional map, it will differ from the actual movable range of the robot. Furthermore, even if BIM data is considered as a three-dimensional map, it is data that uniquely indicates a structure, with only structural data in the space given, and this alone makes it possible to simply generate routes in a space where no structures are located.

On the other hand, the movable range on the map, that is, the movable route, differs for each type of robot. This is because each type of robot has different individual information, such as sensor position and range, body size (width, height, arms, etc. for wheeled robots), posture, and movement mode (road running, flying, etc.). For example, in a simple example, a road running robot cannot pass through areas where structures or materials are placed on the road surface, but a flying robot can. Also, various situations can be imagined, such as a robot that can pass over a step of only a few centimeters but cannot pass over a step of 10 centimeters or more. Therefore, it is necessary to perform simulations that take into account detection of structures, collisions, etc. for each type of robot and give it a movable range.

Here, the robot management platform used in the simulation will be described. FIG. 2 is an image diagram of the robot management platform. As shown in FIG. 2, the robot management platform is a platform for managing robots in a cloud environment. The robot management platform is implemented with programs for managing robots as various modules, and the modules are appropriately linked to perform necessary processing. In this way, the robot management platform realizes automation of construction work using robots. The list of modules shown in FIG. 2 merely shows an example of functional means, and is not limited to these examples. By utilizing such a robot management platform, it is possible to eliminate as much as possible the troublesome setting work for robot operation. In addition, by estimating the robot's self-position based on the BIM data of the construction target, the on-site person in charge can refer to the BIM data to instruct the robot, so that an intuitive operation can be provided to the on-site person in charge. In addition, functions required during construction, such as monitoring the status of the robot during remote operation, can be deployed as a platform service. In FIG. 2, modules related to the method of this embodiment include "SLAM/BIM data linkage," "simulation environment," and "route simulation." In this embodiment, an environmental map is generated by performing "SLAM/BIM data linkage" in the "simulation environment." In addition, the generated environmental map will be used to perform an actual "route simulation" of the robot.

As described above, in this embodiment, a simulation environment is constructed based on BIM data in which only unambiguous spatial information is defined, and a simulation is performed. In the simulation, spatial recognition is performed in the planar and vertical directions for each type of robot, and an environmental map is obtained in which the movable range is predefined according to the type of robot. In the environmental map in which the movable range is predefined, the movable range is predefined according to the attribute information related to the type of robot. In other words, according to this embodiment, an ambiguous environmental map can be obtained according to the type of robot.

The matters to be considered in the simulation will be explained using a specific type of robot as an example. For example, in the case of a four-wheel drive robot intended for garbage collection, it is necessary to set in the simulation environment the location where the collected garbage will be accumulated and the location where the robot will stop after the work is completed. In addition, in the case of a four-wheel drive robot intended for transportation, it is necessary to read the QR code (registered trademark) attached to the material and automatically obtain the destination (area). It is also necessary to align the material pallets in a predetermined row in the material yard. In the simulation, these necessary tasks and the control related to the tasks must be considered as individual information for each type of robot. In addition, the gradient of the ramp that can be walked or driven on, the steps that can be passed, and the height at which the robot can fly in the case of an aircraft must also be considered as individual information.

Figure 3 is a block diagram showing the configuration of a map conversion system according to an embodiment of the present invention. As shown in Figure 3, in the map conversion system 100, a map conversion device 110, a terminal 140, and each of a number of robots 150 (150A to 150C) are connected via a network N. The network N is, for example, an Internet line or a public wireless LAN. Hereinafter, when referring to a robot to be controlled in general, it will be simply referred to as robot 150.

The terminal 140 is a terminal operated by various personnel, and performs input and output necessary for the processing of the map conversion device 110. The various personnel here correspond to the robot management platform described above. For example, the person in charge of acquiring the environmental map is an "equipment management person", and the person in charge of individual operation control of the robot 150 is an "on-site person in charge". When logging in to the terminal 140, authority is assigned according to the various personnel, but since this is not the main processing of the map conversion system 100, an explanation is omitted here. In this embodiment, the terminal 140 is used to input BIM data, input individual information of various robots, input the type of robot 150 that is the subject of the BIM data, check the environmental map, and input control input data for the robot. The control input data for the robot is information including, for example, the destination, the object to be transported, etc.

The robot 150 is the robot for which the environmental map is to be acquired. The robots 150 are implemented by type as agents in the above-mentioned simulation environment, and their behavior can be emulated. In the actual environment of the construction site, the robots operate according to the generated control information.

The map conversion device 110 includes a communication unit 112, a conversion unit 114, an operation control unit 116, and a storage unit 120. The map conversion device 110 can be configured as a computer including a CPU, a RAM, and a ROM that stores programs and various data for executing each processing unit (not shown). The map conversion device 110 of this embodiment is a server constructed by modularizing some of the functions of the robot management platform of FIG. 2 described above, and each functional unit is an example of a function.

The communication unit 112 transmits and receives various data by communicating with the terminal 140 and the robot 150. For example, it receives BIM data and individual information for each type of robot from the terminal 140 and stores it in the memory unit 120. It also receives the type of robot 150 that is the subject of the BIM data. It also transmits the acquired environmental map to the terminal 140. It also transmits control information generated for the robot 150 to be controlled to the terminal 140 and the robot 150.

The storage unit 120 stores BIM data and individual information for each type of robot 150. In addition, an environmental map for each type of robot 150 acquired by the conversion unit 114 is stored. In addition, control information for the robot 150 to be controlled, generated by the operation control unit 116, is stored. FIG. 4 is a diagram showing an example of individual information for each type of robot 150 stored in the storage unit 120. As shown in FIG. 4, the individual information is a type indicating the type of robot, the size of the vertical width (h) and horizontal width (w), information on the sensors provided, etc. In addition, the individual information also includes information on the necessary work according to the type of robot and the necessary control related to the work. If the movement form is a road surface, it is a four-wheel drive robot that runs on the road surface and works, and if the movement form is flying, it is a flying type robot (drone) that flies in a space at a predetermined height. The sensor information has information for the number of sensors provided by the robot. Examples of sensor information include the type of sensor (ranging sensor, camera, etc.), sensor position, sensor range, etc., but may include other information. As an example of sensor information, a four-wheel drive robot for cleaning purposes (hereinafter, a cleaning robot) will be described. For example, the cleaning robot has a laser scanner and a bumper sensor as sensor information. The laser scanner is a sensor that detects surrounding walls and obstacles, and by using the laser scanner, the cleaning area can be determined by detecting reflective markers placed at the four corners of the area to be cleaned. In addition, the cleaning robot moves within the cleaning area along a path that can clean up all the dirt within the cleaning area surrounded by the reflective markers, so the movement path can be automatically generated within the cleaning area. The bumper sensor is a sensor that detects contact with an obstacle, and detects and avoids obstacles within the cleaning area. Simulations are performed by combining the individual information given to each type of robot as described above.

The conversion unit 114 obtains an environmental map by performing a simulation using self-position estimation based on the BIM data and individual information in the memory unit 120.

The processing steps of the conversion unit 114 up to the acquisition of the environmental map in the conversion unit 114 will be described. In the conversion unit 114, first, the target BIM data is acquired from the storage unit 120, and a simulation environment is constructed by the BIM data. The target BIM data is data of a construction site that requires an environmental map. Next, for the acquired BIM data, the type of robot 150 that is the target of the BIM data is accepted from the terminal 140, and set as the acquisition target (N types). The acquisition target here refers to the type of robot 150 that needs to acquire an environmental map for the acquired BIM data, among all types of robots 150. The reason for determining the type of target robot 150 according to the BIM data in this way is that it is assumed that the type of robot 150 required at the construction site may differ. Next, for the N types of acquisition targets, an agent in the simulation environment is created for each acquisition target. The individual information in the storage unit 120 is used to create the agent. The created agent is placed at an arbitrary initial position in the simulation environment, and the environmental map is updated while performing self-location estimation so as to scan the entire space, and this is repeated until scanning of the entire space is completed. In the simulation, the collision in the height direction between the agent according to the individual information and the structure of the BIM data is reflected. The collision in the height direction will be explained using the running type and the flying type as examples. In the case of a running type robot, if a structure of a certain height or more exists in the space, the robot will collide with the structure and the environmental map is updated to show that the robot cannot move. On the other hand, in the case of a flying type robot, even if a structure of a certain height or more exists, if there is a space above the structure that is larger than the size, the robot will avoid colliding with the structure and the environmental map is updated to show that the robot can move. The final environmental map is acquired and stored in the storage unit 120. The map may be output to the corresponding terminal 140 (a terminal operated by the equipment manager) together with the storage. The conversion unit 114 acquires the environmental map through the above processing steps.

The operation control unit 116 generates control information for the robot 150 based on the control input data for the robot 150 to be controlled received from the operation control terminal 140 (a terminal operated by a site worker) and an environmental map corresponding to the type of the robot 150. If the route to the destination in the control input data includes multiple environmental maps, the control information is generated using the multiple environmental maps. The generated control information is sent to the terminal 140 and the robot 150. The robot 150 operates according to the control information. The control information is generated as information including a route simulation and necessary work processes to the destination included in the control input data, using an environmental map corresponding to the type of the robot 150. Specific examples of route simulations will be described later.

Here, the terminal 140 involved in the operation control has a display interface that allows confirmation of the route on the environmental map in the control information of the robot 150 and real-time status monitoring. Various aspects of the display interface can be envisioned. For example, the display interface may be one that can switch between a two-dimensional display and a three-dimensional display of the route on the environmental map. The display interface may also display an estimated arrival time to the destination, a progress status according to status monitoring, etc.

Next, the operation of the map conversion device 110 according to an embodiment of the present invention will be described. FIG. 5 is a flowchart showing the drawing conversion process in the map conversion device 110 according to an embodiment of the present invention. The drawing conversion process is performed by the CPU reading and executing a program and various data from the ROM. The CPU functions as each part of the map conversion device 110. It is assumed that the BIM data and individual information for each type of robot are stored in advance in the memory unit 120.

In step S100, the conversion unit 114 retrieves the target BIM data from the storage unit 120 and constructs a simulation environment using the BIM data.

In step S102, the conversion unit 114 receives from the terminal 140 the type (N types) of robot 150 that is the subject of the acquired BIM data.

In step S104, the conversion unit 114 sets the type of robot for the acquisition target n (the initial value of the acquisition target n is n=1).

In step S106, the conversion unit 114 creates an agent in the simulation environment for the acquisition target n using the individual information in the memory unit 120.

In step S108, the conversion unit 114 places the created agent at an arbitrary initial position in the simulation environment.

In step S110, the conversion unit 114 repeats updating the environmental map while estimating the agent's self-position so as to scan the entire space, until scanning of the entire space is completed. In the simulation, as an example, the vertical collision between the agent according to the individual information and the structure of the BIM data is reflected.

In step S112, the conversion unit 114 obtains the final environment map and stores it in the memory unit 120.

In step S114, the conversion unit 114 determines whether the acquisition target n is greater than or equal to N. If the condition is met, the process ends, and if the condition is not met, the process returns to step S104, counting up n = n + 1 to update the acquisition target n, and repeats the process.

Next, we will explain the operation of controlling the movement of the robot 150 in a real environment. Figure 6 is a flowchart showing the movement control process in the map conversion device 110 according to an embodiment of the present invention.

In step S200, the communication unit 112 receives control input data for the robot 150 to be controlled, which is transmitted from the terminal 140 related to the operation control.

In step S202, the operation control unit 116 acquires an environmental map corresponding to the control input data and the type of the robot 150 from the storage unit 120. Note that if the route to the destination in the control input data includes multiple environmental maps, multiple environmental maps are acquired.

In step S204, the operation control unit 116 generates control information based on the control input data and the acquired environmental map.

In step S206, the operation control unit 116 transmits the generated control information to the terminal 140 and the robot 150. The robot 150 operates according to the received control information.

Here, examples of route simulation for each type of robot will be described with reference to Figures 7 to 10. Note that while route simulation will be used as an example, the same considerations can also be applied to simulations for generating an environmental map.

Figure 7 is a diagram showing an example of an environmental map in which various obstacles are placed in a certain space. Figures 8 to 10 are examples of route simulations performed for robots of various types, such as a humanoid walking robot, a four-wheel drive robot, and a drone, moving through the space of the three-dimensional BIM data in Figure 7. A starting point and a destination point are set in the space, and obstacles are placed therein. The obstacles that are placed include, for example, obstacles with a fixed height (h), obstacles that can only be passed through the lower space, and obstacles that cannot be passed through. Each obstacle has a restriction on movement. For example, obstacles with fixed heights include 3.0 m and 0.5 m, and only drones that can fly at heights higher than this can move through them. An obstacle that can only be passed through the lower space indicates that a four-wheel drive robot can pass through it. The spaces between the obstacles that cannot be passed through are narrow, and the humanoid walking robot can move through the space within the space. There is also a step in front of the destination point, and the humanoid walking robot can move over it, but the four-wheel drive robot must move through a slope.

Figure 8 is a diagram showing an example of a route simulation of the movement of a humanoid walking robot. As shown in Figure 8, in the case of a humanoid walking robot, a route from a starting point to a destination point while passing between impassable obstacles is obtained in the route simulation. In the route simulation, for example, a combination of size and movement form is used as individual information of the humanoid walking robot. In the route simulation, the combination of these individual information is reflected as the fact that the humanoid walking robot can pass between impassable obstacles if it is the size of its width, and that it can step over steps because its movement form is walking. In terms of collisions with obstacles, the route simulation reflects collisions when passing between impassable obstacles, collisions with obstacles that can only be passed through the lower space, collisions when stepping over steps, etc.

Figure 9 is a diagram showing an example of a route simulation of the movement of a four-wheel drive robot. As shown in Figure 9, in the case of a four-wheel drive robot, the route simulation determines a route from a starting point to a destination point that avoids obstacles that cannot be passed, passes through obstacles that can only be passed through the lower space, and travels via a slope. In the route simulation, a combination of size and movement form is used as individual information of the four-wheel drive robot, for example. In the route simulation, the combination of these individual information is reflected as the fact that the height of the four-wheel drive robot means that it can pass through obstacles that can only be passed through the lower space, and that the width means that it cannot pass between obstacles that cannot be passed. In addition, it is reflected as the fact that the movement form is road running and therefore cannot cross steps and must travel via a slope.

Figure 10 is a diagram showing an example of a route simulation of drone movement. As shown in Figure 10, in the case of a drone, the route simulation determines a route that moves from a starting point to a destination point, avoiding obstacles that are impassable, passing above obstacles with a set height. In the route simulation, a combination of size, sensors, and movement mode, for example, is used as individual information of the drone. In the route simulation, the combination of size and sensors reflects the difficulty of control in a small range, etc. Also, because the movement mode is flight, it reflects the fact that flight above obstacles below a certain height is possible.

As described above, the map conversion system 100 according to an embodiment of the present invention enables precise control of moving objects within a building.

The present invention is not limited to the above-described embodiment, and various modifications and applications are possible without departing from the spirit of the invention.

For example, in the above embodiment, the map conversion system includes an operation control unit, but the present invention is not limited to this. A separate system for other operation control may be constructed, and operation control may be performed by the other system.

In addition, the map conversion system is not limited to target environments such as construction sites; it can be applied to any space that has BIM data, such as inside a building currently in use.

The target robot is not limited to the above-mentioned examples, but may be, for example, a ride-on type automatic transport vehicle. As an example, the method according to this embodiment may be applied to the above-mentioned robots such as personal mobility devices and electric wheelchairs.

100 Map conversion system 110 Map conversion device 112 Communication unit 114 Conversion unit 116 Operation control unit 120 Storage unit 140 Terminal 150 Robot

Claims (3)

3D BIM data is a three-dimensional model of a building's internal structure and the attributes of the structure, which represent the space defined by the data.
Based on the 3D BIM data and specific individual information according to the type of moving object, a simulation is performed using self-location estimation using Simultaneous Localization and Mapping (SLAM) ,
In the simulation, an agent is created according to the type of the moving object and the individual information;
In the simulation, the agent scans the entire space of the three-dimensional BIM data, and creates an environmental map corresponding to the moving object, thereby obtaining a created environmental map, which is either a two-dimensional map or a three-dimensional map that defines a possible moving route of the moving object.
A map conversion system , comprising:
The simulation includes reflecting a collision between the agent and a structure of the 3D BIM data in a height direction in the scanning, the collision differing depending on the type of the moving object and the individual information.
Map conversion system. The map conversion system according to claim 1, wherein the individual information includes at least one of the following information: sensor position, physique, and movement mode, which corresponds to the type of the moving body. 3D BIM data is a three-dimensional model of a building's internal structure and the attributes of the structure, which represent the space defined by the data.
Based on the 3D BIM data and predetermined individual information according to the type of moving object, a simulation is performed using self-location estimation using Simultaneous Localization and Mapping (SLAM) ,
In the simulation, an agent is created according to the type of the moving object and the individual information;
In the simulation, the agent scans the entire space of the three-dimensional BIM data, and creates an environmental map corresponding to the moving object, thereby obtaining a created environmental map, which is either a two-dimensional map or a three-dimensional map that defines a possible moving route of the moving object.
A map conversion program for causing a computer to execute a process,
In the scanning of the simulation, a collision in a height direction between the agent and a structure of the 3D BIM data is reflected, which collision differs depending on the type of the moving object and the individual information.
Map conversion program.