JP2012033119A - Three-dimensional environment generation system - Google Patents

Abstract

A large-scale point cloud data is replaced with a CG model with a small amount of calculation and a small error.
A point cloud data with reflection intensity obtained by measuring a 3D shape in an environment and a 3D CG model group with a marker are input, a marker arrangement calculated from the point cloud data with a reflection intensity, and an input with a marker A model candidate extraction unit that extracts a three-dimensional CG model having a marker with a similar arrangement by collation processing with the marker arrangement of the three-dimensional CG model group, and the marker arrangement calculated from the point cloud data with reflection intensity, A point group-CG replacement unit that outputs a three-dimensional CG model with the smallest matching error as a corresponding three-dimensional CG model by combining the markers of the three-dimensional CG model.
[Selection] Figure 1

Description

  The present invention relates to a system for generating an indoor environment with three-dimensional CG, and more particularly to a system for replacing point cloud data with a CG model.

  One method of digitizing the shape of a building or the like in the real world is a method of measuring a target in detail using a laser distance sensor or the like and expressing a three-dimensional situation by a point cloud. For technology that reproduces this reality as it is, there may be needs in a wide range of fields such as plant, civil engineering and heavy industry, preservation of cultural heritage, confirmation of the current state of buildings, environmental recognition for robot movement, and amusement. Specific applications include, for example, in plant facilities and the like, accurately measuring and diagnosing damage and deterioration of existing equipment, and measuring on-site dimensions for equipment repair or modification. . In addition, this technology will be deployed in the field of factory simulation, such as changing the layout of the factory, examining the installation location when introducing new equipment, checking the operating range of hand robots operating in the factory, and checking the operation of mobile robots. It is also possible to do.

  However, there are many problems in creating a model of a large facility using point clouds. Since the amount of data in point clouds is enormous, it is difficult to handle on a PC, the noise generated in point clouds is large and the amount of abnormal values is large to measure vast places, and it is generated directly from point clouds There is a problem that the data amount of the mesh model is enormous. At present, there is no known method for efficiently creating a three-dimensional model from a point cloud measurement data of a large facility with little effort.

  Non-patent document 1 discloses a technique for applying a shape model to point cloud data. Targeting the modeling of plant piping, the point cloud is automatically created based on the data obtained by manually cutting the pipe from the measured data or by creating a point cloud of the pipe that contains abnormal values on a trial basis. Is applied to a cylinder.

Toshiki Miyagawa, Shoichi Tsunoda, Taiki Iritani, Yoshifumi Iikawa, Hiroshi Masuda, Ichiro Tanaka, “Plant Modeling Based on Laser Scan Data”, Precision Engineering Society Spring Lecture, March 1, 2006, A69

  In modeling from the point cloud described in Non-Patent Document 1, the point cloud created experimentally is created on the assumption that a cylinder is applied, and the curves that can be handled are limited to the cylinder. The cylinder can only be applied to a point group obtained by cutting out a portion that is known to be a cylinder in advance. Therefore, it cannot cope with shapes other than the cylinder.

  In the general environment where we live, there are objects of various shapes as well as cylinders. Moreover, it is unknown what kind of shape is placed in what position and orientation. Furthermore, a method for extracting a point cloud for each object has not been established in order to fit a model to the point cloud.

  To model the indoor environment acquired with a point cloud, first recognize the location of the object, then select the model that matches the point cloud at that location, and change the model to the correct position and orientation A way to do this is conceivable.

  The present invention has been made in view of the above, and an object of the present invention is to select a corresponding model and calculate a position and orientation simply and with little error when applying a CG model to measured data by a point cloud. It is to provide a dimensional environment generation system.

  A desirable mode of the present invention is the following three-dimensional environment generation system. That is, the three-dimensional environment generation system according to the present invention receives as input the distance data with reflection intensity obtained by measuring the three-dimensional shape in the environment and the three-dimensional CG model group with a marker, and generates a specific reflection from the distance data with reflection intensity. A marker position output unit that extracts a region with strength and stores it as a marker region with the coordinate value, and a neighborhood that searches for a marker near the marker and stores the coordinate value of the searched marker as a marker location A marker search unit, a marker arrangement calculated from the point group, and a model candidate extraction unit that extracts a three-dimensional CG model having a marker with a similar arrangement by collating the marker arrangement of the input three-dimensional CG model group with a marker; 3D CG model position and orientation by matching the marker of the 3D CG model with the extracted marker to the marker arrangement calculated from the point cloud A model initial position calculation unit that is temporarily determined, a model verification unit that performs processing for verifying a three-dimensional CG model installed at the initial position of the calculation model with point cloud data, and a three-dimensional CG model with the least verification error. A three-dimensional CG model is provided, and a corresponding part in the point group data and a point group-CG replacement unit that replaces the three-dimensional CG model and outputs the three-dimensional environment model are provided.

  According to the present invention, point cloud data can be automatically replaced with a three-dimensional CG model with a small amount of calculation.

It is the functional block diagram which showed the flow of 3D environment model creation. It is a flowchart of 3D environment model creation. It is a flowchart of a model collation process. It is the figure which showed the example of point cloud data. It is the figure which showed the example of the 3D CG model with a marker. It is the figure which narrowed down a three-dimensional CG model candidate by marker arrangement collation processing. It is an image figure of a point group and a three-dimensional CG model collation process. It is an image figure of selecting the three-dimensional CG model that most closely matches the point group. It is an image figure of selecting the three-dimensional CG model that most closely matches the point group. It is an image figure of selecting the three-dimensional CG model that most closely matches the point group.

  Hereinafter, embodiments will be described with reference to the drawings.

  FIG. 1 is a functional block diagram showing the flow of creating a three-dimensional environment model. The three-dimensional environment model is obtained by modeling a three-dimensional shape of a real space, for example, an indoor space in which we live with CG.

There is a method of pasting a marker serving as a mark at an appropriate location in the real space in order to easily associate the data in the real space with data acquired by a sensor such as a laser distance sensor or a camera.
For the marker, for example, an object of a specific color, shape, or size that can be easily recognized by a camera, or an IC tag that reacts to a specific frequency that can be easily recognized by an ID reader is used. Use a sensor suitable for the sensor used.
Here, a material that can be easily recognized by the laser distance sensor and returns a special value upon laser irradiation is used as a marker. This is a reflection marker.
Using the distance sensor, the distance data 100 with the reflection intensity obtained by measuring the distance / angle between the distance sensor position and the surrounding obstacle and the reflection intensity of the laser at the measurement destination is input. In order to obtain data of specific reflection intensity, it is necessary to install a device that returns a special value at the time of laser irradiation in any place in the real space to be measured.

  For objects having the same shape, markers are attached in the same arrangement. The marker position output unit 200 extracts a region having a high reflection intensity from the distance data with reflection intensity 100, and if the shape and size of the region match the pre-registered marker information, the location is set as a marker application location. To do. Then, the coordinate value of the point located at the center of the marker area is output as the marker coordinate value 102. The marker coordinate value 102 is output for all regions having high reflection intensity existing in the point cloud data. An arbitrary marker is selected from the output marker coordinate values 102, and a marker in the vicinity thereof is searched. The search range is set in advance. For example, when the rule is set that the range of the marker to be set on the object to be modeled in the environment is within a radius of 30 cm, the coordinate values of other markers within the radius of 30 cm are extracted from the selected marker. The coordinate values of the markers found in the vicinity are combined and output as the marker arrangement 103.

  The point cloud data database stores the point cloud data file name, the marker coordinate value, and the marker arrangement, as indicated by 200 in FIG. In the marker coordinate value, the three-dimensional coordinate values of all markers in the point cloud data, numbered 1, 2, 3,... Are stored. In the marker arrangement, a combination of the three-dimensional coordinate value numbers is stored as a marker arrangement.

  Next, a three-dimensional CG model that matches the marker arrangement 103 extracted from the point cloud data is determined. A marker-added three-dimensional CG model 110 is prepared in advance. The 3D CG model with a marker is data obtained by adding marker information to the 3D CG model. As shown by 300 in FIG. 10, information (number, size, arrangement) on the shape model file name and the marker is stored on the database.

  The model candidate extraction unit 220 performs a matching process between the marker arrangement 103 extracted from the point group and the marker arrangements of all the three-dimensional models 110 with markers. An allowable range of marker position deviation is determined in advance as, for example, 5 cm, and all the markers arranged within the allowable range are extracted as 3D CG models with markers having similar arrangements, and 3D CG model candidates with markers are extracted. As 112, data is secured.

  In the model initial position calculation unit 230, by matching the marker arrangement of the three-dimensional CG model 112 with the marker arrangement 103 of the point cloud data, an approximate three-dimensional CG model for collating the point group with the three-dimensional CG model is obtained. Is calculated, and the model position / angle 113 is output. In the model checking process 240, the point cloud data 100 and the three-dimensional CG model 112 are checked, the position with the smallest error is calculated, and the model position / angle 113 data is updated. If the error at the time of model matching is lower than the threshold value, it is determined that the model matches the point cloud, and the point cloud-CG replacement unit 250 replaces the corresponding part in the point cloud data with CG and replaces the three-dimensional environment model 120. create. Here, the three-dimensional environment model 120 may be data in which point cloud data and a CG model coexist, or all point cloud data is replaced with a CG model, and all data becomes CG data. There is also.

FIG. 2 is an overall flowchart of creating a three-dimensional environment model according to this embodiment.
First, in step S1000, measurement data is input. In the laser distance sensor, for example, the laser beam is irradiated every 180 degrees around, and the distance to the obstacle is measured according to the time until the laser beam hits the obstacle, the angle θ from the sensor, The distance r to the obstacle at that angle is output as distance data. Further, the reflection intensity I when the laser beam hits the obstacle is also output at the same time. The measurement data is distance data from the sensor measured by the laser distance sensor to surrounding obstacles and reflection intensity data, and is obtained from the laser distance sensor position (rx, ry, rz) and the distance data (θ, r). The three-dimensional coordinate values (x, y, z) of the laser irradiation destination and the measured reflection intensity I are stored as a pair. Collected three-dimensional coordinate values of the laser irradiation destination are set as point cloud data. The distance data is integrated into the same coordinate system based on the coordinate value of the measurement point and the distance and angle from the coordinate value, and each coordinate point has the coordinate value of the coordinate system. An example of point cloud data is shown in FIG. Reference numeral 100 represents point cloud data, and reference numeral 101 represents point cloud data that returns a high reflection intensity.

  In step S1010, a region having a specific reflection intensity is extracted from the measurement data. In order to distinguish the marker portion, a high reflection tape that returns a high reflection intensity value upon laser irradiation is attached to the object in a predetermined arrangement. Then, the area of the extracted region is calculated, and only when the area is approximately equal to the marker size, for example, an error within 10%, the place is set as the place where the marker is set. As an example of the area calculation method, point cloud data viewed from an arbitrary coordinate value is output as a two-dimensional image in which a portion with high reflection intensity is expressed in white and a portion other than that in black, and the area of the white portion is obtained. A method is mentioned.

  In step S1020, the center position of the marker area is set as the marker installation position, and the coordinate value is output. The coordinate values of all the markers detected in the point cloud data are output and stored in the database together with the marker numbers as indicated by 202 in FIG.

  In step S1030, another marker in the vicinity of the output marker position is searched. In step S1040, the marker number output in S1020 and the neighboring marker number searched in S1030 are as shown in FIG. Assuming that the markers are for the same object, marker numbers are collectively stored for each object as indicated by 203 in FIG.

  In step S1050, a CG model candidate is selected. When matching a point cloud and a model, first, model candidates are narrowed down using matching by marker arrangement. This is because, if all models are applied and verified, the amount of calculation becomes enormous. If the approximate position and orientation of the model are unknown, the search range becomes too wide. An example of a pre-registered three-dimensional CG model is shown in FIG. A 3D CG model of an object in the room is registered in the DB in advance. As shown at 200 in FIG. 10, the coordinate name 111 of the marker indicating the file name 110 of the shape model, the number of markers, the marker size, and the marker arrangement is registered in the DB for each model.

  In selecting model candidates in step 1050, as shown in FIG. 7, the marker arrangement (derived from actual measurement) output in step S1040 is compared with the marker arrangement of the model registered in advance (registered together with the CG model). Then, a CG model in which the marker is arranged within a predetermined allowable range of the marker position deviation, for example, within 5 cm, is extracted and set as a model candidate.

  If a model candidate is found, numbers 1 to n are assigned to the corresponding model. In step S1060, it is determined whether there is a model candidate. If no model candidate is found, the process ends. When a model candidate is found, a matching process is performed between the model candidate and the measured point cloud data in step S1070 to search for a matching model. The flow of the verification process will be described in detail with reference to FIG.

  In step S1080, the point cloud data is replaced with a CG model. The model [m] determined in step S1070 is installed in the point cloud data according to the position and orientation of the model [m] output in step S1070. Next, the point group in the CG model [m] and the point group around the CG model [m] (range of measurement error of the laser distance sensor) are deleted.

  By executing this series of flows for all the markers detected in the point cloud data, the point cloud data is replaced with a CG model. Even if the arrangement of objects in the room is changed due to a layout change or the like, the laser distance sensor measures the room again, acquires distance data and reflection intensity data, and performs the point group-CG replacement process, The corresponding model can be automatically converted into a CG model at the changed position and direction.

  FIG. 3 is a flowchart of the model matching process, and details the flow of the model matching process in step S1070.

  In step S1060, model candidates 1 to n are selected. Here, all the model candidates are sequentially applied to the point group, and the best match is determined as the CG model to be replaced.

  First, collation processing is performed from model number m = 1. It is necessary to set the initial position / posture of the CG model m as a pre-process of the collation process. Therefore, in step S1071, as shown in FIG. 8, the position / posture of the CG model is set so that the marker arrangement of the model (registered together with the CG model) matches the marker arrangement of the point group (derived from actual measurement). change. In step S1072, the point cloud and the CG model are collated. The matching process includes, for example, an ICP (Interactive Closest Point) algorithm that is generally used for a point cloud matching process, a distance between a point cloud and a model's neighboring points (on the surface of the model that is closest to the point and the point) For example, an algorithm for searching for a position and orientation with which the least square error of (calculates the distance to the coordinate value of all points) is reduced. In order to obtain an accurate result using this algorithm, the setting of the initial position is important. By giving the initial value of the position and orientation in advance, the collation process between the point group and the CG model can be facilitated and the accuracy can be improved. In the present invention, since the alignment is simply performed in advance by using the marker arrangement, an approximate initial position can be automatically set.

  In step S1073, it is determined whether the calculation error exceeds a predetermined threshold value. If the calculation error exceeds the threshold value, it is determined that the model is not a model that matches the point group, and the process proceeds to step S1075. If the threshold is not exceeded, it is determined that there is a high possibility that the model can be replaced with a point cloud, and the model [m] is retained as a candidate. Furthermore, the place where the error is smallest in the matching process is stored as the position / posture of the model [m].

  In step 1075, it is determined whether all candidate models have been collated. That is, it is determined whether or not the model number m that has just been collated is smaller than the total number n of candidate models. If there is a model that has not yet been collated, 1 is added to the model number and the process returns to step S1071. When all the models have been collated, it is determined in step S1076 whether or not model candidates remain, and if it is determined that they remain, in step S1077, as shown in FIG. The model candidates to be compared are compared and their errors are compared. The model having the smallest error is determined as the model to be replaced with the point group, and the model number m and the position and orientation of the model [m] are returned. If there is no model candidate in step S1076, the process ends with no corresponding model.

  When the above-described processing is performed for all markers in the environment and all replaceable point groups are replaced with the three-dimensional CG model, the matching processing ends.

  In this way, by performing the marker arrangement matching process in advance, the candidates for the CG model can be easily narrowed down. Furthermore, by giving the initial position before the matching process based on the marker arrangement, it is possible to reduce an error when the point cloud and the model are matched.

  In order to limit the search range when selecting a CG model candidate, an attribute representing a constraint condition may be set for the three-dimensional CG model with a marker. For example, it is necessary to consider the possibility of rotation around the X axis or around the Y axis when searching for similar marker arrangements for things such as shelves and desks that are unlikely to be installed in a fallen state. Therefore, an attribute indicating XY axis fixation is set. There is also a method of giving a constraint condition on the assumption that the bottom surface is in contact with the floor. For example, an attribute indicating whether or not the Z coordinate value is fixed is set.

  Another possible attribute is that indicating the possibility of movement. For example, assuming that a three-dimensional environment in an office is generated, a chair is likely to move daily, but a locker is unlikely to move. Therefore, when a three-dimensional environment model is once generated and only data update is performed thereafter, only a model having a possibility of movement needs to be updated, and the search range can be limited.

DESCRIPTION OF SYMBOLS 100 ... Point group, 101 ... High reflection intensity point group, 110 ... Three-dimensional CG model, 111 ... Marker, 200 ... Point group database, 300 ... Three-dimensional CG model with marker Database

Claims (3)

With the input of point cloud data with reflection intensity obtained by measuring the 3D shape in the environment and the 3D CG model group with markers,
A marker position output unit that extracts a region having a specific reflection intensity from point cloud data with reflection intensity, and stores the region as a marker region together with a coordinate value;
A nearby marker search unit that searches for a marker in the vicinity of an arbitrary marker and stores the coordinate value of the found marker as a marker arrangement;
A model candidate extraction unit that extracts a three-dimensional CG model having a marker with a similar arrangement by a matching process between the marker arrangement calculated from the point cloud data with reflection intensity and the marker arrangement of the input three-dimensional CG model group with a marker; ,
A model initial position calculation unit that temporarily determines a 3D CG model position and orientation by matching a marker of the 3D CG model with the extracted marker to a marker arrangement calculated from point cloud data with reflection intensity;
A model matching unit that performs a matching process between the point cloud data with reflection intensity and the three-dimensional CG model installed at the initial position of the calculated model;
As a result of the matching process, the three-dimensional CG model with the smallest matching error is set as the corresponding three-dimensional CG model, and the corresponding portion in the point cloud data with reflection intensity is replaced with the corresponding three-dimensional CG model. A point group-CG replacement unit that outputs as
A three-dimensional environment model creation system characterized by comprising:   The 3D environment model creation system according to claim 1, wherein the 3D CG model with a marker adds an attribute to the marker according to a constraint condition of a model posture.   The three-dimensional environment model creation system according to claim 1, wherein the three-dimensional CG model with a marker adds different attributes to the marker depending on whether the target is a fixed object or a moving object.

Images