JPH0273471A - Estimating method for three-dimensional form - Google Patents

Abstract

PURPOSE:To automatically estimate a 3-dimensional picture of an object from only the object image that is photographed by a TV camera, etc., with no intervention of man power by providing a form converting part, a range calculating part, a contour detecting part, a feature point position detecting part, etc. CONSTITUTION:When a 3-dimensional model 3 is processed in a process where a 3-dimensional form of a certain object is obtained as the numerical data from the picture obtained by photographing the object, the model 3 is prepared and photographed in plural directions by cameras 1a and 1b. The photographed pictures of the models 3 are stored in the picture memories 2a and 2b. Then the attitude size detecting parts 5a and 5b and the form converting parts 4a and 4b magnify, reduce, move and rotate the form of the model 3 so that it is coincident with the size of the object and the attitude of the object set at photographing. While the range calculating parts 6a and 6b and the contour detecting parts 7a and 7b obtain the contours of the object and a range of the featuring part of the object. Then the model form is deformed via the feature point position detecting parts 8a and 8b and a form deforming part 9 so that the feature point of the model and the apex position of the contour are coincident with the positions obtained by each detecting part. Thus a 3-dimensional picture of the object is automatically estimated.

Description

[Detailed Description of the Invention] "Industrial Application Field" This invention is directed to CAD (con+puter aid
ed design), CA M (computer
The present invention relates to a method for obtaining the three-dimensional shape of any object required for computer graphics and the like. In particular, a method for obtaining the three-dimensional shape of a specific object with a certain width by targeting a collection of objects with similar structures and three-dimensional shapes, that is, objects with typical structures and three-dimensional shapes. Regarding.

In the following, description will be given using a person as an example of a collection of objects, but it goes without saying that the present invention itself is widely applicable to various objects other than people.

"Prior art" Video creation using computer graphics for broadcasting and movies (for example, [Reference 1] Komatsuji "-" - Curved face model of Yarakuta Ayumensin's face, Information Processing Society of Japan Graphics and CΔ1] Study Group Fund Sai131-5,
1988) and analysis/recognition marks in image communication 5) (
For example, (Reference 2) Aizawa et al.: "Model construction and facial expression synthesis in analysis-synthesis coding J, Institute of Electronics, Information and Communication Engineers Image Engineering Research Group Technical Report IE87-2, 1987", which simulates human movement and deformation using a computer. Attempts are being made to apply this method to various fields.For this purpose, a three-dimensional shape of a specific person is required.

Conventional methods for obtaining the shape of a three-dimensional object include (1) obtaining the three-dimensional coordinates of each point on the object's surface by shining a slit light or laser on the actual object, or using moiré to obtain the three-dimensional coordinates of each point on the surface; A method of constructing a three-dimensional shape by obtaining the depth of
) A method for constructing a three-dimensional shape from many C'F slice images, and (3) a method for creating a three-dimensional shape by manually modifying the original three-dimensional shape data little by little to resemble a specific person while looking at a graphic display. A method was used to obtain the original shape.

Therefore, the method of 11th Amendment (1) or (2) requires a special measuring device, and the person being measured must remain stationary within the special measuring device until the measurement is completed. There is a drawback that the accuracy decreases when trying to shorten the measurement time. Furthermore, the method (3) has the disadvantage that in order to obtain a three-dimensional shape, a person who is experienced in operating a device such as a graph ink display must spend a long time working on it.

There are attempts to automatically obtain a three-dimensional shape by photographing a person with a TV camera or other means and using various types of image processing, but this method does not automatically obtain a three-dimensional shape based only on the person's image without knowing the shape of the target object. It is difficult to obtain a three-dimensional shape, and this has not yet been achieved.

The present invention eliminates these drawbacks and allows TV cameras to
8. To automatically obtain a three-dimensional shape of a person by a machine from only some shadowed images of the person without human intervention.

"Means for Solving the Problem" The present invention prepares a three-dimensional model of a typical human body with body parts arranged in advance, and uses that model to create a special (pastoral) image of a person photographed with a TV camera or the like. The position and shape of each body part (for example, ears, eyes, nose, mouth, etc. in the head) are automatically detected, and the position and shape of each detected body part match the shape of the three-dimensional model. The three-dimensional shape of a specific person is estimated by automatically transforming the shape of a model.Hereinafter, it will be explained in detail using the drawings.

``Example'' Figure 1 shows an example of the present invention configured to estimate a three-dimensional shape from a front image and a side image of a person, and 1a and tb are images for photographing a front image and a side image, respectively. Cameras 2a and 2b are image memories that store front and side images, respectively; 3 is a storage unit for a three-dimensional model representing the three-dimensional shape and structure of a typical human; 4a and 4b are three-dimensional The shape of the three-dimensional model 3 can be enlarged, compressed or moved so that the shape of the model 3 matches the posture and size of the person when the front and side images were taken respectively.
5a and 5b are posture and size detection units that detect the posture and size of the person in the photographed frontal and side images, respectively; 6a and 6b are the cubic shape converters converted by 4a and 4b, respectively; The original model 3 is projected two-dimensionally under the same conditions as the receding conditions of the front and side images, and from the results, points that characterize the shape of the object, such as the corner of the eyes, the tip of the nose, and both ends of the mouth (hereinafter referred to as feature points)
Range calculation units 7a and 7b calculate the existing range Iγ of the position of the chin and the outline of the head in both images, respectively, using the ranges calculated by the range calculation units 6a and 6b, Contour detection units 8, 8 and 8b detect contour shapes of the head, chin, etc. from the range calculation unit 6a6b, respectively.
The feature point position detection section 9 detects the position of the feature point from the front image and the side image using the calculated fa area.
Contour detection units 7a, 7b and feature point position detection units 8a, 8
This is a shape transforming unit that transforms the shape of the transformed three-dimensional model so that it matches the front and side contour shapes and the positions of feature points obtained from b. Next, the embodiment shown in FIG. The operation will be explained in detail. To simplify the explanation, it is assumed that the processing target is limited to a human head and its three-dimensional shape is estimated.

The 7-dimensional model consists of the three-dimensional shape of the head as shown in Figure 2, the data of vertices corresponding to feature points in that shape such as the corners of the eyes, the tip of the nose, both ends of the mouth, and the ear holes, and the front and side views. Suppose that it is composed of data of vertices on the outline of the chin and head when viewed from above.

First, a person is placed in a predetermined position and made to assume a predetermined posture in order to facilitate processing in the posture size detection units 5a and 5b. And camera la.

lb and 1i shadow, and store the front image and side image in image memories 2a and 2b, respectively.

The posture size detection units 5a and 5b separate the head image and the background from the image by threshold processing, etc., and determine the vertical and horizontal sizes of the head image. l1ll and calculate the center position of the head.

The shape converters 4a and 4b expand, contract, and move the shape of the three-dimensional model 3 so that the shape of the three-dimensional model 3 matches the size and position measured by the posture and size detection section.
Rotate.

In the range calculation unit 6a, the shape of the converted three-dimensional model is projected two-dimensionally under the same shooting conditions as the camera la to create a two-dimensional figure.Then, by superimposing the two-dimensional figure on the front image, The feature point and contour data set in the three-dimensional model are used to determine the range of features (several points and contours) on the frontal image.Similar to 6d, the range calculation unit 6b also calculates the range of features in the side view E. Determine the range of feature points and contours.

Transport 1 (In the detection units 7a and 7b, the range calculation unit Ga.

The circle is detected using the contour existence range calculated in step 6b. For example, superimpose the two-dimensional figure on each of the front and side images, extend a straight line from the center of the head image to each vertex on the outline of the two-dimensional figure, and find the position where the straight line has a tolerance with the outline of the actual head image. Detect and find the position of the vertex on the contour.

In the feature point position detection unit Oa, 8b, the range calculation unit 6y+,
The actual position of each feature point is detected from the front and side images in the image memories 2a and 2b based on the existence range of the feature point given from 6b.

For example, a pattern of brightness changes near a feature point is prepared in advance, and the pattern is matched with the image while being shifted little by little within a given range of existence, and the position that best matches is determined as the position of the feature point.

In the shape deformation unit 9, the feature points in the three-dimensional model and the ring ff1
The shape of the three-dimensional model has been expanded/reduced/moved/rotated so that the position of the vertex on i matches the position obtained by the contour detection units 7a, 7b and the feature point position detection units 8a, 8b. Transform the whole thing.

For example, using the position of each feature point in the front image obtained by the contour detection section 7a and the feature point position detection section 8a and the position of the vertex on the contour, the shape shown in FIG. Similarly, the (y, z) coordinates are transformed from the position of each feature point obtained from the side image and the position of the vertex on the contour, and the three-dimensional shape of the head is obtained by combining them.

For example, among the vertices in the three-dimensional model shape, the vertices corresponding to the feature points and the vertices on the contour are moved to the measured positions, and the other vertices are moved by the amount of movement of the vertices to be moved. Move by interpolating.

In general, automatically detecting the position and shape of the above-mentioned parts from an image using a computer etc. means knowing where in the image they are, in what color or intensity, and in what size. It is difficult because there is no However, even if the shape of a person differs between individuals, the basic structure and shape of a person are the same and the shape is almost the same. Automatic detection becomes regression by determining the range and contour before the detection process. In other words, when attempting to separate a certain part by binarizing an a-light image, for example, if the range and rough shape of the part are known, a figure close to the given shape within the range of existence can be extracted. If it is not found, the search may be repeated by changing the binarization threshold.

In the above explanation, the person was retracted from two directions, the front and the side, and the three-dimensional shape was estimated using two images of the person.
It is also easy to expand the system to use three or more human images viewed from more than one direction. In that case, the accuracy of shape estimation can be improved.

"Effects of the Invention" As explained above, according to the present invention, by simply projecting images of various objects, including a person, from one or more directions, the person's three-dimensional image is automatically created without human intervention. Since the shape can be estimated, the present invention can be applied to applications that require the three-dimensional shape of a specific object, such as the generation of human images by computer graphics, personal identification based on the three-dimensional shape of a person, and the three-dimensional shape of industrial products. It is extremely effective in many applications such as original image generation, shape identification of industrial products and agricultural products, inspection, and automatic sorting.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing an embodiment of the present invention, and FIG. 2 is a diagram showing an example of a three-dimensional model of a face shape represented by a triangular patch. Patent applicant Nippon Telegraph and Telephone Corporation

Claims (1)

[Claims] (1) In a processing method for obtaining the three-dimensional shape of an object as numerical data from an image taken of the object, a three-dimensional model representing the general structure and three-dimensional shape of the object to be processed is created in advance. Prepare the object whose shape you want to estimate from one direction or multiple directions, and enlarge, reduce, move, and rotate the shape of the three-dimensional model to match the size and orientation of the object at the time of photographing. Then, by projecting the enlarged, reduced, moved, and rotated shape of the three-dimensional model onto the image under the same conditions as the shooting conditions of the captured image, the outline and characteristic parts of the object on the image can be determined. Find the existing range, use the existing range as a clue to detect the outline of the object and the actual position of the characteristic parts on the image, and detect the detected outline of the shape of the three-dimensional model and the actual position of each part. A three-dimensional shape estimation method characterized by obtaining a three-dimensional shape of an object by deforming the shape of a three-dimensional model that has been enlarged, reduced, moved, or rotated so as to match the shape of the object.