JP2013080389A - Vanishing point estimation method, vanishing point estimation device, and computer program - Google Patents

Abstract

To improve vanishing point estimation accuracy.
A straight line included in an input image is detected, a vertical straight line that is a substantially vertical straight line is detected from the straight lines, and a non-vertical straight line among the straight lines included in the image is detected. Among the combinations of the vertical straight line and the vertical straight line, a combination in which the distance between the end points is shorter than the threshold value is detected as a combination candidate. Are selected as vanishing line combinations, and the positions of vanishing points are estimated based on a plurality of non-vertical lines in the selected plurality of vanishing line combinations.
[Selection] Figure 1

Description

  The present invention relates to a technique for estimating a vanishing point of an image.

  In recent years, a technique for estimating depth information from an image for which depth information is not given based on vanishing points has been proposed. For example, in the technique described in Non-Patent Document 1, three-dimensional information is restored based on the position of the vanishing point. Therefore, a technique for estimating the position of the vanishing point of the image is required.

Naoki Ikeda and two others, “Statistical optimization of 3D reconstruction from a single image”, IPSJ Research Report, CVIM, November 11-12, 2004, pp. 117-124

  In general, there are various straight lines other than a straight line (for example, an edge of a ceiling, a wall, etc.) that forms a vanishing point in an image. Therefore, if the vanishing point is estimated based on a straight line other than the straight line forming the vanishing point, there is a possibility that an incorrect vanishing point may be obtained.

  In view of the above circumstances, an object of the present invention is to provide a technique for improving the estimation accuracy of a vanishing point of an image.

  According to one aspect of the present invention, a computer detects a straight line included in an input image, and a vertical straight line detection step detects a vertical straight line that is a substantially vertical straight line from the straight lines. The combination of the non-vertical straight line that is not the vertical straight line among the straight lines included in the image and the vertical straight line is detected as a combination candidate whose distance between the end points is shorter than a threshold value. A combination candidate detection step for selecting a combination candidate in which the vertical straight line and the non-vertical straight line are similar to each other as a erasure line combination, and a plurality of selected combinations A vanishing point estimation method for estimating a vanishing point position based on a plurality of the non-vertical direction straight lines in the vanishing line combination of .

  One aspect of the present invention is the vanishing point estimation method described above, wherein the combination candidate detection step includes a scene estimation step in which the computer estimates an approximate position of a vanishing point in an input image, and the approximate Determining a feature of a straight line extending in the direction of the vanishing point in the input image based on the position and selecting the non-vertical direction straight line having the feature; and the selected non-vertical direction A candidate detecting step of detecting, as a combination candidate, a combination in which a distance between the end points is shorter than a threshold value among combinations of the straight line and the vertical direction straight line.

  According to one aspect of the present invention, a straight line detection unit that detects a straight line included in an input image, a vertical direction straight line detection unit that detects a vertical straight line that is a substantially vertical straight line from the straight lines, and the image A combination candidate that detects, as a combination candidate, a combination whose non-vertical direction straight line that is not the vertical straight line among the straight lines included in the line and the vertical straight line has a distance between end points shorter than a threshold value A detection unit; a vanishing line combination selection unit that selects, from the combination candidates, a combination candidate in which the vertical straight line and the non-vertical straight line are similar to each other as a vanishing line combination; and a plurality of selected vanishing lines A vanishing point estimation device comprising: a position estimation unit that estimates the position of a vanishing point based on a plurality of the non-vertical direction straight lines in a combination.

  One aspect of the present invention is the vanishing point estimation device described above, wherein the combination candidate detection unit is based on a scene estimation unit that estimates an approximate position of a vanishing point in an input image, and the approximate position. Determining a feature of a straight line extending in the direction of the vanishing point in the input image and selecting the non-vertical direction straight line having the feature; the selected non-vertical direction straight line; and A candidate detection unit that detects, as a combination candidate, a combination in which the distance between the end points is shorter than the threshold among the combinations with the vertical straight line.

  One aspect of the present invention is a computer program for causing a computer to execute the above steps.

  According to the present invention, it is possible to improve the estimation accuracy of the vanishing point of an image.

It is a schematic block diagram showing the functional structure of 1st embodiment of a vanishing point estimation apparatus. It is a figure which shows the outline of a process of a straight line detection part. It is a specific example of an image showing a processing result of a vertical direction straight line detection unit. It is a schematic block diagram which shows the function structure of the combination candidate detection part in 2nd embodiment. It is a figure which shows the outline of a scene classification | category. It is a figure which shows the outline of a scene classification | category. It is a figure which shows the outline of the estimation process of a scene estimation part. It is a figure which shows the specific example of a feedback process.

[Overview]
The vanishing point estimation device estimates the position of the vanishing point of the image by analyzing the image. Specific processing is as follows. First, the vanishing point estimation device detects a straight line extending in a direction close to vertical (hereinafter referred to as “vertical direction straight line”) among straight lines included in the image. Next, the vanishing point estimation device uses a straight line that is not a vertical straight line (hereinafter, referred to as a “non-vertical straight line”) among straight lines included in the image and a vertical straight line between the end points. A combination whose distance is shorter than the threshold (hereinafter referred to as “combination candidate”) is detected. The vanishing point estimation device selects a combination candidate satisfying a predetermined condition as a vanishing line combination from the combination candidates based on the similarity of the straight lines. Then, the vanishing point estimation device estimates the position of the vanishing point based on the plurality of non-vertical direction straight lines in the selected plurality of vanishing line combinations.

[First embodiment]
Next, details of the vanishing point estimation apparatus will be described.
FIG. 1 is a schematic block diagram illustrating a functional configuration of the first embodiment of the vanishing point estimation device. The vanishing point estimation device 100 includes a CPU (Central Processing Unit), a memory, an auxiliary storage device, and the like connected by a bus, and executes a vanishing point estimation program. By executing this program, the vanishing point estimating apparatus 100 performs an image input unit 101, a straight line detecting unit 102, a vertical straight line detecting unit 103, a combination candidate detecting unit 104, a vanishing line combination selecting unit 105, a position estimating unit 106, It functions as a device including the output unit 107. All or some of the functions of the vanishing point estimation apparatus 100 may be realized using hardware such as an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA). good. The vanishing point estimation program may be recorded on a computer-readable recording medium. The computer-readable recording medium is, for example, a portable medium such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, or a storage device such as a hard disk built in the computer system. The vanishing point estimation program may be communicated via a telecommunication line.

  The image input unit 101 receives image data input to the vanishing point estimation apparatus 100. The image input unit 101 may read image data recorded on a recording medium such as a CD-ROM or a USB memory (Universal Serial Bus Memory). The image input unit 101 may receive an image captured by a still camera or a video camera from the camera. When the vanishing point estimation apparatus 100 is built in a still camera or a video camera, the image input unit 101 may receive a captured image or an image before imaging from the bus. Further, the image input unit 101 may receive image data from another information processing apparatus via a network. The image input unit 101 may be configured in a different manner as long as it can receive input of image data. An image whose input is accepted by the image input unit 101 is referred to as an “input image”.

  The straight line detection unit 102 detects a straight line from the input image. The vertical direction straight line detection unit 103 detects a straight line extending in a direction close to vertical from the plurality of straight lines detected by the straight line detection unit 102. The combination candidate detection unit 104 detects, as a combination candidate, a combination in which the distance between the end points is shorter than the threshold among the combinations of the non-vertical direction straight line and the vertical direction straight line. Vanishing line combination selection section 105 selects a combination candidate having a similar vertical straight line and non-vertical straight line as a disappearing line combination from among the combination candidates. The position estimation unit 106 estimates the position of the vanishing point based on the plurality of non-vertical direction straight lines in the selected plurality of vanishing line combinations. The output unit 107 outputs the estimation result obtained by the position estimation unit 106.

  Next, the process of the straight line detection unit 102 will be described in detail. FIG. 2 is a diagram showing an outline of processing of the straight line detection unit 102. 2A is a specific example of an input image, and FIG. 2B is a specific example of an image showing a straight line detection result. The straight line detection unit 102 extracts edges from the input image and detects a continuous edge as a single straight line. Edge extraction may be performed using, for example, a Canny edge detector, may be performed using Hough transform, or may be performed by other methods. For example, the technique disclosed in the following document may be applied to the Canny edge detector. Canny, J., A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8: 679-714, 1986.

  Next, the process of the vertical direction straight line detection unit 103 will be described in detail. The vertical straight line detection unit 103 calculates the angle of each straight line detected by the straight line detection unit 102. For example, the vertical direction straight line detection unit 103 calculates the angle between each straight line and the x axis when the horizontal axis of the image is the x axis and the vertical axis is the y axis. Then, the vertical direction straight line detection unit 103 detects a straight line whose difference between the calculated angle and 90 degrees is an angle threshold (for example, 20 degrees) or less as a vertical straight line. The angle threshold is a value stored in advance in the vertical direction straight line detection unit 103. FIG. 3 is a specific example of an image showing the processing result of the vertical direction straight line detection unit 103. Through such processing, the vertical direction straight line detection unit 103 extracts a straight line that is nearly vertical as a straight line that connects the floor and the ceiling or the ground and the sky.

  Next, the process of the combination candidate detection unit 104 will be described in detail. The combination candidate detection unit 104 calculates the distance between the nearest end points for each combination of the vertical direction straight line detected by the vertical direction straight line detection unit 103 and the non-vertical direction straight line. Then, the combination candidate detection unit 104 detects a combination in which the calculated distance is equal to or less than the endpoint distance threshold (for example, 3 pixels) as a combination candidate. The endpoint distance threshold is a value stored in the combination candidate detection unit 104 in advance.

  Next, the process of the vanishing line combination selection unit 105 will be described in detail. The vanishing line combination selection unit 105 determines whether each combination candidate is a straight line (edge portion) generated by the same object. A non-vertical straight line in a combination of a vertical straight line and a non-vertical straight line generated by the same object is likely to be a combination of straight lines generated by a rectangular edge in the screen. For this reason, it can be said that the non-vertical direction straight line described above is generally highly likely to be a straight line (vanishing line) constituting the vanishing point of the screen. Therefore, the vanishing line combination selection unit 105 can extract the vanishing line with high accuracy by selecting a combination candidate that is a combination of straight lines generated by the same object from the combination candidates.

  The processing of the vanishing line combination selection unit 105 will be described using a specific example of FIG. For example, when the wall surface 201 (FIG. 2A) exists in the image, a straight line is generated at the edge of the wall surface 201. Specifically, a vertical straight line 201a is generated at the joint of the wall surface 201, and a non-vertical straight line 201b is generated at the boundary between the wall surface 201 and the ceiling. These two straight lines are straight lines generated by the same object (wall surface 201). For this reason, pixel values (for example, RGB values in the input image) and edge strengths in these two straight lines are similar. Accordingly, the disappearance line combination selection unit 105 can select a combination of straight lines generated by the same object from among the combination candidates by comparing the analog values of pixel values and edge strengths.

  Hereinafter, a specific example of more detailed processing of the vanishing line combination selection unit 105 will be described. The vanishing line combination selection unit 105 may select the vanishing line combination by, for example, calculating a Bhattacharyya Distance. The Batachariya distance represents the distance between the two distributions. More specifically, the batcha rear distance is a distance defined as a self-information amount with respect to a probability (simultaneous probability) that two events occur simultaneously when two distributions are regarded as independent events.

Specific processing is as follows. The erasure line combination selection unit 105 generates a histogram of pixel values of vertical and non-vertical straight lines for each combination candidate. The histogram of the straight line in the vertical direction is represented as hist _a, and the histogram of the straight line in the non-vertical direction is represented as hist _b . In this case, the distance between the two batteries is expressed by the following formulas 1 and 2.

  Vanishing line combination selection section 105 selects a combination candidate whose calculated batcha distance is smaller than the buttery rear threshold as the vanishing line combination. The virtual threshold value is a value stored in advance in the disappearance line combination selection unit 105.

Next, the process of the position estimation unit 106 will be described in detail. The position estimation unit 106 estimates the position of the vanishing point based on a plurality of non-vertical direction straight lines included in the plurality of vanishing line combinations. Hereinafter, some patterns of the processing of the position estimation unit 106 will be described.
The position estimation unit 106 may select two vanishing line combinations from the vanishing line combinations, and calculate an intersection of these non-vertical straight lines as a vanishing point.

The position estimation unit 106 may select three or more vanishing line combinations from the vanishing line combinations, and calculate the center of gravity or average of the intersection points of the selected plurality of vertical line direction straight lines as vanishing points.
The position estimation unit 106 selects two vanishing line combinations from the vanishing line combinations and obtains their intersection. Next, the position estimation unit 106 detects a non-vertical direction straight line passing through the intersection from the vanishing line combination, and sets the number as the number of votes obtained at the intersection. The position estimation unit 106 may acquire the number of votes for all intersections and calculate the intersection with the largest number of votes as the vanishing point.
The position estimation unit 106 regards the length of the non-vertical line as the reliability, and calculates the intersection of the non-vertical line with the highest reliability and the non-vertical line with the second highest reliability as the vanishing point. Also good.

The vanishing point estimation apparatus 100 calculates an intersection using only the non-vertical direction straight line estimated to form the vanishing point among the straight lines included in the input image, and estimates the calculation result as the vanishing point. Therefore, the vanishing point estimation accuracy can be improved.
More specifically, it is as follows. In general, a wall of a building has many straight lines forming a rectangle, such as a rectangular or square window, a door, or a poster. Therefore, the vanishing point estimation device 100 detects a combination of straight lines (vanishing line combination) that is highly likely to form a rectangle. An index representing the possibility of forming a rectangle is calculated based on, for example, the distance between end points of straight lines, pixel values on the straight lines, and the like. Then, the vanishing point estimation device 100 calculates the vanishing point based on the non-vertical direction straight line included in the selected vanishing line combination. By such processing, it is possible to select the straight line forming the vanishing point as a non-vertical direction straight line with high accuracy. Therefore, it is possible to improve the vanishing point estimation accuracy by estimating the position of the vanishing point based on the selected non-vertical direction straight line. Therefore, the vanishing point can be accurately estimated even from one input image having many straight lines other than the straight line forming the vanishing point. In addition to this, it is possible not only to generate a stereoscopic image in a pseudo manner, but also to perform segmentation of the image using the depth estimation value and extract a desired region.

<Modification>
Vanishing line combination selection unit 105 may calculate cost C based on Equation 3 below, and may select a combination candidate whose cost C is equal to or less than the cost threshold as the vanishing line combination. The cost threshold is a value stored in advance in the disappearance line combination selection unit 105.

  In Equation 3, α and β are arbitrary coefficients. De (x1, x2) is the minimum value of the distance between the end point of the vertical straight line x1 and the end point of the non-vertical straight line x2. The length of x1 represents the length of the vertical straight line. Db (x1, x2) represents the distance between the vertical straight line x1 and the non-vertical straight line x2.

[Second embodiment value]
Next, a second embodiment of the vanishing point estimation device will be described. In the second embodiment of the vanishing point estimation device, the configuration of the combination candidate detection unit is different from the combination candidate detection unit 104 of the first embodiment, and the other configuration is the same as that of the first embodiment. Therefore, in the following description, only the combination candidate detection unit will be described.

FIG. 4 is a schematic block diagram illustrating a functional configuration of the combination candidate detection unit 104a in the second embodiment. The combination candidate detection unit 104a includes a scene estimation unit 401, a non-vertical direction straight line selection unit 402, and a candidate detection unit 403.
The scene estimation unit 401 estimates the approximate position of the vanishing point in the input image. For example, the scene estimation unit 401 estimates whether the position of the vanishing point is the left side in the image or the left side outside the image, the vicinity of the center in the image, the right side in the image, or the right side outside the image. The non-vertical direction straight line selection unit 402 has a high possibility of forming a vanishing point from non-vertical direction straight lines in the input image according to the position of the vanishing point estimated by the scene estimation unit 401 ( Hereinafter, “candidate non-vertical direction straight line” is selected. The candidate detection unit 403 detects, as a combination candidate, a combination in which the distance between the end points is shorter than the threshold among the combinations of the candidate non-vertical direction straight line and the vertical direction straight line.

  Next, the process of the scene estimation unit 401 will be described in detail. The scene estimation unit 401 stores a plurality of scene classifications, associations of input image feature amounts in each scene classification, and vanishing point positions in each scene classification in advance. The scene estimation unit 401 selects one scene classification from among a plurality of scene classifications by calculating the feature quantity of the input image and comparing it with a feature quantity stored in advance. 5 and 6 are diagrams showing specific examples of scene classification. As shown in FIG. 5, the interior of a building is generally composed of left and right wall surfaces A and C, a back surface D, a ceiling surface B, and a floor surface E.

  When photographing from the viewpoint VA in FIG. 5 in the direction of the arrow, the wall surface A is photographed so as to become smaller in the right direction as shown in FIG. 6A. That is, in the image of FIG. 6A, a scene including a wall surface A and a ceiling surface B perpendicular to the ground on the left side is photographed. In this case, the vanishing point exists in the right direction (the right side in the image or the right side outside the image), and the straight line toward the vanishing point has a large number of straight lines descending to the right.

  When photographing from the viewpoint VB in FIG. 5 in the direction of the arrow, the wall surface A, the ceiling surface B, the right wall surface C, etc. are photographed so as to become smaller toward the center as shown in FIG. 6B. In the image of FIG. 6B, a scene composed of a wall surface A and a wall surface C perpendicular to the left and right grounds and a ceiling surface B is photographed. In this case, the vanishing point exists near the center of the image, and the straight line toward the vanishing point has more straight lines toward the center (in other words, radiation from the center).

When photographing from the viewpoint VC in FIG. 5 in the direction of the arrow, photographing is performed so that the wall surface C becomes smaller in the left direction as shown in FIG. 6C. In the image of FIG. 6C, a scene composed of a wall surface C and a ceiling surface B perpendicular to the ground on the right side is photographed. In this case, the vanishing point exists in the left direction (the left side in the image or the left side outside the image), and the straight line toward the vanishing point has a large number of straight lines descending to the left.
The scene estimation unit 401 estimates which scene category the input image belongs to.
Although three types of scenes have been described as the scene classification, the scene classification need not be limited to those described above.

  FIG. 7 is a diagram showing an outline of the estimation process of the scene estimation unit 401. Next, a specific example of the estimation process of the scene estimation unit 401 will be described with reference to FIG. In the specific example described below, the scene estimation unit 401 estimates scene classification based on supervised learning. However, the combination of representative differential values described below is merely an example of a feature amount that can represent the scene configuration, and other values may be used as feature amounts that can represent the scene configuration.

  First, preprocessing will be described. A plurality of learning images are prepared for each scene. Next, images with a plurality of resolutions are created for each learning image. Next, each image is divided into 16 equal parts of 4 in the vertical direction and 4 in the horizontal direction. In the following description, 16 images generated by 16 equal divisions are referred to as “partial images”. FIG. 7A is a diagram illustrating a specific example of an image divided into 16 equal parts.

  Next, for each partial image, a differential image is created by extracting edge components in each direction. In the example of FIG. 7B, the edge in the vertical direction (y-axis direction), the edge in the horizontal direction (x-axis direction), the edge rotated 45 degrees counterclockwise from the x-axis, and the edge rotated 45 degrees clockwise from the x-axis A differential image obtained by extracting edge components in the four types of directions is generated. With this process, four types of differential images are generated for each direction for each partial image.

Next, a representative differential value of each differential image is calculated. The representative differential value is a statistical value (total value, average value, mode value, etc.) of the differential value in each differential image. More specifically, the differential image may be binarized, and the number of pixels having a pixel value of 1 (a value indicating that the edge strength is strong) may be used as a representative differential value. Alternatively, the differential image may be an image with 256 gradations, and the total value of the values of each pixel may be used as the representative differential value. Alternatively, the differential image may be an image having 256 gradations, and the average value of the pixel values may be used as the representative differential value.
At this time, 16 partial images are generated for one image, four differential images are generated for each partial image, and one representative differential value is calculated for each differential image. Therefore, 16 × 4 = 64 representative differential values are calculated for one image. The sequence of numbers of the representative differential values is used as the feature amount of the image. The above processing is performed for each resolution image, and each feature amount is calculated.

Next, a discriminator for estimating the scene classification is created based on the feature amount of each learning image having each resolution calculated as described above. FIG. 7C is a diagram illustrating a model of a discriminator. FIG. 7D is a diagram illustrating a pattern of the identification result by the classifier. The neural network learns the feature amount and the scene classification indicated by the feature amount as teacher data. As the neural network, for example, techniques disclosed in the following documents may be applied. Computer Vision State-of-the-Art Guide 2, Yoji Yagi, Hideo Saito, Adcom Media Co., Ltd., June 2010, p.96-98. The classifier is not necessarily limited to the neural network, and other classifiers such as SVM may be used. For example, techniques disclosed in the following documents may be applied as the SVM. Computer Vision Advanced Guide 2, Yagi Yasushi and Saito Hideo, Adcom Media Co., Ltd., published in June 2010, p.108-113.
The above processing is preprocessing.

  Next, processing of the scene estimation unit 401 will be described. The scene estimation unit 401 performs scene classification using the discriminator created in advance by preprocessing as described above. Specifically, it is as follows. The scene estimation unit 401 calculates the feature amount for the input image in the same manner as the process for creating the classifier (pre-processing). Then, the scene estimation unit 401 classifies the scene of the input image based on the calculated feature amount and a preset classifier. And the scene estimation part 401 estimates the position where a vanishing point exists in an input image based on a classification result.

  Next, the processing of the non-vertical direction straight line selection unit 402 will be described in detail. The non-vertical direction straight line selection unit 402 determines characteristics of a straight line (vanishing line) extending in the direction of the vanishing point based on the position of the vanishing point estimated by the scene estimation unit 401. Then, the non-vertical straight line having the determined characteristics is selected as a candidate non-vertical straight line from the non-vertical straight lines included in the input image.

  For example, when there is a vanishing point on the right side of the image (in the case of FIG. 6A), the non-vertical direction straight line selection unit 402 determines that the characteristic is a non-vertical direction straight line to be selected, that is, a downward slope. Then, a non-vertical straight line descending to the right is selected from the non-vertical straight lines included in the input image. For example, assuming a coordinate system in which the x-axis extends in the right direction and the y-axis extends in the upward direction, the non-vertical direction straight line selection unit 402 selects a non-vertical direction straight line with a negative slope of y = f (x). To do.

  For example, when there is a vanishing point at the center of the image (in the case of FIG. 6B), the non-vertical direction straight line selection unit 402 is a straight line that goes down toward the center of the image as a feature of the non-vertical direction straight line to be selected. judge. Then, a non-vertical line that falls toward the center of the image is selected from non-vertical lines included in the input image. For example, assuming a coordinate system in which the x-axis extends in the right direction and the y-axis extends in the upward direction, the non-vertical direction straight line selection unit 402 has a slope of y = f (x) for the straight line located on the left side of the screen. Select a non-vertical line that is negative. On the other hand, the non-vertical direction straight line selection unit 402 selects a non-vertical direction straight line having a positive slope for the straight line located on the right side of the screen.

  For example, when there is a vanishing point on the left side of the image (in the case of FIG. 6C), the non-vertical direction straight line selection unit 402 determines that the characteristic is a non-vertical direction straight line to be selected, that is, a left-down shoulder. Then, a non-vertical straight line descending to the left is selected from the non-vertical straight lines included in the input image. For example, assuming a coordinate system in which the x-axis extends in the right direction and the y-axis extends in the upward direction, the non-vertical direction straight line selection unit 402 selects a non-vertical direction straight line with a positive slope of y = f (x). To do.

  Next, the process of the candidate detection unit 403 will be described in detail. The candidate detection unit 403 determines the distance between the nearest end points for each combination of the candidate non-vertical direction straight line selected by the non-vertical direction straight line selection unit 402 and the vertical direction straight line selected by the vertical direction straight line detection unit 103. Is calculated. Then, the candidate detection unit 403 detects a combination in which the calculated distance is equal to or less than the endpoint distance threshold (for example, 3 pixels) as a combination candidate. The end point distance threshold is a value stored in advance in the candidate detection unit 403.

  In the second embodiment configured as described above, a non-vertical direction straight line is selected according to a scene estimation result. Therefore, it is possible to select the vanishing line constituting the vanishing point with higher accuracy. As a result, the vanishing point estimation accuracy can be improved.

<Modification>
The vanishing point estimation apparatus of the second embodiment may be configured to perform feedback processing. In the feedback processing, the scene estimation unit 401 corrects the estimation result based on the vanishing point position calculated by the position estimation unit 106. Specifically, the scene estimation unit 401 determines whether or not the position of the intersection calculated by the position estimation unit 106 matches the position of the vanishing point according to the scene estimation result.

  FIG. 8 is a diagram illustrating a specific example of feedback processing. In the scene of FIG. 8, there are two vanishing points, point 801 and point 802. When the scene estimation unit 401 estimates that the scene classification has only one vanishing point, the scene estimation unit 401 estimates the result of the feedback into the scene classification having two vanishing points as shown in FIG. To correct. The non-vertical direction straight line selection unit 402 selects a non-vertical direction straight line based on the corrected estimation result. Thereafter, based on the selection result, the vanishing point estimation device estimates the vanishing point.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes designs and the like that do not depart from the gist of the present invention.

DESCRIPTION OF SYMBOLS 100 ... Vanishing point estimation apparatus, 101 ... Image input part, 102 ... Straight line detection part, 103 ... Vertical direction straight line detection part, 104 ... Combination candidate detection part, 105 ... Vanishing line combination selection part, 106 ... Position estimation part, 107 ... Output unit 401 ... Scene estimation unit 402 ... Non-vertical direction straight line selection unit 403 ... Candidate detection unit

Claims (5)

Computer
A straight line detection step for detecting a straight line included in the input image;
A vertical straight line detecting step for detecting a vertical straight line that is a substantially vertical straight line from the straight lines;
Among combinations of a non-vertical direction straight line that is not the vertical direction straight line among the straight lines included in the image and the vertical direction straight line, a combination in which the distance between the end points is shorter than a threshold is detected as a combination candidate. A combination candidate detection step;
An erasure line combination selection step of selecting, from among the combination candidates, a combination candidate in which the vertical direction straight line and the non-vertical direction straight line are similar to each other,
A position estimating step for estimating a position of a vanishing point based on a plurality of the non-vertical direction straight lines in a plurality of selected vanishing line combinations;
A vanishing point estimation method comprising: In the combination candidate detection step, the computer
A scene estimation step for estimating the approximate position of the vanishing point in the input image;
Determining a feature of a straight line extending in the direction of the vanishing point in the input image based on the approximate position, and selecting the non-vertical straight line having the feature;
A candidate detection step of detecting, as a combination candidate, a combination in which the distance between the end points is shorter than a threshold among the selected combination of the non-vertical straight line and the vertical straight line;
The vanishing point estimation method according to claim 1, wherein: A straight line detection unit for detecting a straight line included in the input image;
A vertical straight line detection unit for detecting a vertical straight line that is a substantially vertical straight line from the straight lines;
Among combinations of a non-vertical direction straight line that is not the vertical direction straight line among the straight lines included in the image and the vertical direction straight line, a combination in which the distance between the end points is shorter than a threshold is detected as a combination candidate. A combination candidate detection unit;
An erasure line combination selection unit that selects, as the erasure line combination, a combination candidate in which the vertical line and the non-vertical line are similar from the combination candidates;
A position estimation unit that estimates a position of a vanishing point based on a plurality of the non-vertical direction straight lines in a plurality of selected vanishing line combinations;
A vanishing point estimation device comprising: The combination candidate detection unit
A scene estimator that estimates the approximate position of the vanishing point in the input image;
Determining a feature of a straight line extending in the direction of the vanishing point in the input image based on the approximate position, and selecting the non-vertical direction straight line having the feature;
A candidate detection unit that detects, as a combination candidate, a combination in which the distance between the end points is shorter than a threshold among the selected combinations of the non-vertical direction straight line and the vertical direction straight line;
The vanishing point estimation apparatus according to claim 3, comprising:   The computer program for making a computer perform each step of Claim 1 or Claim 2.