JP2009146150A - Feature position detection method and feature position detection apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect a feature position from image data having enhancement processing with subpixel accuracy. <P>SOLUTION: In a method, a template corresponding to the frame number (n-1) is applied to the image data of the frame number n in each frame whose image is successively taken, thereby detecting a tentative feature point position of the frame number n. The imaging object is a plate in which a graphic with the feature position at each of four vertexes of a rectangle is described. Next, the template corresponding to the feature position on the image of the frame number n is generated on the basis of the frame tentative feature point position. Subsequently, the template corresponding to the frame number n is applied to the image data of the frame number n, thereby detecting the frame feature point position of the frame number n. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image recognition technique.

At present, as a calibration method for a camera (imaging device; for example, a digital camera), a plate depicting a checkered pattern (for example, a plate P11 depicting a checkered pattern _PICH in FIG. 20) is used as the camera (for example, in FIG. 20). captured by the camera C), (intersection checkered on the image data) (e.g., separate the checkered pattern P _ICH (or constituting) image the captured imaginary line (or border-like shades該模(Shading boundary line)) A method for obtaining an internal parameter of a camera based on each intersection K _P ) of the groups K ₁ and K ₂ is widely used. In the method for obtaining the internal parameters, since it is important to obtain the intersection position with high resolution at the resolution of the subpixel, conventionally, a method for detecting the intersection position in many subpixels has been proposed.

  For example, in the above-described example of the intersection position detection method (see, for example, Non-Patent Document 1), first-order and second-order luminances are obtained at the intersection position obtained with pixel accuracy, and sub-pixel accuracy is obtained from these differential values. The intersection position (that is, a kind of feature point) is calculated. Thus, in the conventional method, it calculates based on the 1st derivative and 2nd derivative of a brightness | luminance.

As a related technique, a posture estimation method from a position group of feature points (for example, see Non-Patent Document 2) is also known.
Dazhi Chen, Guangjun Zhang, “Short Papers; A New Sub-Pixel Detector for X-Corners in Camera Calibration Targets,” UNION Agency Pc. 97-100. Koichiro Deguchi, “Unified Solution of PnP Camera Problems by Projective Geometry”, Information Symposium, 1990 (Heisei 2), Vol. 90, pp. 41-50.

  However, in a web camera (a kind of digital camera), edge enhancement processing is often performed on captured image data. For example, FIG. 21 shows a luminance curve in image data at a boundary portion in a black-and-white (or light and shade) pattern image. FIG. 21A shows a luminance curve L1 in ideal image data, and FIG. It is the brightness | luminance curve L2 in the emphasized image data.

  For the image data subjected to the edge enhancement processing as shown in FIG. 21B, the above-described differential value is not obtained as a correct value, and as a result, the calculated sub-pixel position of the intersection is inaccurate. There is a known problem.

  Therefore, an object of the present invention is to provide a feature position detection method and a feature position detection apparatus that detect feature positions with sub-pixel accuracy from image data that has undergone edge enhancement processing.

  The present invention is a technical idea created to solve the above-described problems, and detects a provisional feature position for each frame image data of a moving image, and detects the sub-pixel accuracy based on the provisional feature position. A template for this is sequentially created, and this template is applied to finally detect the feature position.

  Specifically, the invention according to claim 1 is a feature position detection method for detecting a feature position on an object from images of each frame obtained by continuously imaging the object, and includes provisional feature point position detection means. Applying a template for the previous frame to the image of the current frame to detect a provisional feature position of the image of the current frame; and a template creating means, based on the provisional feature position, Creating a template for the current frame corresponding to the feature position of the current frame, and detecting a feature position of the image of the current frame by applying the template for the current frame to the image of the current frame by the feature position detecting means It is characterized by having.

  The invention described in claim 2 is characterized in that a template group consisting of images of objects shifted in sub-pixel units is used as the template.

  The invention described in claim 3 uses, as the object, a plate in which a figure having the characteristic position is drawn at each of four vertices of a rectangle, and the figure does not change in local appearance depending on the scale. It is characterized by that.

  According to a fourth aspect of the present invention, the template generating unit creates a luminance distribution group from a luminance pattern of an image obtained by imaging the plate having two shades of light, and based on the luminance distribution group, An intermediate luminance ideal luminance pattern is obtained from the dark portion luminance, a step luminance pattern which is an ideal luminance pattern is created, a synthetic luminance pattern obtained by applying an edge enhancement kernel to the step luminance pattern, and the synthetic luminance pattern And a step of creating a parameter that minimizes a difference between the actual luminance pattern and an edge enhancement step of the current frame image using the parameter and the edge enhancement kernel.

  According to a fifth aspect of the present invention, at least three plane figures in which the areas formed by the first and second boundary lines are alternately painted on the plate are depicted on the plate, Are arranged on the same plane and coincide with the positions of any three of the four intersections corresponding to the intersection of two pairs of parallel straight lines having different directions, The line is characterized by being parallel to the two pairs of parallel straight lines.

  The invention according to claim 6 is a feature position detecting device for detecting a feature position on an object from images of each frame obtained by continuously imaging the object, and for the image of the current frame. Applying a template to detect a temporary feature point position detecting means for detecting a temporary feature position of an image of the current frame, and creating a template for the current frame corresponding to the feature position of the image of the current frame based on the temporary feature position And a feature position detecting means for detecting the feature position of the image of the current frame by applying the template for the current frame to the image of the current frame.

  The invention described in claim 7 is characterized in that a template group consisting of images of objects shifted in sub-pixel units is used as the template.

  The invention described in claim 8 uses, as the object, a plate in which a figure having the characteristic position is drawn at each of four vertices of a rectangle, and the figure does not change in local appearance depending on the scale. It is characterized by that.

  In the invention according to claim 9, the template generation means generates a brightness distribution group from a brightness pattern of an image obtained by imaging the plate having two shaded areas, and based on the brightness distribution group, Means for obtaining an intermediate luminance ideal luminance pattern from the dark portion luminance, creating a step luminance pattern which is an ideal luminance pattern, means for creating a synthetic luminance pattern obtained by applying an edge enhancement kernel to the step luminance pattern, and the synthetic luminance pattern And means for creating a parameter that minimizes the difference between the actual luminance pattern and the means for edge enhancement of the image of the current frame by the parameter and the edge enhancement kernel.

  The invention according to claim 10 is characterized in that at least three plane figures in which the area formed by the gray boundary line 1 and the gray boundary line 2 are alternately painted are drawn on the plate, Are arranged on the same plane and coincide with the positions of any three of the four intersections corresponding to the intersection of two pairs of parallel straight lines having different directions, The line is characterized by being parallel to the two pairs of parallel straight lines.

  In the present invention, it is preferable to use the frame (n−1) for the current frame n as the previous frame, but the present invention is not limited to this, and the frame (n−2.3. K).

  According to the first to tenth aspects of the present invention, it is possible to detect the feature position with high accuracy with sub-pixel accuracy from the image data captured by the imaging device that is automatically applied using the template image data.

  Hereinafter, a feature position detection method according to an embodiment of the present invention will be described in detail with reference to the drawings. In the present embodiment, image data that has been captured by a camera (for example, a web camera) that has been subjected to edge enhancement processing and edge-enhanced is targeted.

  In the present embodiment, since the method for obtaining the position of the intersection point by subpixel based on the first and second luminances of the luminance near the intersection of the checkered pattern as described above cannot be used, the gray level (or luminance) pattern of the intersection is not used. A method of using template image data (hereinafter simply referred to as a template) is employed.

  However, in the following description, the template has a variation substantially corresponding to a combination of two angles (openings described later). The reason for the number of combinations is as follows. For example, when the degree of discreteness is set as a condition, 32400 (= 180 × 180) types of templates are required, and in addition, when it is necessary to detect with subpixel accuracy, more templates are required. Need. It is actually difficult to hold all template groups for realizing such subpixel detection accuracy on a storage unit (for example, a memory) while maintaining a sufficiently fine discreteness.

The feature position detection method in this embodiment has the following procedures.
(1) Template Group Creation Procedure (2) Subpixel Accuracy Feature Position Estimation Procedure A template group creation procedure in the feature position detection method of this embodiment will be described with reference to FIG. FIG. 1 is a main flowchart of a template group creation procedure.

  First, a parameter for edge enhancement processing is estimated from image data (hereinafter referred to as template input image data) captured by a web camera or the like and edge-enhanced (S101: parameter estimation step for edge enhancement processing).

  Next, a template group for detecting a feature position with subpixel accuracy is generated using supersampling (S102: template group generation step).

  Then, edge enhancement is performed on the template group based on the estimated parameters (edge enhancement processing is performed) (S103: edge enhancement step of the template group).

  The parameter estimation step (S101) of the edge enhancement process will be described based on the flowchart in FIG.

First, a plate (for example, FIG. ₃ ) on which a light / dark region with a slight inclination of a light / dark (or black / white or low / high luminance) region boundary line (virtual line) K ₃ is drawn by the camera (for example, camera C in FIG. 3). 3 is imaged. The captured image data (that is, template input image data) is fetched with a luminance pattern X _j for each scan line (i), and a histogram (luminance histogram data; for example, histogram H _{g in} FIG. 4) is created (S201: Luminance pattern capture histogram creation step).

Next, an intermediate luminance I _M (= (I _B + I _D ) / 2) is obtained from the bright portion luminance I _B and the dark portion luminance I _D on the scan line S _C based on the created histogram, and the step is an ideal luminance pattern A brightness pattern Y _j (eg, step brightness pattern Y _j in FIG. 5) is created (S202: step brightness pattern creation step).

Next, a synthesized luminance pattern Z _ij (see the following formula) is created by applying the Gaussian kernel with the parameter σ and the edge enhancement kernel with the parameter α to the created step luminance pattern Y _ij (S203: synthesized luminance pattern) Creation step).

  In the above equation, G is a Gaussian kernel with a parameter σ, and H is an edge enhancement kernel with a parameter α. Note that the edge emphasis kernel H is the following expression where the weight is w. FIG. 6 is a diagram illustrating an example of the edge enhancement kernel H.

  Then, parameters σ and α that minimize the evaluation value E shown in the following equation are obtained, and the obtained parameter α is regarded as a parameter for edge enhancement processing (S204: parameter calculation step for edge enhancement processing).

  Equation 2 is an equation for evaluating the difference between the result of applying the Gaussian kernel and the edge enhancement kernel to the ideal step luminance pattern and the actual luminance pattern, and Equation 1-1 makes it easy to understand part of the above evaluation. To be separated.

  The template group generation step (S102) will be described based on the flowchart in FIG. 7 and FIG. It is assumed that the size of the template to be created is smaller than the size of the template input image data. The template to be created is a template in which a checkered pattern (hereinafter referred to as a one-intersection checkered pattern) having only one intersection is drawn (hereinafter referred to as a modified one-intersection checkered pattern).

First, assume a blank template (blank template having a size of 16 × 16 (= t _X × t _Y ) pixels) t _B having the same size as the template to be created, and further, in the blank template t _B , _Assume K _P1 (= coordinates (K _X , K _Y )). Subsequently, a blank template t _MB having a size obtained by scaling the blank template at a magnification N (hereinafter referred to as a scaled blank template) _tMB is created (S301: Step of creating an image having a size N times that of the template). The blank template is, for example, image data in which all pixels are filled with a pixel value 0 (black pixel) or a maximum pixel value (white pixel).

For example, when the size of the template to be created is 16 × 16 pixels and the magnification N = 10, the size of the scaled blank template to be created is 160 × 160 pixels. In FIG. 8, the size of the template to be created is 16 × 16 pixels, and the magnification N = 2. That is, the size t _MBX × t _MBY of the variable-magnification blank template t _MB is 32 × 32 pixels.

Reference point K _P1, in zooming blank template t _MB, the reference point K _P11 (= coordinates _{(K X × N, K Y} × N)) conversion reference point region containing the (size of the area of the N × N pixels) K Converted to _PM .

Deformation 1 intersection The intersection of the checkered pattern is the origin of the reference point K _P11 in the variable magnification blank template t _MB , and each pixel K _P (= coordinate (K _X × N + I, K) in the N × N pixel size region extending in the XY direction. _Y × N + J), 0 ≦ I ≦ N−1, or 0 ≦ J ≦ N−1). In the following description, the coordinates of the intersection point K _P of the deformation 1 intersection checkered pattern will be described as the coordinates of the reference point K _P11 .

Next, a deformed one-intersection checkered pattern (light / dark pattern) is converted into a variable blank template t _MB based on the light / dark area boundary lines K ₄ and K ₅ with the reference point K _P11 as an intersection. Drawing is performed to obtain a magnification template t _MP (S302: Step of generating a pattern with an opening degree designated as a reference position). Note that the opening is an angle formed by the inclination of a boundary (a shaded area boundary line that is a border of a shaded pattern) due to a skew generated on an image when a checkered plate is tilted with respect to the camera (for example, FIG. 9 is the opening degree θ ₁ , θ ₂ ). For the sake of convenience, values of the opening degrees θ ₁ and θ ₂ in FIG. 9 are different from those in FIG. 10 (a) and 10 (b) are examples of the opening when the shaded area boundary lines K ₄ and K ₅ are orthogonal to each other, and the first quadrant in FIG. 10 (a) is black and the second quadrant. Is white, the opening θ ₁ is 0 degree and the opening θ ₂ is 90 degrees. When the first quadrant in FIG. 10B is white and the second quadrant is black, the opening θ ₁ is 90 degrees and the opening θ ₂ is 180 degrees.

Next, the scaling template t _{MP on} which the deformation 1 intersection checkered pattern (light and shade pattern) is drawn is reduced to 1 / N, and a template t _P having the same size as the template to be created is obtained (S303: 1 / N). Step to shrink).

Then, it is checked whether or not the number of generated templates t _P is smaller than N × N (S304). If the number of generated templates t _P is smaller than N × N, the process proceeds to step S305. If the number of generated templates t _P is not smaller than N × N, the process is terminated.

In step S305 (step of shifting the reference position by one pixel), the intersection point K _P is shifted by one pixel, and the process proceeds to step S301.

  Through the above processing, the above processing is performed at all reference positions to create a template group. For example, when N = 10, the number of templates in the created template group is 100. Each template in the template group is identified with the reference position at the time of creation, that is, the two-dimensional coordinates (I, J) in the conversion reference point region on the scaling template image as an identifier.

  The template group edge enhancement step (S103) will be described in more detail. The image of each template in the template group created in step S102 is edge-enhanced using the parameter α estimated in step S101 and the edge enhancement kernel H shown in FIG.

  A procedure for estimating a feature position with sub-pixel accuracy in the feature position detection method of this embodiment will be described with reference to FIG. FIG. 11 is a main flowchart of a procedure for estimating a feature position with sub-pixel accuracy.

First, in each pixel position (x, y) (for example, a region having the pixel position d in FIG. 12) of input image data (for example, input image data B _P in FIG. 12), a template (0, 0) ( For example, by applying the template t ₀ in FIG. 12, the pixel position (initial similar pixel position) most similar to the template (0, 0) is determined as the pixel position (X, Y) (S401: initial similar pixel position). Decision step). Note that when checking the similarity, for example, by aligning the pixel position d _t on pixel position d and the template t ₀ in FIG. 12 checks the similarity.

Next, all the templates (I, J) of the template group (for example, template group T in FIG. 12) are applied to the pixel position (X, Y) and its eight neighboring pixel positions, and the most similar to them. The pixel position to be considered is an 8-neighborhood similar pixel position (pixel position (X _S , Y _S )), and the most similar 8-neighborhood similar pixel position template (template (I _S , J _S )) is set (S402: 8 Similar pixel position determination step for neighboring pixels).

Then, the position (X _S + I _S / N, Y _S + J _S / N) is set as a subpixel accuracy feature position (ie, intersection) (S403: subpixel accuracy feature position calculation step). N is the magnification N described above.

  Next, the configuration of the feature position detection apparatus in the present embodiment will be described with reference to FIG.

  The feature position detection device in FIG. 13 executes the above-described subpixel accuracy feature position estimation procedure (steps S401 to S403) on the input image data read from the image data management unit 103 (described later) to obtain a template. Using the template group stored in the management unit 104 (described later), the feature point position detection unit 101 that detects the feature position, the above-described template group creation procedure (steps S101 (or S201 to S204), S102 (or S301) To S305) and S103) to create a template group and store it in a template management unit 104 (described later), an image data management unit that stores and manages image data that is a target for detecting feature positions 103, a template image for calculating parameters of edge enhancement processing Including a template management unit 104 that stores and manages data and template groups, a CPU (Central Processing Unit), an OS (Operating System), etc., and a control unit (not shown) that controls each unit, a memory, a hard disk drive device, etc. A storage unit (not shown) that stores information or data used in the apparatus is provided, and is, for example, a computer.

  The operation of each part of the feature position detection device has been described in detail in the above procedure. All or part of the feature position detection device may be implemented as a program executable by a computer. The image data management unit 103 and the template management unit 104 may be implemented in the storage unit described above.

  Below, an example of operation | movement of the characteristic position detection apparatus in this embodiment is demonstrated. Refer to the above procedure for details of the operation.

  The template creation unit 102 acquires the template image data obtained by imaging the plate on which the shaded area where the shaded area boundary line is slightly inclined is captured from the template management unit 104, and captures the luminance pattern for each scan line of the template image data. Create brightness histogram data.

  Next, the template creation unit 102 obtains intermediate brightness from the bright part brightness and the dark part brightness based on the created histogram data, and creates a step brightness pattern that is an ideal brightness pattern.

  Next, the template creation unit 102 creates a composite luminance pattern in which the created step luminance pattern is subjected to a Gaussian kernel with a parameter σ and an edge enhancement kernel with a parameter α.

  Next, in the expression for evaluating the difference between the result of applying the Gaussian kernel and the edge enhancement kernel to the ideal step luminance pattern and the actual luminance pattern, the template creation unit 102 obtains a parameter that minimizes the evaluation value based on the expression. The obtained parameter is regarded as a parameter α for edge enhancement processing.

  Next, the template creation unit 102 scales the blank template image data to be created having a reference point at a magnification N, and the scale change has a converted reference point region in which the reference point is converted. Create blank template image data.

  Next, the template creation unit 102 uses a point included in the conversion reference point region as an intersection, and uses a variable one-point checkerboard pattern based on a shaded region boundary line that forms a predetermined opening degree as the scaling blank template. The scaling template image data drawn in (1) is obtained.

  Next, the template creation unit 102 reduces the scaled template image data on which the modified 1 intersection checkerboard pattern is drawn to 1 / N, and obtains template image data having the same size as the template image data to be created, The template image data is stored in the template management unit 104.

  Next, the template creation unit 102 edge-enhances the image of each template image data of the template group stored in the template management unit by using the estimated parameter α and the edge enhancement kernel, and the template management is newly performed as template image data. Stored in the unit 104.

  Next, the feature point position detection unit 101 applies the specific template image data stored in the template management unit 104 at each pixel position of the input image data acquired from the image data management unit 103, and performs the specification. The initial similar pixel position most similar to the template image data is determined.

  Next, the feature point position detection unit 101 applies all the template image data of the template image data group to the initial similar pixel position and the eight neighboring pixel positions of the initial similar pixel position, and is most similar to them. The pixel position is regarded as an 8-neighborhood similar pixel position, and the most similar template is regarded as an 8-neighborhood similar pixel position template.

  Then, the feature point position detection unit 101 adds the value obtained by dividing the coordinate of the intersection point in the conversion reference point region of the 8-neighborhood similar pixel position template by the magnification N to the 8-neighborhood similarity pixel position, and the addition is performed. Is regarded as the feature position of subpixel accuracy.

[Example]
The feature position detection method according to the present embodiment sequentially creates a template group for detection with sub-pixel accuracy based on a provisional feature position for each frame image data in a moving image, and applies the created template group again. By doing so, the feature position with sub-pixel accuracy is finally estimated.

  Next, an example in which the feature position detection method in the present embodiment is applied will be described. In this embodiment, a plate on which a checkerboard pattern described below is drawn and an image data group obtained by imaging the plate (for example, an image group obtained by continuously imaging the plate while tilting each frame image data in a moving image) (Hereinafter simply referred to as a frame image). It is assumed that a number (frame number n) for identifying each frame image is given to the frame image.

  A plate on which a checkered pattern used in this embodiment is drawn will be described with reference to FIG.

Four checkered P _ICH1 in FIG _{14, P ICH2, P ICH3,} P ICH4 is 1 intersection a checkered pattern, in the plate P31, intersection _{K P21, K P22, K P23} , K P24 virtual square It is drawn so as to be arranged at the position of the vertex of Q. However, there are three or more checkered checkered patterns, and they may be arranged to form an arbitrary polygon instead of a linear arrangement. However, in order to reduce the calculation cost of posture calculation, in the following description, the one-intersection checkered pattern is arranged line-symmetrically and point-symmetrically.

  The feature position detection method in the present embodiment will be described with reference to the flowchart of FIG. 15 and FIG. FIG. 16 is a diagram illustrating how the feature position is detected. In the following, the one corresponding to the frame number n is described as U (n), for example.

First, the frame template group T (n−1) is applied to the frame image F (n) to detect the frame provisional feature point position K _TP (n) (S501: frame provisional feature point position detection step). ). For example, in FIG. 16, the frame temporary feature point position K _TP2 is detected by applying the frame template group T ₁ to the frame image F ₂ . In step S501, the frame provisional feature point position K _TP (n) is detected using the above-described procedure for estimating the feature position with sub-pixel accuracy. The template shown in FIG. 10 may be used for the frame template group T ₁ .

Next, based on the frame temporary feature point position K _TP (n), a frame template group T (n) corresponding to each one intersection checkered pattern on the frame image F (n) is generated (S502: frame template). Group generation step). Details will be described later. For example, in FIG. 16, a frame template group T ₂ corresponding to each one checkered checkerboard pattern on the frame image F ₂ is generated based on the frame provisional feature point position K _TP2 .

Next, the frame feature point position K _P (n) is detected by applying the frame template group T (n) to the frame image F (n) (S503: frame feature point position detecting step). For example, in FIG. 16, the frame feature point position K _P2 is detected by applying the frame template group T ₂ to the frame image F ₂ . In step S503, the frame feature point position K _P (n) is detected by using the above-described procedure for estimating the feature position with sub-pixel accuracy.

If all the frame image groups have not been processed, the image number n is advanced (added), and the process proceeds to step S501. For example, in FIG. 16, the frame image F ₃ is processed to generate a frame temporary feature point position K _TP3 , a frame template group T ₃ , and a frame feature point position K _P3 . If the above-described processing is performed for all the frame image groups F, the processing is terminated.

  As described above, it is possible to detect feature point positions with higher accuracy by repeating the processes of step S502 and step S503.

  Next, step S502 will be described in detail based on the flowchart of FIG. 17 and FIG.

First, the vanishing point positions V1 and V2 on the frame image F (n) are calculated based on the frame temporary feature point position K _TP (n) (S601: vanishing point calculating step). For example, feature points in FIG. 18 (i.e., 'the intersection of K _P21 variant 1 intersection checkerboard P _ICH1)' and the feature point (i.e., deform one-intersection checkerboard P 'intersection of K _{P22 ICH2)'} straight connecting L _V11 and the feature point (i.e., 'the intersection of K _P24 variant 1 intersection checkerboard P _ICH4)' intersection of the feature points (i.e., deformation 'intersection of K _P23 1 intersection checkerboard P _ICH3)' straight line L _V12 connecting the Is calculated as the vanishing point position V1. Similarly, the vanishing point position V2 is calculated based on the straight line L _V21 and the straight line L _V22 .

Next, based on the calculated vanishing point positions V1 and V2, the opening degree (that is, the shading) at the frame temporary feature point position K _TP (n) that is the intersection position of each one intersection checkered pattern on the frame image F (n). (Slope of region boundary line) is calculated (S602: opening degree calculating step).

For example, the opening degree at the intersection K _P21 ′ is calculated based on the inclination of the straight line L _V11 connecting the vanishing point position V1 and the intersection K _P21 ′ and the inclination of the straight line L _V21 connecting the vanishing point position V2 and the intersection K _P21 ′. To do.

  Then, based on the opening calculated in step S602, the above-described template group creation procedure is used to generate a frame template group T (n) (S603: frame template group generation step based on opening). .

In general, the position and orientation of the plate are estimated from the frame provisional feature point position K _TP (n), and on the frame image F (n) based on the position and orientation of each checkered checkerboard pattern on the plate that is known in advance. Alternatively, the gradient of the shaded area boundary line at the intersection of each one intersection checkered pattern may be obtained.

  In addition, since the well-known method (refer nonpatent literature 2) is known about the method of estimating the position and orientation of a plate from the feature point position on the frame image F (n), description is abbreviate | omitted. However, “posture estimation from the position group of feature points” has many existing techniques and is not limited to the method described in Non-Patent Document 2.

  Next, the feature position detection apparatus in the present embodiment will be described with reference to FIG.

  The feature position detection apparatus in FIG. 19 estimates the feature position with the above-mentioned sub-pixel accuracy for the image data (frame image) group obtained by imaging the plate as described above read from the image data management unit 204 (described later). Feature point positions that execute the procedure (steps S401 to S403) and steps S501 and S503, and detect the frame temporary feature point position and the frame feature point position using the template group stored in the template management unit 205 (described later) The detection unit 201, the above-described template group creation procedure (steps S101 (or S201 to S204), S102 (or S301 to S305), S103) and step S502 are executed, and a plate posture detection unit 203 (described later) is used. , Create a template group, template management unit 205 (after The template creation unit 202 stored in step S601), the plate posture detection unit 203 that calculates the opening degree based on the frame provisional feature point position using the procedure in steps S601 to S603 described above, and the frame that is the target for detecting the feature position. An image data management unit 204 for storing and managing image data, a template management unit 205 for storing and managing template groups, a CPU (Central Processing Unit), an OS (Operating System), and the like, and a control unit (not shown) for controlling each unit , A memory, a hard disk drive device, etc., and a storage unit (not shown) for storing information or data used in the device, for example, a computer.

  The operation of each part of the feature position detection apparatus in the present embodiment has been described in detail in the above procedure. All or part of the feature position detection device may be implemented as a program executable by a computer. The image data management unit 204 and the template management unit 205 may be implemented in the storage unit described above.

  Below, an example of operation | movement of the characteristic position detection apparatus in a present Example is demonstrated. Refer to the above procedure for details of the operation.

The feature point position detection unit 201 applies the frame template image data group T (n−1) stored in the template management unit to the frame image data F (n) acquired from the image data management unit 204. The frame temporary feature point position K _TP (n) is detected (temporary feature point position detecting means).

Next, the plate posture detection unit 203 calculates an opening degree corresponding to the frame number n based on the detected frame temporary feature point position K _TP (n) (opening degree calculating means).

  Next, the template creation unit 202 selects a frame template group T (n) corresponding to each one intersection checkered pattern on the frame image data F (n) based on the opening degree corresponding to the calculated frame number n. Generate (frame template group generation means).

Then, the feature point position detection unit 201 applies the frame template image group T (n) to the frame image data F (n) to detect the frame feature point position K _P (n) (feature point position detection). means).

Further, the opening degree calculating means calculates a vanishing point position on the frame image data F (n) based on the frame provisional feature point position K _TP (n), and based on the calculated vanishing point position, the frame image. An opening degree corresponding to the frame number n at the frame temporary feature point position, which is the intersection position of each one intersection checkered pattern on the data F (n), is calculated.

Further, the frame temporary feature point position detecting means detects a frame template image data group T (n−) stored in the template management unit 205 at each pixel position of the frame image data F (n) acquired from the image data management unit 204. 1) is applied, and the pixel position that is most similar to the specific template image data in the frame template image data group is determined as the initial similar pixel position. Subsequently, all the template image data of the template image data group T (n−1) are applied to the initial similar pixel position and the eight neighboring pixel positions of the initial similar pixel position, and the pixel position most similar to them is applied. Is considered as an 8-neighborhood similar pixel position, and the most similar template is considered as an 8-neighborhood similar pixel position template. Then, a value obtained by dividing the coordinate of the intersection point in the conversion reference point region of the 8-neighborhood similar pixel position template by the magnification N is added to the 8-neighborhood similar pixel position, and the added value is used as the frame temporary feature point position. _Consider K _TP (n).

  Further, the frame template group generation means acquires template image data obtained by imaging a plate on which a gray area with a slight gray area boundary line slightly inclined is acquired from the template management unit 205, and for each scan line of the template image data. The brightness pattern is taken in and brightness histogram data is created. Subsequently, based on the created histogram data, intermediate brightness is obtained from the bright part brightness and the dark part brightness, and a step brightness pattern which is an ideal brightness pattern is created. Subsequently, a synthesized luminance pattern is created by applying the Gaussian kernel with the parameter σ and the edge enhancement kernel with the parameter α to the created step luminance pattern. Next, in the equation that evaluates the difference between the result of applying the Gaussian kernel and edge enhancement kernel to the ideal step luminance pattern and the actual luminance pattern, find the parameter that minimizes the evaluation value by that equation, and calculate the calculated parameter This is regarded as a parameter α for edge enhancement processing.

  Further, the frame template group generation means scales the blank template image data to be created having a reference point by a magnification N, and the scale template has a conversion reference point region into which the reference point has been converted. Create double blank template image data. Subsequently, a one-intersection checkered pattern having only one intersection point based on a shaded region boundary line having an opening corresponding to the calculated frame number n as an intersection point, which is included in the conversion reference point region The scaled template image data obtained by drawing the deformed one-intersection checkered pattern, which is a pattern transformed from the above, on the scaled blank template is obtained. Then, the scaled template image data on which the modified 1 intersection checkerboard pattern is drawn is reduced to 1 / N, template image data having the same size as the template image data to be created is obtained, and the template image data is obtained as a template image. The data is stored in the template management unit 205 as template image data of the data group T (n). If necessary, when the number of template image data to be acquired is less than a predetermined number, the intersection point included in the conversion reference point region is shifted by one pixel.

  Further, the frame template group generation means edge-enhances each template image data image of the template image data group T (n) stored in the template management unit 205 using the estimated parameter α and the edge enhancement kernel, and re-templates the template. It is stored in the template management unit 205 as template image data of the image data group T (n).

  Further, the frame feature point position detection means uses the frame template image data corresponding to the frame number n stored in the template management unit 205 at each pixel position of the frame image data F (n) acquired from the image data management unit 204. The group T (n) is applied, and the pixel position most similar to the specific template image data is determined as the initial similar pixel position. Subsequently, all the template image data of the template image data group corresponding to the frame number n is applied to the initial similar pixel position and the eight neighboring pixel positions of the initial similar pixel position, and the pixel position most similar to them Is considered as an 8-neighborhood similar pixel position, and the most similar template is considered as an 8-neighborhood similar pixel position template. Then, a value obtained by dividing the coordinate of the intersection point in the conversion reference point region of the 8-neighborhood similar pixel position template by the magnification N is added to the 8-neighborhood similar pixel position, and the added value is determined according to the frame number n. It is considered as a feature position with sub-pixel accuracy.

  It should be noted that each step in the method or procedure in the present embodiment and examples may be configured as means (for example, means configured by a computer program), and the feature position detection apparatus may be implemented.

  According to the above-described embodiment, the feature point position detection method for one piece of image data obtained by imaging a plate at a certain position and orientation is applied, for example, from a series of images obtained by imaging a plate whose position and orientation changes slowly. It can be well adapted to tracking such as detecting feature point positions one after another.

  Although the embodiments of the present invention have been described above, the present invention is not limited to the described embodiments, and various modifications can be made within the scope described in each claim.

  For example, although the image data management unit and the template management unit are individually provided, they may be mounted together in a general-purpose database.

  In the present embodiment, the image data of the current frame is F (n) while the image data of the previous frame is F (n−1). However, the present invention is not limited to this. F (n−2.3... K) may be used for the image data of the previous frame.

The figure which shows the preparation procedure of the template group in this embodiment. The figure which shows the procedure which estimates the parameter of edge emphasis processing. The figure which shows an example of the plate which drew the shade area | region in the case of estimating the parameter of an edge emphasis process. The figure which shows an example of the histogram of a shading area | region. The figure which shows an example of the step brightness | luminance pattern based on a histogram. The figure which shows an example of an edge emphasis kernel. The figure which shows the procedure which produces | generates the template group for detecting a feature position with subpixel precision. The figure which shows a mode that a template group is created. The figure which shows Example 1 of an opening degree. FIG. 10A is an example of an opening when the gray area boundary lines are orthogonal, and FIG. 10A shows the opening when the first quadrant is black and the second quadrant is white, and FIG. 10B shows the first quadrant. The figure which shows the opening degree when it is white and the 2nd quadrant is black. The figure which shows the estimation procedure of the feature position of the subpixel precision in this embodiment. The figure which shows a mode that a feature position is estimated using a template group. The block diagram of the feature position detection apparatus in this embodiment. The figure which shows the plate by which the checkered pattern used by a present Example was drawn. The figure which shows the feature position detection method in a present Example. The figure which shows a mode that the feature position of subpixel accuracy is detected. The figure which shows the template group production | generation procedure for frames. The figure which shows the step which calculates an opening degree. The block diagram of the characteristic position detection apparatus in a present Example. The figure which shows the plate which drew the checkerboard pattern in calibration. FIG. 21A shows a luminance curve in the image data of the boundary portion in the monochrome pattern image, FIG. 21A shows a luminance curve in the ideal image data, and FIG. 21B shows a luminance curve in the edge-enhanced image data. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ... Feature point position detection part 102 ... Template preparation part 103 ... Image data management part 104 ... Template management part 201 ... Feature point position detection part 202 ... Template preparation part 203 ... Plate attitude | position detection part 204 ... Image data management part 205 ... Template Management unit B _P ... Input image data C ... Camera F ₂ , F ₃ ... Frame image H ... Edge enhancement kernel H _g ... Histogram K ₁ , K ₂ ... Virtual line group K ₃ ... Light / dark region boundary line (virtual line)
K ₄ , K ₅ ... shaded area boundary line K _P ... intersection point K _P1 , K _P11 ... reference point K _P2 , K _P3 ... frame feature point position K _P21 , K _P22 , K _P23 , K _P24 ... intersection point of checkered pattern K _P21 ′, K _P22 ′, K _P23 ′, K _P24 ′… Characteristic points (intersection of 1 intersection checkerboard pattern)
K _PM ... conversion reference point area K _TP2 , K _TP3 ... frame provisional feature point position L1, L2 ... luminance curve L _V11 , L _V12 , L _V21 , L _V22 ... straight line N ... vertical and horizontal sizes of the conversion reference point area ( Magnification value)
P _ICH ... checkered pattern _{P ICH1, P ICH2, P ICH3} , P ICH4 ... 1 intersection checkered pattern _{P ICH1 ', P ICH2',} P ICH3 ', P ICH4' ... deformation 1 intersection checkered pattern P11, P21, P31 ... plate Q ... Virtual square S _C ... Scan line T ... Template group T ₁ , T ₂ , T ₃ ... Frame template group V1, V2 ... Vanishing point position Y _j ... Step luminance pattern d ... Specific pixel position of input image data d _t ... template pixel position t ₀ ... template t _B ... blank template t _MB ... scaling blank template t _MBX ... scaling blank template horizontal size t _MBY ... scaling blank template vertical size t _X ... template horizontal size t _Y ... Vertical size of template t _MP ... Scaling template t _P ... Template θ ₁ , θ ₂ ... Opening

Claims (10)

A feature position detection method for detecting a feature position on an object from images of each frame obtained by continuously capturing the object,
A provisional feature point position detection unit that applies a template for a previous frame to an image of a current frame to detect a provisional feature position of the image of the current frame;
A template creating means creating a template for the current frame corresponding to the feature position of the image of the current frame based on the provisional feature position;
Applying a template for the current frame to the image of the current frame to detect a feature position of the image of the current frame;
A feature position detection method characterized by comprising:   The feature position detection method according to claim 1, wherein a template group including images of an object shifted in units of subpixels is used as the template. As the object, a plate in which a figure having the characteristic position is drawn at each of four vertices of a rectangle is used,
3. The feature position detection method according to claim 1, wherein the graphic does not change in local appearance depending on a scale. The template generation means is
Creating a luminance distribution group from a luminance pattern of an image obtained by imaging the plate having two shaded areas;
Based on the luminance distribution group, obtaining an intermediate luminance ideal luminance pattern from the bright portion luminance and the dark portion luminance, and creating a step luminance pattern that is an ideal luminance pattern;
Creating a composite luminance pattern obtained by applying an edge enhancement kernel to the step luminance pattern;
Creating a parameter that minimizes the difference between the synthesized luminance pattern and the actual luminance pattern;
Edge enhancing the current frame image with the parameters and the edge enhancement kernel;
The feature position detection method according to claim 3, wherein: In the plate, at least three plane figures are drawn in which the areas formed by the first and second boundary lines are alternately shaded,
The intersections of the two shaded area lines are arranged on the same plane and coincide with the positions of any three of the four intersections corresponding to the intersections of two pairs of parallel straight lines having different directions. ,
5. The feature position detection method according to claim 3, wherein each of the shade area lines is parallel to the two pairs of parallel straight lines. A feature position detection device for detecting a feature position on an object from images of each frame obtained by continuously capturing the object,
Provisional feature point position detecting means for applying a template for the previous frame to the image of the current frame to detect the provisional feature position of the image of the current frame;
Template creating means for creating a template for the current frame corresponding to the feature position of the image of the current frame based on the provisional feature position;
Applying a template for the current frame to an image of the current frame to detect a feature position of the image of the current frame; and
A feature position detection apparatus comprising:   The feature position detection apparatus according to claim 6, wherein a template group including images of an object shifted in units of subpixels is used as the template. As the object, a plate in which a figure having the characteristic position is drawn at each of four vertices of a rectangle is used,
The feature position detection method according to claim 6, wherein the graphic does not change in local appearance depending on a scale. The template generation means includes
Means for creating a luminance distribution group from a luminance pattern of an image obtained by imaging the plate having two shaded areas;
Based on the luminance distribution group, an intermediate luminance ideal luminance pattern is obtained from the bright portion luminance and the dark portion luminance, and a step luminance pattern that is an ideal luminance pattern is created;
Means for creating a combined luminance pattern obtained by applying an edge enhancement kernel to the step luminance pattern;
Means for creating a parameter that minimizes the difference between the synthesized luminance pattern and the actual luminance pattern;
Means for edge enhancing the image of the current frame with the parameter and the edge enhancement kernel;
The feature position detection method according to claim 8, further comprising: In the plate, at least three plane figures are drawn in which the areas formed by the first and second boundary lines are alternately shaded,
The intersections of the two shaded area lines are arranged on the same plane and coincide with the positions of any three of the four intersections corresponding to the intersections of two pairs of parallel straight lines having different directions. ,
10. The feature position detection method according to claim 8, wherein each of the shade area lines is parallel to the two pairs of parallel straight lines.

Images