CN107368829B - Method and apparatus for determining a rectangular target area in an input image - Google Patents

Abstract

The invention discloses a method and equipment for determining a rectangular target area in an input image. The method comprises the following steps: acquiring a horizontal line segment and a vertical line segment in an input image; generating candidate line segment combinations, wherein each candidate line segment combination comprises two horizontal line segments and two vertical line segments, and the line segments or the extension lines of the line segments in each candidate line segment combination are intersected to obtain four intersection points; calculating the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the preset reference aspect ratio and the camera internal parameters of the input image; selecting a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and determining an area surrounded by four line segments or extensions thereof in the selected candidate line segment combination as a rectangular target area.

Description

Method and apparatus for determining a rectangular target area in an input image

technical field

The present invention generally relates to the field of image processing. In particular, the present invention relates to a method and apparatus capable of determining a rectangular target area in an input image for perspective correction in a complex background.

Background technique

In modern society, there are various certificates, cards, documents, etc., such as ID cards, business cards, bank cards, household registration books, driver's licenses, passports, and household registration management documents of police stations. Some units or individuals need to collect or archive this information frequently, and need to keep these certificates, cards, documents, etc. electronically. Except for some special reading tools, the usual electronic method is to take pictures, and then store the image or identify the image and store the identified information.

In the process of photographing and retaining, it is often necessary to solve the problem of perspective transformation. This is because: due to the limitations of the environment or equipment, when shooting these documents, cards, documents, etc. (rectangular targets), it may not be directly facing the surface of the subject, but the normal direction of the surface of the subject. There is a certain angle between them, which causes the result of taking pictures to be affected by tilt, which is usually called perspective transformation. For further identification and storage, perspective correction must be performed on the image, and then subsequent processing such as layout analysis and identification can be performed.

The traditional method is to analyze the captured image, find its edges and corners, combine the width and height information of the rectangular target, establish a perspective transformation formula, and perform perspective correction.

However, traditional methods require accurate edge and corner locations. When the background of the rectangular target in the input image is complex, it is difficult to accurately locate the edge and corner positions, and the traditional method is no longer applicable.

The present invention aims to solve the above problems, and can accurately determine a rectangular target area for perspective correction under the condition of complex background in the input image.

SUMMARY OF THE INVENTION

The following presents a brief summary of the present invention in order to provide a basic understanding of certain aspects of the invention. It should be understood that this summary is not an exhaustive overview of the invention. It is not intended to identify key or essential parts of the invention nor to limit the scope of the invention. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is discussed later.

The purpose of the present invention is to propose a method and apparatus for determining a rectangular target area in an input image.

In order to achieve the above object, according to an aspect of the present invention, a method for determining a rectangular target area in an input image is provided, the method comprising: obtaining horizontal line segments and vertical line segments in the input image; generating a candidate line segment combination, Each candidate line segment combination includes two horizontal line segments and two vertical line segments, and the line segments or their extension lines in each candidate line segment combination intersect to obtain four intersection points; according to each candidate line segment combination, the corresponding four intersection points, Predetermining the reference aspect ratio and the in-camera parameters of the input image, calculating the aspect ratio of the perspective-corrected rectangular area corresponding to each candidate line segment combination; based on the comparison between the aspect ratio corresponding to each candidate line segment combination and the predetermined reference aspect ratio , select a candidate line segment combination; and determine the area surrounded by four line segments or their extension lines in the selected candidate line segment combination as a rectangular target area.

According to another aspect of the present invention, there is provided a device for determining a rectangular target area in an input image, the device comprising: a line segment obtaining device configured to: obtain horizontal line segments and vertical line segments in the input image; candidate A line segment combination generating device is configured to: generate candidate line segment combinations, each candidate line segment combination includes two horizontal line segments and two vertical line segments, and the line segments or their extension lines in each candidate line segment combination intersect to obtain four line segments intersection; an aspect ratio calculating device, configured to: calculate the perspective-corrected rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the predetermined reference aspect ratio, and the in-camera parameters of the input image the aspect ratio; the selection device is configured to: select a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with the predetermined reference aspect ratio; and the rectangular area determination device is configured to: The area enclosed by the four line segments or their extension lines in the selected candidate line segment combination is determined as the rectangular target area.

According to yet another aspect of the present invention, an information processing apparatus is provided. The information processing device includes: a controller configured to: obtain horizontal line segments and vertical line segments in an input image; and generate candidate line segment combinations, each candidate line segment combination including two horizontal line segments and two vertical line segments Class line segment, the line segments or their extension lines in each candidate line segment combination intersect to obtain four intersection points; according to the four intersection points corresponding to each candidate line segment combination, the predetermined reference aspect ratio, and the in-camera parameters of the input image, calculate each candidate line segment. The aspect ratio of the perspective-corrected rectangular area corresponding to the line segment combination; selecting a candidate line segment combination based on the comparison between the aspect ratio corresponding to each candidate line segment combination and the predetermined reference aspect ratio; and combining the selected candidate line segment combinations. The area enclosed by the four line segments or their extension lines is determined as the rectangular target area.

In addition, according to another aspect of the present invention, a storage medium is also provided. The storage medium includes machine-readable program code which, when executed on an information processing apparatus, causes the information processing apparatus to perform the above-described method according to the present invention.

Furthermore, according to yet another aspect of the present invention, a program product is also provided. The program product comprises machine-executable instructions which, when executed on an information processing apparatus, cause the information processing apparatus to perform the above-described method according to the invention.

It should be noted that the present invention can solve the technical problem that traditional methods cannot handle images with complex backgrounds, and the present invention can also handle images with simple backgrounds.

Description of drawings

The above and other objects, features and advantages of the present invention will be more easily understood with reference to the following description of the embodiments of the present invention in conjunction with the accompanying drawings. The components in the drawings are merely intended to illustrate the principles of the invention. In the drawings, the same or similar technical features or components will be denoted by the same or similar reference numerals. In the attached picture:

FIG. 1 shows a flowchart of a method of determining a rectangular target area in an input image according to an embodiment of the present invention.

FIG. 2 shows an embodiment of step S1.

FIG. 3 shows an embodiment of step S3.

FIG. 4 shows an example of an input image.

Figure 5 shows all edge segments detected.

Figure 6 shows a rectangular target area found according to the method of the present invention.

FIG. 7 shows the result of perspective correction based on the present invention.

FIG. 8 shows a block diagram of the structure of a device according to an embodiment of the present invention.

FIG. 9 is a block diagram showing the structure of an information processing apparatus according to the present invention.

Figure 10 shows a schematic block diagram of a computer that may be used to implement methods and apparatus according to embodiments of the present invention.

Detailed ways

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the interest of clarity and conciseness, not all features of an actual implementation are described in the specification. It should be appreciated, however, that many implementation-specific decisions must be made in the development of any such actual implementation in order to achieve the developer's specific goals, such as compliance with those constraints related to the system and the business, and these Restrictions may vary from implementation to implementation. Furthermore, it should also be appreciated that while development work can be very complex and time consuming, such development work would be a routine undertaking for those skilled in the art having the benefit of this disclosure.

Here, it should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only device structures and/or processing steps closely related to the solution according to the present invention are shown in the drawings, and are omitted. other details not relevant to the present invention. Additionally, it is noted that elements and features described in one figure or embodiment of the present invention may be combined with elements and features shown in one or more other figures or embodiments.

The flow of a method for determining a rectangular target area in an input image according to an embodiment of the present invention will be described below with reference to FIG. 1 .

FIG. 1 shows a flowchart of a method of determining a rectangular target area in an input image according to an embodiment of the present invention. As shown in FIG. 1 , the method according to an embodiment of the present invention includes the following steps: obtaining horizontal line segments and vertical line segments in an input image (step S1 ); generating candidate line segment combinations, each candidate line segment combination including two The horizontal line segment and the two vertical line segments, the line segments or their extension lines in each candidate line segment combination intersect to obtain four intersection points (step S2); according to the four intersection points corresponding to each candidate line segment combination, the predetermined reference aspect ratio , input the in-camera parameters of the image, and calculate the aspect ratio of the perspective-corrected rectangular area corresponding to each candidate line segment combination (step S3); based on the comparison between the aspect ratio corresponding to each candidate line segment combination and the predetermined reference aspect ratio, One candidate line segment combination is selected (step S4 ); and the area surrounded by four line segments or their extension lines in the selected candidate line segment combination is determined as a rectangular target area (step S5 ).

In step S1, horizontal line segments and vertical line segments in the input image are obtained.

The input image is, for example, an image containing a rectangular object such as a certificate, card, document, or the like.

Figure 4 shows an example of an input image, which includes a rectangular target area (card below) and a complex background distraction (N-time sticker above, which has two rectangle-like structures), before using conventional methods for perspective When correcting, it is easy to identify the background as a rectangular target area and incorrectly perform perspective correction.

First, for example, the input image can be preprocessed, including but not limited to noise removal, resizing, image enhancement, and the like.

Then, as shown in FIG. 2, in step S21, line segments in the input image are acquired.

Specifically, detecting edges in the input image; and detecting line segments from the edges.

Methods for detecting edges in images and methods for detecting straight line segments from edges are well known to those skilled in the art, for example, edges can be detected based on CANNY operator or based on gradients. To detect line segments from edges, for example, a Hough line detection algorithm or the like can be used. The present invention is not limited to this.

In step S22, the acquired line segments are classified into horizontal line segments and vertical line segments.

First, merge all the obtained line segments, and merge the line segments with close distances, the same slope or close (eg, within 5 degrees), and merge them into one line segment. Among them, the distance between line segments whose slopes are inconsistent and not close is defined as the maximum value. The methods of combining are, for example, fitting multiple line segments, selecting line segments in the middle, representing by average slope, clustering, and so on. It should be noted that the merge step is optional.

Then, all the merged line segments are classified into horizontal line segments and vertical line segments according to their slopes.

The horizontal line segment is a substantially horizontal line segment, for example, a line segment whose slope difference from the horizontal direction is within a predetermined range. The vertical line segment is a substantially vertical line segment, for example, a line segment whose slope difference from the vertical direction is within a predetermined range.

Of course, the order of the steps of merging the line segments and the step of classifying the line segments into horizontal line segments and vertical line segments can be interchanged.

Figure 5 shows all detected edge line segments, including the four sides of the card and the seven sides of the N-time stickers.

In step S2, a candidate line segment combination is generated, each candidate line segment combination includes two horizontal line segments and two vertical line segments, and the line segments or their extension lines in each candidate line segment combination intersect to obtain four intersection points. It should be noted that the candidate line segment combinations here are, for example, obtained exhaustively.

Since the target area is rectangular, and the rectangle includes two horizontal sides and two vertical sides that are perpendicular to each other, the candidate line segment combination should include two horizontal-like line segments and two vertical-like line segments.

Four intersection points can be obtained by intersecting the line segments or their extended lines in each candidate line segment combination generated. It should be noted that if at least one of the four obtained intersections is not within the range of the input image, such a candidate line segment combination is discarded.

In step S3, the aspect ratio of the perspective-corrected rectangular area corresponding to each candidate line segment combination is calculated according to the four intersection points corresponding to each candidate line segment combination, the predetermined reference aspect ratio, and the in-camera parameters of the input image.

Specifically, as shown in FIG. 3 , in step S31 , a corresponding homography matrix for perspective correction is obtained according to the four intersection points corresponding to each candidate line segment combination and a predetermined reference aspect ratio.

It is assumed that the final obtained horizontal class line segments are _{sh1, s h2 ,} and so on. The vertical line segments are s _v1 , s _v2 and so on. The candidate line segment combination (s _h1 , s _h2 , s _v1 , s _v2 ) corresponds to four intersection points (s _h1 , s _v1 ), (s _h1 , s _v2 ), (s _h2 , s _v1 ), (s _h2 , s _v2 ), named p ₁ , p ₂ , p ₃ , p ₄ .

The present invention is based on the assumption that the aspect ratio of the target is known, referred to as the predetermined reference aspect ratio R.

Based on a predetermined reference aspect ratio R, a unit target rectangle can be constructed whose four vertex coordinates are p _m1 (0,0), p _m2 (1,0), p _m3 (0, R), p _m4 (1, R ).

For each candidate line segment combination, there are four intersection points p ₁ , p ₂ , p ₃ , p ₄ . Those skilled in the art know that the homography matrix H used for perspective correction is:

where h ₃₃ =1.

Correspondingly, the perspective transformation formula is:

h ₂₁ x _ix +h ₂₂ x _iy +h ₂₃ =(h ₃₁ x _ix +h ₃₂ x _iy +h ₃₃ )m _iy

h ₁₁ x _ix +h ₁₂ x _iy +h ₁₃ =(h ₃₁ x _ix +h ₃₂ x _iy +h ₃₃ )m _ix

Among them, x _ix , x _iy are the abscissa and ordinate of p ₁ , p ₂ , p ₃ , and p ₄ , and _mix and m _iy are p _m1 (0,0), p _m2 (1,0), p _m3 (0,R), the abscissa and ordinate of p _m4 (1,R). Therefore, a total of 8 equations are obtained. Since R is known, there are 8 unknowns in total, namely h ₁₁ , h ₁₂ , h ₁₃ , h ₂₁ , h ₂₂ , h ₂₃ , h ₃₁ , and h ₃₂ . The homography matrix H can be solved. It should be noted that this homography matrix corresponds to the unit target rectangle in the projective space (that is, after perspective transformation) based on a predetermined reference aspect ratio R, and is not the final homography matrix used for perspective correction. And, this homography matrix corresponds to a candidate line segment combination. Therefore, each candidate line segment combination corresponds to a homography matrix H.

In addition, when the aspect ratio r in the projective space (that is, after perspective transformation) is unknown, through the above derivation steps, it can be obtained including h ₁₁ , h ₁₂ , h ₁₃ , h ₂₁ , h ₂₂ , h ₂₃ , h ₃₁ , h ₃₂ have a total of 8 equations with 8 unknowns, and there are 4 intersection points P corresponding to each candidate line segment combination and a unit target rectangle based on an unknown aspect ratio r that can be used to solve the equation system. Thus, there are 9 unknowns in total.

In addition, based on the correspondence between the world coordinates of the four points before the perspective correction and the projective coordinates of the four points after the perspective correction in the projective space (that is, the above-mentioned perspective transformation formula), and the known characteristics of the camera's external parameter matrix, it can be known that the perspective transformation matrix and The constraint relationship between the parameters in the camera:

f为相机的焦距，w、h为相机的分辨率，a、b为相机的传感器尺寸。Among them, the camera internal parameter matrix A is:

f is the focal length of the camera, w and h are the resolution of the camera, and a and b are the sensor size of the camera.

Thus, there are 9 equations and 9 unknowns in total.

can be solved:

Among them, each candidate line segment combination can solve an r. The above formula is an example of a predetermined relationship.

Therefore, in step S32, according to the homography matrix corresponding to each candidate line segment combination and the in-camera parameters of the input image, and based on a predetermined relationship, the aspect ratio of the perspective corrected rectangular region corresponding to each candidate line segment combination is calculated.

In step S4, a candidate line segment combination is selected based on the comparison between the aspect ratio r corresponding to each candidate line segment combination and the predetermined reference aspect ratio R.

Specifically, the candidate line segment combination whose aspect ratio is the smallest difference from the predetermined reference aspect ratio is selected. Since there is only one rectangular target area in the input image, only one candidate line segment combination corresponds to the correct r, that is, the closest to the predetermined reference aspect ratio R.

In step S5, the area surrounded by the four line segments or their extension lines in the selected candidate line segment combination is determined as a rectangular target area.

FIG. 6 shows a rectangular target area found according to the method of the present invention, the four corners of which are marked with dots.

In addition, according to the homography matrix corresponding to the finally selected aspect ratio r, perspective correction may be performed on the rectangular target area determined in step S5 to obtain a corrected image.

FIG. 7 shows the result of perspective correction based on the present invention. The lower card has been corrected to the standard card shape, and no longer has the effect of perspective transformation. At the same time, the upper N times stickers have been corrected to be deformed. This is because the homography matrix on which the perspective correction of the present invention is based is based on the correct one. Targets are generated from cards. In the traditional method, since it is difficult to judge which of the card and the N-time sticker is the target area, it is likely that the N-time sticker is better corrected, and the card is corrected to be deformed.

Hereinafter, an apparatus for determining a rectangular target area in an input image according to an embodiment of the present invention will be described with reference to FIG. 8 .

FIG. 8 shows a block diagram of the structure of a device according to an embodiment of the present invention. As shown in FIG. 8 , the device 800 for determining a rectangular target area in an input image according to the present invention includes: a line segment obtaining device 81, configured to: obtain horizontal line segments and vertical line segments in the input image; The device 82 is configured to: generate a candidate line segment combination, each candidate line segment combination includes two horizontal class line segments and two vertical class line segments, and the line segments or their extension lines in each candidate line segment combination intersect to obtain four intersection points; The aspect ratio calculation device 83 is configured to: calculate the perspective correction rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the predetermined reference aspect ratio, and the in-camera parameters of the input image. aspect ratio; selecting means 84, configured to: select a candidate line segment combination based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and a rectangular area determining means 85, configured to: The area enclosed by the four line segments or their extension lines in the selected candidate line segment combination is determined as a rectangular target area.

In one embodiment, the aspect ratio calculating device 83 includes: a homography matrix obtaining unit, configured to: combine the corresponding four intersection points of each candidate line segment and a predetermined reference aspect ratio to obtain a corresponding one for perspective correction The homography matrix of the The aspect ratio of the back rectangular area.

In one embodiment, the predetermined relationship is constructed based on the correspondence between the world coordinates of the four points before the perspective correction and the projective coordinates of the four points after the perspective correction in the projective space, and the characteristics of the camera's extrinsic parameter matrix.

In one embodiment, the line segment obtaining device 81 includes: an obtaining unit configured to obtain line segments in the input image; and a classification unit configured to classify the obtained line segments into horizontal line segments and vertical line segments.

In one embodiment, the line segment obtaining apparatus 81 further includes: a merging unit, configured to merge at least some of the horizontal line segments and/or the vertical line segments respectively according to a predetermined rule.

In one embodiment, the acquisition unit is further configured to: detect edges in the input image; and detect line segments from the edges.

In one embodiment, the acquisition unit is further configured to: detect edges in the input image based on the gradients; and detect line segments from the edges using the Hough transform.

In one embodiment, the selecting means 84 is further configured to select the candidate line segment combination whose aspect ratio is the smallest in difference from the predetermined reference aspect ratio.

Since the processing in each device and unit included in the apparatus 800 according to the present invention is respectively similar to the process in each step included in the method described above, the details of these devices and units are omitted here for the sake of brevity. describe.

In addition, according to still another aspect of the present invention, as shown in FIG. 9, an information processing apparatus is provided. The information processing apparatus 100 includes a controller 1001 . The controller 1001 is configured to: obtain horizontal line segments and vertical line segments in the input image; generate candidate line segment combinations, each candidate line segment combination includes two horizontal line segments and two vertical line segments, each candidate line segment Four intersection points are obtained by intersecting the line segments or their extension lines in the line segment combination; according to the four intersection points corresponding to each candidate line segment combination, the predetermined reference aspect ratio, and the in-camera parameters of the input image, the perspective correction corresponding to each candidate line segment combination is calculated. The aspect ratio of the rear rectangular area; based on the comparison of the aspect ratio corresponding to each candidate line segment combination with the predetermined reference aspect ratio, select one candidate line segment combination; and select four line segments or their extensions in the selected candidate line segment combination The area enclosed by the line is determined as the rectangular target area.

In addition, it should be pointed out here that each component device and unit in the above-mentioned apparatus may be configured by means of software, firmware, hardware or a combination thereof. The specific means or manners that can be used for the configuration are well known to those skilled in the art, and are not repeated here. When implemented by software or firmware, the program constituting the software is installed from a storage medium or a network to a computer having a dedicated hardware configuration (for example, a general-purpose computer 900 shown in FIG. 10 ). When the computer is installed with various programs, Can perform various functions and the like.

Figure 10 shows a schematic block diagram of a computer that may be used to implement methods and apparatus according to embodiments of the present invention.

In FIG. 10 , a central processing unit (CPU) 901 executes various processes according to a program stored in a read only memory (ROM) 902 or a program loaded from a storage section 908 to a random access memory (RAM) 903 . In the RAM 903, data required when the CPU 901 executes various processes and the like is also stored as necessary. The CPU 901 , the ROM 902 and the RAM 903 are connected to each other via a bus 904 . An input/output interface 905 is also connected to the bus 904 .

The following components are connected to the input/output interface 905: an input section 906 (including a keyboard, mouse, etc.), an output section 907 (including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.), A storage part 908 (including a hard disk, etc.), a communication part 909 (including a network interface card such as a LAN card, a modem, etc.). The communication section 909 performs communication processing via a network such as the Internet. A driver 910 may also be connected to the input/output interface 905 as desired. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. can be mounted on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

In the case where the above-described series of processing is realized by software, a program constituting the software is installed from a network such as the Internet or a storage medium such as the removable medium 911 .

It should be understood by those skilled in the art that such a storage medium is not limited to the removable medium 911 shown in FIG. 10 in which the program is stored and distributed separately from the device to provide the user with the program. Examples of the removable medium 911 include magnetic disks (including floppy disks (registered trademark)), optical disks (including compact disk read only memory (CD-ROM) and digital versatile disks (DVD)), magneto-optical disks (including minidiscs (MD) (registered trademark) )) and semiconductor memory. Alternatively, the storage medium may be the ROM 902, a hard disk included in the storage section 908, or the like, in which programs are stored and distributed to users together with the apparatuses including them.

The present invention also provides a program product storing machine-readable instruction codes. When the instruction code is read and executed by a machine, the above-described method according to the embodiment of the present invention can be executed.

Correspondingly, a storage medium for carrying the above-mentioned program product storing the machine-readable instruction code is also included in the disclosure of the present invention. The storage medium includes, but is not limited to, a floppy disk, an optical disk, a magneto-optical disk, a memory card, a memory stick, and the like.

In the above description of specific embodiments of the invention, features described and/or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments as the features of the other embodiments. In combination with, or in place of, features in other embodiments.

It should be emphasized that the term "comprising/comprising" when used herein refers to the presence of a feature, element, step or component, but does not exclude the presence or addition of one or more other features, elements, steps or components.

In addition, the method of the present invention is not limited to be executed according to the chronological sequence described in the specification, and can also be executed according to other chronological sequence, in parallel or independently. Therefore, the execution order of the methods described in this specification does not limit the technical scope of the present invention.

While the present invention has been disclosed above by way of descriptions of specific embodiments thereof, it should be understood that all of the above-described embodiments and examples are illustrative and not restrictive. Various modifications, improvements or equivalents of the present invention may be devised by those skilled in the art within the spirit and scope of the appended claims. These modifications, improvements or equivalents should also be considered to be included within the protection scope of the present invention.

Postscript

1. A method of determining a rectangular target area in an input image, comprising:

Obtain the horizontal and vertical line segments in the input image;

Generate candidate line segment combinations, each candidate line segment combination includes two horizontal class line segments and two vertical class line segments, and the line segments or their extension lines in each candidate line segment combination intersect to obtain four intersection points;

Calculate the aspect ratio of the perspective-corrected rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the predetermined reference aspect ratio, and the in-camera parameters of the input image;

selecting a candidate line segment combination based on a comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and

The area enclosed by the four line segments or their extension lines in the selected candidate line segment combination is determined as a rectangular target area.

2. The method as described in appendix 1, wherein the perspective-corrected rectangular area corresponding to each candidate line segment combination is calculated according to the four intersection points corresponding to each candidate line segment combination, the predetermined reference aspect ratio, and the in-camera parameters of the input image. The aspect ratios include:

According to the four intersection points corresponding to each candidate line segment combination and a predetermined reference aspect ratio, a corresponding homography matrix for perspective correction is obtained;

According to the homography matrix corresponding to each candidate line segment combination and the in-camera parameters of the input image, and based on a predetermined relationship, the aspect ratio of the perspective-corrected rectangular region corresponding to each candidate line segment combination is calculated.

3. The method as described in appendix 2, wherein the predetermined relationship is based on the correspondence between the world coordinates of the four points before the perspective correction and the projective coordinates of the four points after the perspective correction in the projective space, and the characteristics of the camera's external parameter matrix. Construct.

4. The method according to appendix 1, wherein obtaining the horizontal line segment and the vertical line segment in the input image comprises:

get the line segments in the input image; and

The acquired line segments are classified into horizontal line segments and vertical line segments.

5. The method described in Supplementary Note 4, obtaining the horizontal line segment and the vertical line segment in the input image further comprises:

According to a predetermined rule, at least some of the horizontal line segments and/or the vertical line segments are merged respectively.

6. The method described in Supplementary Note 4, obtaining the line segment in the input image comprises:

detect edges in the input image; and

Detect line segments from edges.

7. The method of appendix 6, wherein,

based on gradients, detect edges in the input image; and

Line segments are detected from edges using the Hough transform.

8. The method of appendix 1, wherein, based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio, selecting a candidate line segment combination comprises:

The candidate line segment combination whose aspect ratio is the smallest difference from the predetermined reference aspect ratio is selected.

9. A device for determining a rectangular target area in an input image, comprising:

a line segment obtaining device, configured to: obtain horizontal line segments and vertical line segments in the input image;

The device for generating a candidate line segment combination is configured to: generate a candidate line segment combination, each candidate line segment combination includes two horizontal class line segments and two vertical class line segments, and the line segments or their extension lines in each candidate line segment combination intersect to obtain four line segments. a point of intersection;

The aspect ratio calculation device is configured to: calculate the width of the perspective-corrected rectangular region corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the predetermined reference aspect ratio, and the in-camera parameters of the input image high ratio;

selecting means configured to: select a candidate line segment combination based on a comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and

The rectangular area determination device is configured to determine the area surrounded by four line segments or their extension lines in the selected candidate line segment combination as a rectangular target area.

10. The device according to appendix 9, wherein the aspect ratio calculating means comprises:

a homography matrix obtaining unit, configured to: obtain a corresponding homography matrix for perspective correction according to the four intersection points corresponding to each candidate line segment combination and a predetermined reference aspect ratio;

The aspect ratio calculation unit is configured to: according to the homography matrix corresponding to each candidate line segment combination and the in-camera parameters of the input image, and based on a predetermined relationship, calculate the width and height of the perspective-corrected rectangular region corresponding to each candidate line segment combination Compare.

11. The device as described in appendix 10, wherein the predetermined relationship is based on the correspondence between the world coordinates of the four points before the perspective correction and the projective coordinates of the four points after the perspective correction in the projective space, and the characteristics of the camera's external parameter matrix. Construct.

12. The apparatus according to appendix 9, wherein the line segment obtaining means comprises:

an acquisition unit configured to: acquire line segments in the input image; and

The classification unit is configured to: classify the acquired line segments into horizontal line segments and vertical line segments.

13. The apparatus according to appendix 12, wherein the line segment obtaining device further comprises:

The merging unit is configured to merge at least some of the horizontal line segments and/or the vertical line segments respectively according to a predetermined rule.

14. The device according to supplementary note 12, wherein the acquiring unit is further configured to:

detect edges in the input image; and

Detect line segments from edges.

15. The device according to supplementary note 14, wherein the acquiring unit is further configured to:

based on gradients, detect edges in the input image; and

Line segments are detected from edges using the Hough transform.

16. The apparatus of appendix 9, wherein the selection means is further configured to:

The candidate line segment combination whose aspect ratio is the smallest difference from the predetermined reference aspect ratio is selected.

17. An information processing device, comprising:

Controller, configured as:

Obtain the horizontal and vertical line segments in the input image;

Generate candidate line segment combinations, each candidate line segment combination includes two horizontal class line segments and two vertical class line segments, and the line segments or their extension lines in each candidate line segment combination intersect to obtain four intersection points;

Calculate the aspect ratio of the perspective-corrected rectangular area corresponding to each candidate line segment combination according to the four intersection points corresponding to each candidate line segment combination, the predetermined reference aspect ratio, and the in-camera parameters of the input image;

selecting a candidate line segment combination based on a comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and

The area enclosed by the four line segments or their extension lines in the selected candidate line segment combination is determined as a rectangular target area.

Claims (8)

1. A method of determining a rectangular target area in an input image, comprising: Obtaining a horizontal-like line segment and a vertical-like line segment in the input image, wherein the horizontal-like line segment is a line segment with a slope difference from the horizontal direction within a first predetermined range, and the vertical-like line segment is a line segment related to the vertical direction The difference between the slopes of the line segment within the second predetermined range; Generate candidate line segment combinations, each candidate line segment combination includes two horizontal class line segments and two vertical class line segments, and the line segments or their extension lines in each candidate line segment combination intersect to obtain four intersection points; According to the four intersection points corresponding to each candidate line segment combination and a predetermined reference aspect ratio, a corresponding homography matrix for perspective correction is obtained; According to the homography matrix corresponding to each candidate line segment combination and the in-camera parameters of the input image, based on a predetermined relationship, the aspect ratio of the perspective-corrected rectangular region corresponding to each candidate line segment combination is calculated, wherein the predetermined relationship is based on the perspective It is constructed by the correspondence between the world coordinates of the first four points and the projective coordinates of the four points in the projective space after perspective correction, and the characteristics of the camera's external parameter matrix; selecting a candidate line segment combination based on a comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and The area enclosed by the four line segments or their extension lines in the selected candidate line segment combination is determined as a rectangular target area. 2. The method of claim 1, wherein obtaining horizontal line segments and vertical line segments in the input image comprises: get the line segments in the input image; and The acquired line segments are classified into horizontal line segments and vertical line segments. 3. The method of claim 2 , obtaining horizontal line segments and vertical line segments in the input image further comprising: According to a predetermined rule, at least some of the horizontal line segments and/or the vertical line segments are merged respectively. 4. The method of claim 2, obtaining line segments in the input image comprising: detect edges in the input image; and Detect line segments from edges. 5. The method of claim 4, wherein, based on gradients, detect edges in the input image; and Line segments are detected from edges using the Hough transform. 6. The method of claim 1, wherein, based on the comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio, selecting a candidate line segment combination comprises: The candidate line segment combination whose aspect ratio is the smallest difference from the predetermined reference aspect ratio is selected. 7. A device for determining a rectangular target area in an input image, comprising: A line segment obtaining device is configured to obtain a horizontal line segment and a vertical line segment in the input image, wherein the horizontal line segment is a line segment with a slope difference from the horizontal direction within a first predetermined range, and the vertical line segment The straight line segment is a line segment whose difference from the slope of the vertical direction is within a second predetermined range; The device for generating a candidate line segment combination is configured to: generate a candidate line segment combination, each candidate line segment combination includes two horizontal class line segments and two vertical class line segments, and the line segments or their extension lines in each candidate line segment combination intersect to obtain four line segments. a point of intersection; The aspect ratio calculation device is configured to: obtain a corresponding homography matrix for perspective correction according to the four intersection points corresponding to each candidate line segment combination and a predetermined reference aspect ratio, and according to each candidate line segment combination corresponding The homography matrix and the in-camera parameters of the input image, based on a predetermined relationship, calculate the aspect ratio of the perspective corrected rectangular area corresponding to each candidate line segment combination, wherein the predetermined relationship is based on the world coordinates of the first four points before perspective correction It is constructed by the corresponding relationship with the projective coordinates of the four points in the projective space after perspective correction and the characteristics of the camera's external parameter matrix; selecting means configured to: select a candidate line segment combination based on a comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and The rectangular area determination device is configured to determine the area surrounded by four line segments or their extension lines in the selected candidate line segment combination as a rectangular target area. 8. An information processing device, comprising: Controller, configured as: Obtaining a horizontal-like line segment and a vertical-like line segment in the input image, wherein the horizontal-like line segment is a line segment with a slope difference from the horizontal direction within a first predetermined range, and the vertical-like line segment is a line segment related to the vertical direction The difference between the slopes of the line segment within the second predetermined range; Generate candidate line segment combinations, each candidate line segment combination includes two horizontal class line segments and two vertical class line segments, and the line segments or their extension lines in each candidate line segment combination intersect to obtain four intersection points; According to the four intersection points corresponding to each candidate line segment combination and a predetermined reference aspect ratio, a corresponding homography matrix for perspective correction is obtained; According to the homography matrix corresponding to each candidate line segment combination and the in-camera parameters of the input image, based on a predetermined relationship, the aspect ratio of the perspective-corrected rectangular region corresponding to each candidate line segment combination is calculated, wherein the predetermined relationship is based on the perspective It is constructed by the correspondence between the world coordinates of the first four points and the projective coordinates of the four points in the projective space after perspective correction, and the characteristics of the camera's external parameter matrix; selecting a candidate line segment combination based on a comparison of the aspect ratio corresponding to each candidate line segment combination with a predetermined reference aspect ratio; and The area enclosed by the four line segments or their extension lines in the selected candidate line segment combination is determined as a rectangular target area.

Images