JP2010152800A - Image processing apparatus, image processing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To accurately recognize only a search window area in an image by analyzing the image displayed on a screen. <P>SOLUTION: The image processing apparatus 10 for detecting a character input area contained in an image includes: a polygon extraction part 11 which extracts images similar to a shape of character input area from the image displayed on the screen; a shape discrimination part 12 which extracts images having the shape of character input area from the images similar to the shape of character input area extracted by the polygon extraction part 11; and a character arrangement discrimination part 14 which extracts, from the images having the shape of character input area extracted by the shape discrimination part 12, an image having the shape of character input area in which the arrangement of characters existing in the image of the character input area is similar to the arrangement of characters to be input to a character input area in keyword search. The image having the shape of character input area extracted by the character arrangement discrimination part 14 is output. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a technique for detecting a video of a character input area included in a video displayed on a screen.

  Conventionally, a character recognition technique for recognizing characters from a given image is known. For example, in the character recognition device disclosed in Patent Document 1, a document is read by a scanner, the image is processed by an image processing unit, and the character string converting unit converts the character string. The feature extraction unit extracts character features from them. The matching unit compares the feature with the dictionary, and outputs the category that is most similar. A character type determination unit is disposed in front of the feature extraction unit, and determines a type classification according to the size of each input character. The dictionary is provided with a classification, and the character classification determination unit classifies the characters in the character string with the maximum height H as a reference and classifies the characters with a size of H / th or less as “lowercase”. . The matching unit performs matching processing only with the dictionary corresponding to the classification obtained in the classification determination process.

  On the other hand, in recent years, on TV commercials and the like, a method of prompting a search by the Internet by displaying a search window of an Internet search engine in an image and presenting a screen on which a keyword there is input is becoming common. In the CM using this search window, the search window works as a kind of character input area, and prompts the user to input characters into the search window and perform a search. The keyword presented in the search window Has been adjusted in advance so that the advertiser's site is at the top of the search results, and it is very meaningful to provide a mechanism that allows easy search using this keyword.

  Therefore, if there is a technique for accurately detecting only a search window area including a desired character from the screen, it is possible to acquire characters included in the search window area in combination with the technique. However, conventionally, there has not been a technique for identifying only the search window area from the television image and accurately detecting the area. On the other hand, there is a telop as an expression method for presenting text in a television image as in the search window area. As a technique for recognizing a telop area in a television program, for example, the following is known.

In Patent Document 2, telops are displayed using the characteristics “(1) displayed in a predetermined area at the top or bottom of the screen” and “(2) luminance change occurs when telops appear and ends.” Detect telop. Further, in Patent Document 3, “(3) The telop is detected with a combination of the detected edge and the luminance binarized image using the feature that the telop has higher luminance than the background image” is performed. ing. Further, in Patent Document 4, “(4) the telop area maintains a stable luminance state for a certain period of time compared to the natural image”, “(5) the telop area shows a uniform luminance level compared to the natural image. The telop is detected using the feature "."
Japanese Patent Laid-Open No. 5-28318 JP-A-12-23062 Japanese Patent Laid-Open No. 12-182053 JP-A-10-233994

  However, since the search window area as the character input area is not displayed in a predetermined area at the top or bottom of the screen, the feature (1) cannot be used. Even if the search window area is displayed, the luminance change does not always occur, and thus the feature (2) cannot be used. In addition, since the search window area is not always brighter than the background image, the feature (3) cannot be used. Further, the feature (4) cannot be used because there are many examples in which the entire screen becomes a still image in a CM having a search window area. The feature (5) has the same tendency in the search window area. However, if the search window area is detected only with this feature, sufficient accuracy cannot be obtained and it is included in the TV image. An object similar to the search window area is erroneously detected as the search window area.

  The present invention has been made in view of such circumstances, and an image processing apparatus, an image processing method, and a program capable of accurately recognizing only a search window area in a television image by analyzing the television image. The purpose is to provide.

  As another means for solving the problem of the present invention, it can be easily considered that the search keyword is included as subtitles. However, as of November 2008, since the subtitle information is not included in the CM broadcast, the above search is performed. The keyword exists only as a video, and there is no means for providing it to the user as text.

  (1) In order to achieve the above object, the present invention takes the following measures. That is, the image processing apparatus of the present invention is an image processing apparatus that detects a video of a character input area included in a video displayed on the screen, and approximates the shape of the character input area from the video displayed on the screen. An extraction unit that extracts an image, a shape determination unit that extracts an image having a shape of a character input region out of images that approximate the shape of the character input region extracted by the extraction unit, and the shape determination unit In the video having the shape of the character input area, the character input area of the character input area is an array that approximates the character input area to be input to the character input area when searching for keywords. And a character arrangement determining unit that extracts a video having a shape.

  With this configuration, it is possible to detect only the character input area in the video with high accuracy. The character arrangement determining unit performs a process of extracting the arrangement of character information in the character input area. This character information arrangement determination process is not a character recognition process (that is, a process of comparing characters with individual dictionaries), but extracts the presence of feature quantities generally found in characters from the image, and determines the arrangement. Do only lightweight processing to investigate. The character information can be extracted with high accuracy by performing position determination processing to exclude candidates that are not character input regions, and unnecessary character recognition processing can be avoided.

  (2) Further, in the image processing apparatus of the present invention, the extraction unit extracts a video approximating the shape of a character input area from the video displayed on the screen by performing polygon approximation. Yes.

  In this way, extraction processing based on the shape expected as the character input area is performed before processing such as color histogram processing and character arrangement determination in the character input area, and the number of character input area candidates is greatly reduced. Thus, it is possible to reduce the processing load and speed up the operation.

  (3) Further, in the image processing apparatus of the present invention, the extraction unit may include a plurality of polygons having four or more vertices among the polygon approximation results, and the longest two straight lines are horizontal with respect to the screen. It is characterized in that a square is extracted as a candidate for a character input area.

  As described above, among the polygon approximation results, the polygon having the longest two straight lines horizontal to the screen is extracted from the polygons having four or more vertices as the character input area candidates. It is possible to reduce and speed up the operation.

  (4) Moreover, in the image processing apparatus of the present invention, the extraction unit has the longest two straight lines out of the polygons having four or more vertices among the polygon approximation results, and is horizontal to the screen. In addition, it is characterized in that polygons having the same length of the two straight lines are extracted as candidates for the character input area.

  In this way, among the polygon approximation results, the longest two straight lines out of the polygons having four or more vertices are horizontal with respect to the screen and the lengths of the two straight lines are equal to each other. Since the input area is extracted as a candidate, it is possible to reduce the processing load and speed up the operation.

  (5) Moreover, in the image processing apparatus of the present invention, the extraction unit obtains the height of the polygon from the distance between two straight lines horizontal to the screen among the results of the polygon approximation, A region whose height is a length equal to or less than a predetermined ratio of the horizontal length of the polygon is extracted as a character input region candidate.

  As described above, the polygon height is obtained from the distance between two straight lines parallel to the screen among the polygon approximation results, and the height is equal to or less than a predetermined ratio of the horizontal length of the polygon. Is extracted as a candidate for a character input area, it is possible to reduce the processing load and speed up the operation. Here, “the height of the polygon is equal to or less than a predetermined ratio of the horizontal length of the polygon” means, for example, that the height of the polygon is 1/3 or less of the horizontal length of the polygon. It can be determined as follows. That is, it is possible to extract only a shape that approximates a generally well-known character input area for search.

  (6) In the image processing apparatus according to the present invention, the character arrangement determining unit calculates a color histogram for the region obtained by the polygon approximation, and determines a background color and a character color from the frequency. Then, it is characterized by determining whether or not it is a character input area from the appearance frequency of the character color.

  In this way, a color histogram is calculated for the area obtained by polygon approximation, the background color and the character color are determined based on the frequency, and whether the character input area is determined based on the appearance frequency of the character color. Therefore, the character input area can be extracted with high accuracy.

  (7) Further, in the image processing apparatus of the present invention, the character arrangement determination unit performs corner detection on the region obtained by the polygon approximation, and determines the character in the region from the frequency of the detected corner. It is characterized by determining the arrangement and determining whether or not it is a character input area from the arrangement of characters.

  In this way, corner detection is performed on the area obtained by polygon approximation, the arrangement of characters in the area is determined from the frequency of the detected corner, and whether or not the character input area is determined from the character arrangement. Therefore, it is possible to extract the character input area with high accuracy.

  (8) Further, the image processing apparatus according to the present invention, when a plurality of character input areas are extracted from the image having the shape of the character input area extracted by the shape determination unit, A position determining unit that removes an area other than the character input area from the length ratio and the positional relationship in the screen is further provided.

  As described above, when a plurality of character input areas are extracted from the image having the shape of the character input area extracted by the shape discriminating unit, the ratio of the horizontal length between them and the position in the screen Since the area other than the character input area is removed from the relationship, the character input area can be extracted with high accuracy.

  (9) In the image processing apparatus of the present invention, the number of characters changes when the number of characters in the image of the character input area changes with time among the images having the shape of the character input area extracted by the shape determining unit. A character input completion determining unit that determines whether or not the character input is completed, and the character arrangement determining unit is an array that approximates an array of characters to be input to the character input area when searching for a keyword when the number of characters has been changed An image having the shape of the character input area is extracted.

  With this configuration, it is possible to extract an image having the shape of the character input area at the timing when the character input is completed, even if the image is a character input to the character input area gradually.

  (10) In the image processing apparatus of the present invention, the video having the shape of the character input area extracted by the character arrangement determination unit is output to the character recognition unit that performs character recognition, and is recognized by the character recognition unit. Characteristic search is performed using characters.

  With this configuration, for example, a keyword search can be performed by pressing a remote control button in a character input area while a user is watching a TV commercial.

  (11) Further, the image processing method of the present invention is an image processing method for detecting a video of a character input area included in a video displayed on a screen, wherein the extraction unit performs polygon approximation, Extracting a video approximating the shape of the character input area from the video displayed on the screen; and, in the shape discriminating unit, out of the video approximating the shape of the character input area extracted by the extracting unit. In the step of extracting a video having a shape, and the character input completion determining unit, when the number of characters in the video of the character input region among the images having the shape of the character input region extracted by the shape determining unit changes with time In the step of determining whether or not the change in the number of characters is completed, and in the character arrangement determining unit, when the change in the number of characters is completed, the shape determining unit extracts In the video having the shape of the character input area, the character input area of the character input area is an array that approximates the character input area to be input to the character input area when searching for keywords. The method includes at least a step of extracting a video having a shape and a step of outputting a video having a shape of the character input area extracted by the character arrangement determination unit.

  With this configuration, it is possible to detect only the character input area in the video with high accuracy. The character arrangement determining unit performs a process of extracting the arrangement of character information in the character input area. This character information arrangement determination process is not a character recognition process (that is, a process of comparing characters with individual dictionaries), but extracts the presence of feature quantities generally found in characters from the image, and determines the arrangement. Do only lightweight processing to investigate. The character information can be extracted with high accuracy by performing position determination processing to exclude candidates that are not character input regions, and unnecessary character recognition processing can be avoided.

  (12) The program of the present invention is a program for detecting a video of a character input area included in a video displayed on the screen, and by performing polygon approximation, characters from the video displayed on the screen are detected. A process of extracting a video approximating the shape of the input area; a process of extracting a video having a character input area shape out of the video approximating the shape of the extracted character input area; and the extracted character input When the number of characters in the image of the character input region changes with time among the images having the shape of the region, the process of determining whether the change in the number of characters has ended, and the extraction when the change in the number of characters ends Among the images having the character input area shape, the character array existing in the character input area image is changed to the character array to be input to the character input area at the time of keyword search. A series of processes of extracting a video having a shape of a character input area that is a similar arrangement and outputting a video having a shape of the extracted character input area can be read and executed by a computer. It is characterized by being commanded.

  With this configuration, it is possible to detect only the character input area in the video with high accuracy. The character arrangement determining unit performs a process of extracting the arrangement of character information in the character input area. This character information arrangement determination process is not a character recognition process (that is, a process of comparing characters with individual dictionaries), but extracts the presence of feature quantities generally found in characters from the image, and determines the arrangement. Do only lightweight processing to investigate. The character information can be extracted with high accuracy by performing position determination processing to exclude candidates that are not character input regions, and unnecessary character recognition processing can be avoided.

  According to the present invention, it is possible to detect only a character input area in a video with high accuracy. As a result, keyword search can be easily performed without using a keyboard.

  FIG. 1 is a block diagram showing a schematic configuration of an image processing apparatus according to an embodiment of the present invention. FIG. 2 is a main flowchart showing the operation of the image processing apparatus according to the embodiment of the present invention. FIG. 3 shows the operation of extracting a character input area (hereinafter referred to as “search window area”). FIG. 4 is a flowchart showing a character input completion determining operation. In FIG. 1, the image processing apparatus 10 includes a polygon extraction unit 11, a shape determination unit 12, a color histogram calculation unit 13, a character arrangement determination unit 14, a position determination unit 16, and a character input completion determination unit 17. Further, the image processing apparatus 10 inputs a video signal from the video signal input unit 19 and outputs a processing result to the character recognition unit 18.

  In the embodiment of the present invention, it is assumed that a television receiver having an Internet connection function incorporates the image processing apparatus 10 of the present invention, and a user can use the functions of the present invention by operating a remote control of the television.

  In FIG. 2, first, the image processing apparatus 10 acquires frame information at regular intervals from a television image that the user is viewing. The search window area is displayed for several seconds because it is necessary for the viewer to remember the search keyword displayed in the area. In general, since a television image is composed of 29.97 frames per second, the image processing apparatus 10 extracts one frame at a cycle of once every several tens of frames (step S1), thereby including a search window area. It is possible to extract all the frames without missing. Next, the search window area is extracted (step S2). This search window region extraction operation is performed according to the flowchart shown in FIG.

  In FIG. 3, first, edge extraction / polygon approximation is performed (step S10). Since the search window area is configured in such a way that the keywords are surrounded by a frame line, the contour can be approximated by a closed polygon. Therefore, the search window area is approximated by a polygon having four or more vertices. For the image frame obtained from the video signal input, the polygon extraction unit 11 first extracts the contour of the input image. Arbitrary methods can be used as a method for extracting the contour. For example, the Canny filter (J. Canny. A Computational Approach to Edge Detection, IEEE Trans. On Pattern Analysis and Machine Intelligence, 8 (6), pp. 679-698 (1986).) To extract edges.

FIG. 5A is a diagram illustrating an example of a current image, and FIG. 5B is a diagram illustrating an example of an edge image extracted therefrom. The edge of the search window area forms a closed frame line. In order to detect this frame line, straight line detection is performed on the extracted edge image. Although any method can be applied as the method of detecting a straight line, for example, a method for detecting a straight line using Radon transform is disclosed in Japanese Patent Laid-Open No. 2005-275447. Next, it is determined whether or not each of the detected straight lines forms a closed frame line. An arbitrary method can be used as the determination method. For example, the following method can be used.
(1) One of the detected straight lines is selected, and it is checked whether there is a straight line in an area adjacent to the straight line.
(2) Move to the straight line found in the process of (1), and examine the adjacent area as in (1).
(3) Repeating the above process and returning to the initially selected straight line, it is determined that it is a closed frame line.
(4) The processes 1 to 3 are repeated until all the detected edges are examined.

  In order to smooth the detected straight line, approximation with a polygon is performed. An arbitrary algorithm can be used as the approximation algorithm. For example, a Douglas-Peucker algorithm or the like can be used. FIG. 5C is a diagram illustrating an example of a result of polygon approximation.

Next, shape discrimination is performed (step S11). It is determined whether the polygon is the shape of the search window (region) (step S12). The shape discriminating unit 12 removes those that are not search window regions from the polygonal shape obtained in step S10. The shape of the search window area is generally a rectangle as shown in FIG. 6A, but shapes as shown in FIGS. 6B and 6C can also be seen. However, the search window (area) is placed parallel to the screen due to the nature of entering characters in it. Also, the search window (area) is horizontally long and the length of the upper and lower line segments because of the necessity of making an Internet search image. Are almost equal polygons. That is, the characters described in trapezoids and rhombuses do not evoke the Internet search sufficiently. Therefore, it is a quadrangle having at least four vertices, and in some cases, it is a polygon more than that. Therefore, the shape discrimination unit 12 performs the following discrimination process based on the polygon approximation result.
(1) Exclude areas with fewer than 4 vertices. This is because the search window region has at least two parallel straight lines, so that the search window region is a polygon that is equal to or more than a quadrangle, and is often a quadrangle.
(2) Select the longest two straight lines included in the polygon system and check that they are sufficiently horizontal to the screen. As a check method, the start and end points of the longest two straight lines are
[About line 1]
Start point (x1, y1), end point (x2, y2)
[About line 2]
Start point (x3, y3), end point (x4, y4)
And when
| Y1-y2 | <P1 and | y3-y4 | <P1
Be. Here, P1 is a threshold value.

It is also effective for improving accuracy to check that the two longest straight lines are sufficiently parallel. As a method of checking that two straight lines are parallel, in addition to the above horizontal condition,
|| y1-y2 |-| y3-y4 || <P2
Find out. Here, P2 is a threshold value smaller than P1.
(3) Subsequently, it is checked that the longest two straight lines are not far apart. As a check method, in the above conditions,
|| x2-x1 |-| x3-x4 || <P3. P3 is a threshold value.
(4) Obtain the height of the search window area from the distance between the two straight lines, and check that it is sufficiently small with respect to the horizontal length of the search window area. Since (2) has already confirmed that the two straight lines are almost horizontal and parallel to the screen, the height of the search window region can be obtained by comparing the average values of the y coordinates. That is, it can be obtained by the following formula.

Here, P4 is a coefficient.

  This shape determination process needs to be calculated only for the coordinate information of the straight line obtained as the frame of the search window area, and is lighter than the process described later. Therefore, by narrowing down the candidates by this process, the overall calculation amount can be reduced. FIG. 9 is a diagram conceptually showing the concept of the shape discrimination process.

  Next, character arrangement determination is performed (step S14). Search keywords are described in the search window area. Since the search window area imitates an Internet search engine, the horizontal position of the search keyword is centered or left-justified, and the vertical position is from the center or bottom. The character arrangement discriminating unit 14 discriminates the character position, thereby removing a scoreboard, a program telop, a table column displayed on the screen, and the like. The character position is determined by either one of the following two processes or both.

(1) Method of Viewing Color Histogram Distribution The color histogram calculation unit 13 calculates a color histogram in the detected polygon. It can be determined that the portion with the highest frequency in the histogram is the background color, and the second largest portion is the character. It is determined from the arrangement of the portion having the second most frequent color value whether or not it is a search window region in which characters are contained.
As the horizontal direction discrimination, when the horizontal coordinate is defined as (x1, y1) as the upper left vertex coordinate and (xn, yn) as the lower right vertex coordinate in the detected search window area candidate, The appearance frequency of the pixel having the second highest frequency is calculated for all the pixels having y1. Similar processing is performed for the pixels y + 1 and y + 2, and the frequency is calculated until the final appearance yn. In the case of a search window region in which characters are entered, the pixel appearance frequency exceeds the threshold value in the y coordinate position where the coordinate is close to yn (that is, bottom), and in the x coordinate direction where the coordinate is close to x1 or Centered (ie left justified or center justified).
(Appearance frequency average near y1) ≧ (Appearance frequency average near yn)
(Appearance frequency average near x1) ≦ (Appearance frequency average near xn)
If it is, it is determined that it is not the search window area (step S13).
(2) Method of viewing the distribution of character corners by corner detection Characters are composed of straight lines, curves, and points in any language. Therefore, the character area includes various corners. Therefore, it is possible to detect the arrangement of characters by detecting this corner. An existing method such as the Harris method can be used as the corner detection method, but the part of determining the character of the search window area of the character arrangement from the detected corner is one of the features of the present invention.

  FIG. 7A is a diagram showing an example of a search window area detected by the processing so far, and FIG. 7B shows a result of corner detection performed on the area. A round part is a detected corner.

  Here, the character arrangement is determined from the appearance frequency of the corners in each of the horizontal direction and the vertical direction. Now, the search window area is divided into appropriate numbers in the vertical and horizontal directions (see FIG. 8), and the number of edges in the divided area is calculated and detected for each area. An area where the number of edges> P is determined as an area including characters.

  Here, among the divided areas, a column composed of only areas where no characters appear is counted from the right, and is defined as “right blank number Empty r”. In FIG. 8, the number of white spaces in the right direction is 2. The number of blanks is similarly calculated for the left direction (Empty l), upper (Empty c), and lower (Empty f).

Here, characters are left justified or center justified in the horizontal direction, and center justified or bottom justified in the vertical direction,
Empty l => T1 * Empty r
Empty f => T2 * Empty c
In this case, it is determined that it is not the search window area. Here, T1 and T2 are constants.

  As described above, if the histogram arrangement in the polygon is appropriate as the search window region, it is determined as a search window candidate (step S15).

Next, position discrimination processing is performed (step S16). In television commercials using a search window area, there are many examples in which an object that imitates a search button is placed near the search window area. Since this object has a tendency to be very close to the inside of the search window area, erroneous detection is likely to occur, but the search button has the following tendency.
“The horizontal length is smaller than the search window area and it is placed to the right of the search window area.”

If one of the extracted search window area candidates satisfies the above, the position determination unit 16 determines that it is a search button and performs a removal process. That is, when a plurality of search window candidate areas are found in one image, the upper right coordinate of the area forming the candidate n is (xn, yn), the horizontal length of the area is wn, and the candidate m is When the upper right coordinate of the area to be formed is (xm, ym) and the horizontal length of the area is wm,
| Yn−ym | <P5 and xn <= xm−wm and wn> wm × P6
When the condition is satisfied, the candidate m is determined as a search button, and is removed from the search window area candidates. If it is a search button in step S16, it is not a search window candidate (step S17). If it is not a search button, the process proceeds to the next process.

  Next, display time determination is performed (step S18). In the search window area advertisement, since the search window area is displayed at least for a time that can be visually confirmed by the user, it is determined whether the search window area is displayed for a predetermined time n. Here, as a feature of the search window area, when characters are dynamically entered in the window, the inside of the search window area changes, but the frame line itself forming the search window area does not change. A feature is used to determine whether this frame line is fixed.

  As the determination method, the above-described edge extraction processing and shape determination processing are performed. Here, as described above, the search window area includes not only a rectangular shape but also a shape as shown in FIG. 6B. Therefore, for simplification, for the display time determination, two horizontal segments are used. If the longest line segment of the search window area candidate in the current frame is in the same position after n frames, it is determined that the search window area candidate exists after n frames. With the processing so far, the search window area candidates are narrowed down to one (step S19).

In FIG. 2, when the extraction of the search window area is completed (step S2), the search window area is presented to the user (step S3). Next, character input completion determination is performed (step S4). This character input completion determination process is performed according to the flowchart shown in FIG. Here, there are the following two display methods for displaying characters in the search window area.
(1) All characters are input from the beginning.
(2) Characters are input gradually.

  For this reason, in the case (2), it is desirable not to directly acquire the character string in the recognized search window area but to acquire it at the timing when the character input is completed. As a method for determining the completion of input of the search window area, since the character input has been completed in the last state of the search window area, a technique of waiting for a scene change and acquiring the search window area can be easily considered. . However, this method has a problem that it takes time until the window area is recognized in the case of (1).

  Therefore, the character input completion determination corresponding to both (1) and (2) is performed based on the flowchart shown in FIG. After acquiring the search window area candidate, the character input completion determining unit 17 similarly acquires a subsequent frame of a frame that is m frames later in time than the target frame (step S20). As described above, since the search window region is displayed at least for a time that can be visually confirmed by the user, the acquisition interval m is performed at an interval smaller than that time.

  Here, a scene change is detected (step S21). When a scene change is detected, the search window area of the current frame is acquired as a state where character input has been completed (step S22). On the other hand, if no scene change is detected in step S21, color histogram analysis is performed on the search window area and compared with the immediately preceding frame (step S23). Here, the subsequent frame histogram information 15 shown in FIG. 1 is used. If there is a change in the histogram, it is determined that characters are still being input, the process proceeds to step S20, and an image after m frames is acquired. If the appearance frequency of the pixel does not change, it is determined that the character input is completed, and an image in the search window area is acquired (step S22).

  FIG. 10 is a diagram illustrating examples of cases where the search window area is recognized as a result of the above-described processes and cases where the search window area is not recognized. Although it cannot be represented by a diagram, after the area is approximated by a polygon, the hue in the area seems to be a search window (that is, there is a character color on a monochromatic background and the histogram is biased in two places. It is also possible to make an additional determination using what occurs.

  As described above, after the image in the search window area is acquired, the character recognition process is performed. That is, a character recognition process is performed on the acquired image in the search window area to acquire a keyword. An existing method can be used for this processing.

  In addition, when a keyword is extracted by each of the above processes, the image processing apparatus 10 notifies the user that the search is possible. This notification is, for example, displayed on the screen as “Search for XX when the Info button is pressed”. If the user is interested in the extracted keyword, the user can activate the browser included in the video reception device by pressing the Info button, and can easily search using the keyword.

  As described above, according to the present embodiment, it is possible to detect only the character input area in the video with high accuracy. As a result, keyword search can be easily performed without using a keyboard.

1 is a block diagram illustrating a schematic configuration of an image processing apparatus according to an embodiment of the present invention. 3 is a main flowchart showing the operation of the image processing apparatus according to the embodiment of the present invention. 6 is a flowchart showing an extraction operation of a character input area (hereinafter referred to as “search window area”). It is a flowchart which shows character input completion determination operation | movement. It is a figure which shows the example of the present image. It is a figure which shows the example of the edge image extracted from FIG. 5A. It is a figure which shows the example of the result of polygon approximation. It is a figure which shows the shape of a search window area | region. It is a figure which shows the shape of a search window area | region. It is a figure which shows the shape of a search window area | region. It is a figure which shows the example of the search window area | region detected by the said process. It is a figure which shows the result of having performed the corner detection with respect to the area | region. It is a figure which shows a mode that the search window area | region was each divided | segmented into the appropriate number about the vertical direction and the horizontal direction. It is a figure which shows a shape discrimination process notionally. It is a figure which shows the example when it is not recognized as a search window area | region as a result of each said process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image processing apparatus 11 Polygon extraction part 12 Shape determination part 13 Color histogram calculation part 14 Character arrangement | positioning determination part 16 Position determination part 17 Character input completion determination part 18 Character recognition part 19 Video signal input part

Claims (12)

An image processing apparatus for detecting a video of a character input area included in a video displayed on a screen,
An extraction unit for extracting a video approximate to the shape of the character input area from the video displayed on the screen;
A shape discriminating unit that extracts a video having the shape of the character input area out of the video approximated to the shape of the character input area extracted by the extraction unit;
Of the video having the shape of the character input area extracted by the shape discriminating unit, the character array existing in the video of the character input area approximates the character array to be input to the character input area at the time of keyword search. An image processing apparatus comprising: a character arrangement determining unit that extracts an image having a shape of a character input area that is an array.   The image processing apparatus according to claim 1, wherein the extraction unit extracts a video approximating a shape of a character input area from a video displayed on the screen by performing polygon approximation.   The extraction unit extracts, as a candidate for a character input area, a polygon in which the longest two straight lines are horizontal with respect to the screen from among polygons having four or more vertices among the polygon approximation results. The image processing apparatus according to claim 2.   The extracting unit selects a polygon having the longest two straight lines that are horizontal with respect to the screen from among the polygons having four or more vertices, and the lengths of the two straight lines are equal. The image processing apparatus according to claim 2, wherein the image processing apparatus is extracted as a character input area candidate.   The extraction unit obtains the height of the polygon from the distance between two straight lines horizontal to the screen among the results of the polygon approximation, and the height is the horizontal length of the polygon. The image processing apparatus according to claim 2, wherein an area having a length equal to or less than a predetermined ratio is extracted as a character input area candidate.   The character arrangement determining unit calculates a color histogram for the region obtained by the polygon approximation, determines a background color and a character color from the frequency, and determines the character color appearance frequency from the appearance frequency of the character in the character input region. The image processing apparatus according to claim 2, wherein it is determined whether or not there is any.   The character arrangement determination unit performs corner detection on the area obtained by the polygon approximation, determines the arrangement of characters in the area from the detected corner frequency, and determines the character input area from the character arrangement. The image processing apparatus according to claim 2, wherein it is determined whether or not the image processing is in progress.   Of the video having the shape of the character input area extracted by the shape determination unit, when a plurality of character input areas are extracted, from the ratio of the horizontal length between them and the positional relationship in the screen, The image processing apparatus according to claim 1, further comprising a position determination unit that removes an area other than the character input area. Of the video having the shape of the character input area extracted by the shape determination unit, when the number of characters in the video of the character input region changes with time, the character input completion determination unit determines whether or not the change in the number of characters has ended. Further comprising
The character arrangement determination unit extracts a video having a shape of a character input area that is an array that approximates an array of characters to be input to the character input area when searching for a keyword when the change in the number of characters ends. The image processing apparatus according to claim 1 or 2.   The video having the shape of the character input area extracted by the character arrangement determination unit is output to a character recognition unit that performs character recognition, and keyword search is performed using the characters recognized by the character recognition unit. The image processing apparatus according to any one of claims 1 to 9. An image processing method for detecting a video of a character input area included in a video displayed on a screen,
In the extraction unit, extracting a video approximate to the shape of the character input area from the video displayed on the screen by performing polygon approximation;
A step of extracting a video having the shape of the character input area from the video approximating the shape of the character input area extracted by the extraction unit in the shape determination unit;
In the character input completion determination unit, when the number of characters in the image of the character input region changes with time among the images having the shape of the character input region extracted by the shape determination unit, it is determined whether or not the change in the number of characters has ended. A step of determining;
In the character arrangement determination unit, when the change in the number of characters is finished, an arrangement of characters existing in the image of the character input area among the images having the shape of the character input area extracted by the shape determination unit is a keyword search. Extracting a video having a shape of a character input area that is an array that approximates an array of characters that are sometimes input to the character input area;
Outputting at least an image having the shape of the character input area extracted by the character arrangement determining unit. A program for detecting a video of a character input area included in a video displayed on a screen,
Processing to extract a video approximate to the shape of the character input area from the video displayed on the screen by performing polygon approximation;
A process of extracting an image having the shape of a character input area from among the images approximating the shape of the extracted character input area;
Among the images having the extracted character input area shape, when the number of characters in the image of the character input area changes with time, a process of determining whether or not the change in the number of characters has ended,
When the change in the number of characters is completed, an array of characters existing in the video of the character input area among the extracted video of the character input area should be input to the character input area at the time of keyword search A process of extracting an image having the shape of a character input area that is an array that approximates the character array;
A program for outputting a video having the shape of the extracted character input area and a series of processes to be read and executed by a computer as a command.