JP2007052575A - Metadata applying device and metadata applying method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a metadata applying device and a metadata applying method for accurately inputting the metadata of an image or an object by a simple method. <P>SOLUTION: This metadata applying device is provided with a display means for displaying an image including an object according to image data; an applying means to which metadata corresponding to the selected object of the image displayed by the display means are input and a meta data recording means for recording the metadata input by the applying means in a recording medium by associating the metadata with the image data. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a metadata grant apparatus and metadata grant method for giving metadata to various image data, and more particularly, a metadata grant apparatus and metadata grant capable of accurately and easily inputting metadata relating to images and subjects. Regarding the method.

  Conventionally, attribute information called metadata is given according to the image in the image data, the contents of the subject, and the like, and this metadata is recorded in association with the image data.

Note that such metadata assignment may be based on automatic recognition of an image or recognition by an operator.
As for the addition and recording of such metadata, what is automatically performed at the time of recording a moving image is described in Patent Document 1 below.

Further, what is recognized and manually input by the operator is described in Patent Document 2 and Patent Document 3 below.
Japanese Patent Laying-Open No. 2005-39354 (first page, FIG. 1) JP 2000-57362 A (first page, FIG. 1) JP 2001-272963 A (first page, FIG. 1)

  In the automatic input method of Patent Document 1 described above, since character information included in an image is automatically recognized and given metadata, it can operate only in a limited situation and is limited. There is a problem that only metadata can be obtained.

  The manual input method of Patent Document 2 described above is to specify and input the position of dust contained in an image using a stylus pen or the like. That is, since only position information can be input, the metadata is limited. Also in Patent Document 3, position information is input using a pointing device.

  Note that metadata can also be handled by general image management software or the like, and it is possible to add arbitrary keyword information to the position information of an arbitrary object or an image. When adding such metadata, the original file is not changed at all, and the data is stored in a database called a catalog that is different from the original file.

  The present invention has been made in view of the above problems, and provides a metadata providing apparatus and a metadata providing method capable of accurately inputting metadata of an image or a subject by a simple method.

That is, the present invention as means for solving the above problems is as described below.
(1) According to the first aspect of the present invention, a display unit that displays an image including a subject according to image data, and an input in which metadata corresponding to the selected subject of the image displayed on the display unit is input And a metadata recording unit that records the metadata input from the adding unit on a recording medium in association with the image data.

  (2) In the invention according to claim 2, the display unit displays the displayed subject candidates as metadata options, and the adding unit includes the subject included in the image displayed on the display unit. A type candidate is selected from the options, and when a subject type is selected from the options, an input of a position or size on an image associated with the option can be received. The metadata recording means records the selected option as metadata, acquires the position or size of the subject on the image designated by the assigning means, and records it as metadata. The metadata providing apparatus according to claim 1.

  (3) In the invention according to claim 3, the display means displays a candidate of a displayed subject as a metadata option, and the assigning means includes a subject included in an image displayed on the display means. The type candidate and the subject part candidate are selected from the options, and when the subject or the subject part is selected from the options, an input of the position or size on the image associated with the option is received. The metadata recording unit records the selected option as metadata and acquires the position or size of the subject on the image designated by the adding unit. The metadata adding apparatus according to claim 1, wherein the metadata adding apparatus records as metadata.

  (4) The invention according to claim 4 is characterized in that, when the metadata recording means cannot acquire the position or size of the subject, it records that the object could not be acquired. Item 4. The metadata providing device according to Item 3.

  (5) The invention according to claim 5 is configured such that the display means is capable of displaying a candidate for a reason when the position or size of the subject cannot be obtained, and the giving means includes the display 5. The metadata providing apparatus according to claim 4, wherein the metadata giving apparatus is configured to be able to accept input of a candidate for a reason displayed in the means.

  (6) In the invention according to claim 6, in addition to the image being displayed, the display means displays a subject model of the same type as the subject type selected or input by the assigning means. It is configured to be able to change the angle on the display screen of the model, and the metadata recording unit acquires the angle of the subject from the subject model changed by the providing unit, and records it as metadata. The metadata providing apparatus according to any one of claims 1 to 5, wherein

  (7) The invention according to claim 7 is characterized in that, when the metadata recording unit cannot acquire the angle of the subject, it records that the metadata could not be acquired. It is a grant device.

  (8) The invention according to claim 8 is configured such that the display means can display a candidate for a reason when the angle of the subject cannot be obtained, and the assigning means displays on the display means The metadata providing apparatus according to claim 7, wherein the metadata adding apparatus is configured to receive an input of a candidate for the reason for being performed.

  (9) The invention according to claim 9 is configured such that the assigning unit is capable of receiving an input of an arbitrary character string. This is a metadata providing apparatus.

  (10) In the invention according to claim 10, the display means displays composition candidates for the displayed image as metadata options, and the assigning means composes the image displayed on the display means. 2. The metadata grant according to claim 1, wherein the candidate is selected from the options, and the metadata recording unit records the selected options as metadata. Device.

  (11) The invention according to claim 11 holds test image data, and repeats the operation of displaying the test image data on the display unit and receiving the input of metadata by the adding unit a plurality of times. The difference between the first input value and the converged input value is extracted as a correction value from the metadata input multiple times, and normal image data is displayed on the display means to the adding means. 11. The metadata providing apparatus according to claim 1, further comprising a control unit that corrects the metadata with the correction value when the metadata is input.

  (12) The invention according to claim 12 is an automatic recognition unit that automatically recognizes input image data and extracts metadata corresponding to an image or a subject, the metadata input from the adding unit, 11. The metadata provision according to claim 1, further comprising: an evaluation unit that compares the metadata extracted by the automatic recognition unit and evaluates the automatic recognition. Device.

  (13) In the invention according to claim 13, the metadata recording means compares the metadata input from the assigning means or at least one of the metadata extracted by the automatic recognition means with the comparison means. 13. The metadata providing apparatus according to claim 12, wherein the compared result data is recorded on a recording medium in association with the image data.

  (14) According to the fourteenth aspect of the present invention, an image including a subject is displayed according to image data using the metadata providing apparatus according to any of the first to thirteenth aspects, and the displayed image A metadata providing method comprising: receiving metadata input according to the selected subject, and recording the input metadata in association with the image data on a recording medium.

In the present invention, the following effects can be obtained.
(1) According to the first aspect of the present invention, an image including a subject is displayed on the display unit according to the image data, and metadata corresponding to the selected subject of the displayed image is input from the operator via the assigning unit. Then, the metadata recording means records the input metadata on the recording medium in association with the image data.

As a result, it is possible to accurately input metadata about the image by a simple method.
(2) In the invention of claim 2, the displayed subject type candidates are displayed as metadata options, and the options selected for the subject type candidates are recorded as metadata and designated. The position or size of the subject on the image is acquired and recorded as metadata.

As a result, the type and position information of the object that is the subject included in the image can be accurately input by a simple method.
(3) In the invention of claim 3, the displayed subject type candidate and subject part candidate are displayed as metadata options, and the subject type candidate and subject part candidate are selected. The option is recorded as metadata, and the position or size of the subject on the designated image is acquired and recorded as metadata.

This makes it possible to accurately input the type of the object that is the subject included in the image and each part and position information included in the object by a simple method.
(4) In the invention according to claim 4, when the position or size of the subject cannot be acquired, the metadata recording means records that the acquisition was not possible.

Thus, even when the position or size of the object that is the subject of the image cannot be specified, the information can be easily input.
(5) In the invention according to claim 5, candidates for reasons when the position or size of the subject cannot be obtained are displayed on the display means, and the selection of the candidates for reasons displayed is selected from the operator via the assigning means. Accept.

As a result, even when the position or size of the object that is the subject of the image cannot be specified, the information can be easily input by selecting the reason candidate.
(6) In the invention described in claim 6, in addition to the image being displayed on the display means, a subject model of the same type as the selected or input subject type is displayed, and the subject model is provided via the assigning means by the operator. In response to the operation of changing the angle on the display screen, the metadata recording means acquires the angle of the subject from the changed subject model and records it as metadata.

  In this way, by displaying the subject model and accepting an operation for changing the angle on the display screen of the subject model from the assigning means, the angle of the subject can be accurately input by a simple method. Become.

(7) In the seventh aspect of the invention, when the subject angle cannot be obtained, the metadata recording means records that the subject angle could not be obtained.
Thus, even when the angle of the object that is the subject of the image cannot be specified, the information can be easily input.

  (8) In the invention according to claim 8, candidates for reasons when the angle of the subject cannot be acquired are displayed on the display means, and selections from the operator are accepted via the assigning means for the displayed reason candidates.

As a result, even when the angle of the object that is the subject of the image cannot be specified, the information can be easily input by selecting the candidate for the reason.
(9) In the invention according to claim 9, the input of metadata of an arbitrary character string from the operator is received through the assigning means.

As a result, it is possible to input various types of metadata using an arbitrary character string with respect to an image, a subject, and a subject part by a simple method.
(10) In the invention described in claim 10, the composition candidate of the displayed image is displayed as a metadata option on the display means, and the selection means from the operator is given to the candidate for the composition of the displayed image. The options selected for the image composition candidates are recorded as metadata.

  In this way, composition candidates are displayed together with images, and selection of composition candidates is accepted from the assigning means, so that image composition candidates can be accurately input by a simple method.

  (11) In the invention described in claim 11, the operation of displaying the test image data on the display means and receiving the metadata input by the assigning means is repeated a plurality of times, and the difference between the initial input value and the converged input value is repeated. Are extracted as correction values, and when normal image data is displayed on the display means and metadata is input from the operator to the assigning means, the metadata is corrected with the correction values.

  In this way, by correcting the normal metadata input by extracting the correction value from the difference between the initial input value and the converged input value, it is possible to perform correction according to the tendency of the stable input value. Therefore, more accurate metadata can be input.

  (12) In the invention described in claim 12, the metadata inputted by the operator via the assigning means is compared with the metadata extracted by the automatic recognition means, and the evaluation for the automatic recognition is performed by the evaluation means. .

  Thus, by comparing the metadata input by the operator with the automatically recognized metadata, it becomes possible to evaluate the metadata automatically recognized by the automatic recognition means, and by taking statistics, etc. It becomes possible to grasp the weak points of automatic recognition.

  (13) In the invention described in claim 13, at least one of the metadata input from the operator via the assigning means or the metadata extracted by the automatic recognition means and the comparison result data compared by the comparison means are It is recorded on a recording medium in association with image data.

  In this way, by comparing the metadata input by the operator with the automatically recognized metadata, and recording at least one of the manually input or automatically recognized metadata and the comparison result data, the automatic recognition means It is possible to evaluate automatically recognized metadata, and it is possible to grasp weak points of automatic recognition by taking statistics and the like.

  (14) In the invention described in claim 14, according to any one of the above (1) to (13), an image including a subject is displayed on the display unit in accordance with the image data, and the selected subject of the displayed image is displayed. The corresponding metadata is input from the operator via the adding unit, and the metadata recording unit records the input metadata in association with the image data on the recording medium.

  As a result, it is possible to accurately input metadata relating to images and subjects by a simple method. Further, it is possible to accurately input the type of the object that is the subject included in the image and each part and position information included in the object by a simple method.

The best mode for carrying out the present invention (hereinafter referred to as an embodiment) will be described below in detail with reference to the drawings.
<First embodiment>
First, with reference to FIG. 1 and subsequent drawings, a specific example of the electrical configuration of the first embodiment when the metadata providing apparatus and the metadata providing method of the present invention are applied to the metadata providing apparatus 100 will be described.

  That is, in the present embodiment, the operation of the metadata providing apparatus is a processing procedure of the metadata providing method, and the execution procedure of the metadata providing program is the operation of the metadata providing apparatus or the processing procedure of the metadata providing method.

<Constitution>
Here, the metadata providing apparatus 100 includes the following units. FIG. 1 is a block diagram showing an example of a specific circuit and apparatus together with a functional configuration of main parts of the metadata providing apparatus 100 of the present embodiment.

The configuration of the metadata providing apparatus 100 is an implementation means for executing the metadata providing method, and is also each routine of the metadata providing program.
Here, reference numeral 110 denotes a control unit that executes various controls for giving metadata in the metadata giving apparatus 100. The control unit 110 is in the metadata adding apparatus 100 and performs control as a metadata recording unit that records input metadata on a recording medium in association with image data.

  The control unit 110 includes a CPU 101 as a central part of a control unit that controls each unit, a ROM 102 that stores various data and various programs, a RAM 103 as a work area in which image data is expanded and processed, have.

  Reference numeral 170 denotes an operation unit as a granting means for an operator (user) to input various instructions to the metadata granting apparatus 100 or to execute an input of metadata grant, and 180 denotes a meta during operation of the metadata grant apparatus 100. A display unit 190 serving as a display unit that displays various images and various states at the time of the data providing program operation, and an HDD as a recording medium on which image data, metadata, and the like are recorded. Reference numeral 200 denotes an external recording device as a recording medium on which image data, metadata, and the like are recorded.

Note that the metadata providing apparatus 100 described above performs the following characteristic operations from the control unit 110 or the control 2 of the CPU 101.
(1) In this embodiment, an image including a subject is displayed on the display unit 180 in accordance with image data in accordance with an instruction from the control unit 110, and metadata corresponding to the selected subject of the displayed image (identifies the subject). (Metadata as information) is input from the operator via the operation unit 170, and the control unit 110 records the input metadata in the HDD 190 or the external storage device 200 in association with the image data.

  (2) In this embodiment, subject type candidates are displayed as metadata options in accordance with instructions from the control unit 110, the options selected for the subject type candidates are recorded as metadata, and a designated image is also displayed. The position or size of the upper subject is acquired and recorded as metadata.

  (3) In this embodiment, subject type candidates and subject part candidates are displayed as metadata options according to instructions from the control unit 110, and options selected for subject type candidates and subject part candidates are displayed. Is recorded as metadata, and the position or size of the subject on the designated image is acquired and recorded as metadata.

(4) In this embodiment, when the position or size of the subject cannot be acquired, the control unit 110 records that the acquisition has failed.
(5) In this embodiment, a candidate for a reason when the position or size of the subject cannot be acquired according to an instruction from the control unit 110 is displayed on the display unit 180, and the operator selects the displayed reason candidate. It accepts via the part 170.

  (6) In this embodiment, in accordance with an instruction from the control unit 110, in addition to the image being displayed on the display unit 180, a subject model of the same type as the selected or input subject type is displayed, and the operator operates the operation unit 170. In response to the operation of changing the angle on the display screen of the subject model, the control unit 110 acquires the angle of the subject from the changed subject model and records it as metadata.

(7) In this embodiment, when the subject angle cannot be obtained, the control unit 110 records that the subject angle could not be obtained.
(8) In this embodiment, according to an instruction from the control unit 110, candidates for reasons when the angle of the subject cannot be acquired are displayed on the display unit 180, and selections from the operator for the displayed reason candidates are displayed on the operation unit 170. Accept through.

(9) In this embodiment, the input of metadata of an arbitrary character string from the operator is received via the operation unit 170 according to an instruction from the control unit 110.
(10) In this embodiment, an image composition candidate is displayed as a metadata option on the display unit 180 according to an instruction from the control unit 110, and the operator selects a displayed image composition candidate from the operator 170. The option selected for the image composition candidate is recorded as metadata.

  (11) In this embodiment, the operation of displaying test image data on the display unit 180 and accepting input of metadata by the operation unit 170 in accordance with an instruction from the control unit 110 is repeated a plurality of times, and the initial input value and convergence are obtained. The difference between the input value and the input value is extracted as a correction value. When normal image data is displayed on the display unit 180 and metadata is input from the operator to the operation unit 170, the metadata is corrected with the correction value. To do.

  (12) In this embodiment, in accordance with an instruction from the control unit 110, metadata by selecting the above options or inputting characters is described hierarchically in a markup language such as XML, and associated with the image data in the HDD 190 or Record in the external storage device 200.

  (13) In this embodiment, when the metadata is associated with the image data and recorded in the HDD 190 or the external storage device 200 in accordance with an instruction from the control unit 110, the metadata is recorded as supplementary information in the image data.

  (14) Alternatively, in this embodiment, when the metadata is recorded in the HDD 190 or the external storage device 200 in association with the image data in accordance with an instruction from the control unit 110, the metadata is recorded as data different from the image data. .

(15) This embodiment has a function of adding an arbitrary item when adding metadata to image data according to an instruction from the control unit 110.
(16) In this embodiment, in accordance with an instruction from the control unit 110, metadata input by the operator via the operation unit 170 as described in the above (1) to (15) and the control unit 110 are provided. The metadata extracted by the automatic recognition means is compared, and the automatic recognition is evaluated by the evaluation means provided in the control unit 110.

  (17) In this embodiment, in accordance with an instruction from the control unit 110, metadata input from the operator via the operation unit 170 as described in (1) to (15) above, or provided in the control unit 110. At least one of the metadata extracted by the automatic recognition means and the comparison result data (matching degree, similarity degree, etc.) compared by the comparing means provided in the control unit 110 are associated with the image data in the HDD 190 or Record in the external storage device 200.

  (18) In this embodiment, in accordance with an instruction from the control unit 110, metadata input from the operator via the operation unit 170 as described in (1) to (15) above, or provided in the control unit 110. When recording at least one of the metadata extracted by the automatic recognition means, the comparison result data (matching degree, similarity degree, etc.) compared by the comparing means provided in the control unit 110 is added to the metadata. Record in the form.

  (19) In this embodiment, in accordance with an instruction from the control unit 110, metadata input from the operator via the operation unit 170 as described in (1) to (15) above and the control unit 110 are provided. When the metadata extracted by the automatic recognition means is compared, the automatic recognition conditions (the version of the program used, the learning data used, the evaluation date, etc.) are added to the recorded metadata.

  (20) In this embodiment, in accordance with an instruction from the control unit 110, metadata input from the operator via the operation unit 170 as described in the above (1) to (15) and the control unit 110 are provided. When the metadata extracted by the automatic recognition means is compared and evaluated again under different conditions, the evaluation result added to the metadata is updated with the newly evaluated result.

  (21) Alternatively, in this embodiment, in accordance with an instruction from the control unit 110, metadata input from the operator via the operation unit 170 as described in (1) to (15) above, and the control unit 110 are provided. In the case of comparing with the metadata extracted by the automatic recognition means, and evaluating again under different conditions, a new evaluation was made while leaving the evaluation results added to the metadata as a history. Append the results.

  (22) In this embodiment, in accordance with an instruction from the control unit 110, learning data used for automatic recognition is created from metadata input from the operator via the operation unit 170 as described in (1) to (15) above. A learning data creating means is provided in the control unit 110, and an automatic recognition learning means for learning based on the created learning data is provided in the control unit 110 to create and learn learning data related to automatic recognition.

  (23) In this embodiment, according to an instruction from the control unit 110, dictionary data used for automatic recognition is created from metadata input from the operator via the operation unit 170 as described in (1) to (15) above. A dictionary data creation means is provided in the control unit 110, and an automatic recognition means for automatically recognizing the created dictionary data is provided in the control unit 110 to perform automatic recognition based on the dictionary data.

  (24) In this embodiment, in accordance with an instruction from the control unit 110, a template for creating a template to be used for automatic recognition from metadata input from the operator via the operation unit 170 as described in (1) to (15) above. A creation unit is provided in the control unit 110, and a template matching processing unit that performs a template matching process using the created template is provided in the control unit 110, and a template matching process is performed using the created template.

  (25) In this embodiment, in accordance with instructions from the control unit 110, image classification means for classifying images according to metadata input from the operator via the operation unit 170 as in the above (1) to (15) is controlled. It is provided in the unit 110 and executes creation of metadata and image classification processing based on the metadata.

<Operation>
In the above configuration, the control unit 110 performs the following operation according to the metadata providing program.

  First, the control unit 110 displays a list of a plurality of image data stored in the HDD 190 or the external storage device 200 in a reduced state on the display unit 180 as shown in FIG. Is selected (S1 in FIG. 2).

  When the operator uses a pointing device to select one of the reduced images by clicking or the like, the control unit 110 enlarges and displays the selected image as a metadata addition target image as shown in FIG. FIG. 4 shows a state in which the metadata addition target image is displayed on the left half of the central portion of the display screen of the display unit 180.

  Here, according to an instruction from the control unit 110, candidates such as whether the metadata application target image is the main subject or the background in the image are displayed as options on the upper right side of the image on the display unit 180 (FIG. 4). (1)).

Here, it is assumed that the operator has selected “main subject” as the metadata addition target image (“main subject” in FIG. 4A is a check mark).
Then, according to an instruction from the control unit 110, whether the main subject that is the metadata addition target image is a person / male, a person / woman, a building, a road, or the like in the central portion on the right side of the image on the display unit 180. Are displayed as options (FIG. 2 S3, FIG. 4 (2)).

  Here, it is assumed that the operator has selected “person” and “woman” as the type of metadata assignment target image (check mark “person” and “woman” in FIG. 2 S4 and FIG. 4 (2)). When the selection from the operator is confirmed (OK in FIG. 2 S5), control unit 110 proceeds to the next step.

  Then, in accordance with an instruction from the control unit 110, a candidate for a subject type such as whether the main subject, which is a metadata addition target image, is a face, a hand, a torso, a foot, or the like is selected at the right end of the display unit 180. (FIG. 2 S6, FIG. 4 (3)).

  Here, it is assumed that the operator has selected “face” as the subject of the metadata assignment target image (check mark “face” in FIG. 2 S7, FIG. 4B). When the selection from the operator is confirmed (OK in FIG. 2 S8), control unit 110 proceeds to the next step.

  Then, according to an instruction from the control unit 110, an input screen for receiving input of the size and position of the subject is displayed on the reduced image in the lower left portion of the display unit 180 (S9 in FIG. 2). At this stage, for example, on the reduced image (FIG. 5 (1)) on the lower left of the screen, the operator uses a pointing device of the operation unit 170 or the like to place a predetermined frame on a desired subject to which metadata is to be added. Move.

  Here, in response to the operation from the operation unit 170, the control unit 110 displays an enlarged image of the frame portion on the center left half of the display screen of the display unit 180 as the frame on the subject moves (FIG. 5). (2)). Here, the operator uses the pointing device of the operation unit 170 to adjust the size of the frame so as to match the size of the subject. In the case of FIG. 5, since the subject is a face, the operator uses a pointing device to adjust the size of the frame so as to surround the entire face.

  As described above, the control unit 110 displays the position of the subject (position on the image) by displaying the frame whose position and size are matched by the operator on the object selected as the subject in the above-described step. And the size of the subject (pixel size on the image) are acquired (S10 in FIG. 2). Here, the control unit 110 displays the pixel size corresponding to the size of the subject acquired by the frame as described above on the display screen of the display unit 180 (FIG. 5 (3)).

  Note that, depending on the frame set by the operator as described above, the control unit 110 may determine the position and size of the subject because the subject's face is hidden by hair, hidden by an obstacle, too bright, or too dark. It may be impossible to obtain In such a case, the control unit 110 displays an option as a candidate for the reason on the display screen of the display unit 180 (FIG. 6 (1)), and causes the operator to select it. In this case, instead of the metadata of the position and size of the subject, the selected option for the reason that the position and size cannot be acquired is given as metadata.

When acquisition of the position of the subject and the size of the subject by the operation from the operator as described above is confirmed (OK in S11 in FIG. 2), the control unit 110 proceeds to the next step.
Then, according to an instruction from the control unit 110, an input screen for receiving the input of the position and name of the part (item) of the subject is displayed on the image in the left half of the center of the display unit 180 (FIG. 2 S12, FIG. 7). For this reason, in accordance with an instruction from the control unit 110, an image of the subject and a candidate for the name of the corresponding part are displayed on the display unit 180 as options.

  At this stage, for example, on the image on the left side of the center of the screen, the operator uses the pointing device of the operation unit 170 to click on the position on the desired subject to which metadata is to be added to form dots or the like. (FIG. 7 (1)). Then, an option of a name corresponding to the part of the subject on which the dot is formed is selected (FIG. 7 (2)). Here, an example is shown in which dots are formed at the position of the right eye corner on the subject image, and an option named “right eye corner” is selected correspondingly (S13 in FIG. 2). It should be noted that there is no problem as long as the association between the selection of the choice of the name of the part of the subject and the click on the image of the position of the part of the subject is associated.

  Here, since the subject was a face, the parts of the subject are the right eyebrows, right black eye, right eye corner, right eye head, right eyeball upper end, right eyeball lower end, left eyebrows, left black eye, left eye corner, left eye head, left eyeball upper end. , Left eyeball lower end, nasal head, right nose end, left nose end lip, right ear lobe, left ear lobe, chin, etc. If the subject is something else, the part of the subject is also corresponding to it.

  Then, in accordance with an instruction from the control unit 110, the input of the position and name of the subject part by the operation from the operator as described above is repeatedly executed as necessary (S14 in FIG. 2). When the input of the part of the subject is confirmed (OK in FIG. 2 S14), the control unit 110 proceeds to the next step.

  Here, the control unit 110 displays a subject model of the same type as the selected or input subject type beside the enlarged image of the subject displayed in the center left half of the display screen of the display unit 180 (FIG. 2S15, FIG. 8 (1)).

  Then, according to an instruction from the control unit 110, an operation for changing the angle on the display screen of the subject model via the pointing device of the operation unit 170 by the operator (operation for rotating the subject model, FIG. 8 (2)). Accept. Then, the control unit 110 serving as the metadata recording unit acquires the angle of the subject from the subject model whose orientation has been changed, and generates metadata (S16 in FIG. 2). When the input of the angle of the subject model is confirmed (OK in FIG. 2 S17), the control unit 110 proceeds to the next step.

  In the above input and selection, a candidate option is selected, but an arbitrary character string may be accepted. Moreover, you may enable it to add an arbitrary item and arbitrary candidates at the stage of initial setting.

  As described above, if there is no correction in the input and selection of the image type, the type of the subject, the position and size of the subject, the type of the part of the subject, and the angle of the subject, the control unit 110 performs the above input and selection. Generate metadata from the contents of. Then, the control unit 110 records the generated metadata in the HDD 190 or the external storage device 200 in a state in which the generated metadata is associated with the metadata addition target image.

  Here, in accordance with an instruction from the control unit 110, metadata based on selection of the above options or character input is hierarchically described in a markup language such as XML, and is associated with the image data in the HDD 190 or the external storage device 200. Record.

  FIG. 9 is an example of metadata described in XML as a markup language. Here, a meta image when “face” is selected as the subject included in the image data aaa.jpg and the positions of the center of the right eyebrow, the center of the left eyebrow, the right eye, and the left eye are specified as the subject part. The state of data description is shown.

  In this embodiment, when the metadata is associated with the image data and recorded in the HDD 190 or the external storage device 200 in accordance with an instruction from the control unit 110, the metadata is recorded in the image data as supplementary information. In this case, metadata can be recorded inside the header of image data. Alternatively, in this embodiment, when the metadata is associated with the image data and recorded in the HDD 190 or the external storage device 200 according to an instruction from the control unit 110, the metadata is recorded as data different from the image data. Also good.

  In addition, if metadata addition is to be executed for the other images displayed in the list shown in FIG. 3, the control unit 110 returns to step 1 and repeats the same procedure from the selection of the image (Y in FIG. 2 S20). If the addition of metadata for another image is not executed, the above processing is terminated (Y in FIG. 2).

  In the embodiment described above, candidates for the “composition” of the image displayed on the display screen of the display unit 180 are displayed as metadata options, and the operator selects the composition candidate for the displayed image. It is also possible to record, as metadata, options received via the operation unit 170 and selected for image composition candidates.

Here, “composition” means
・ Landscape: Upper half sky, mountain, ...
・ Portrait: One, two, three ...
・ Building: right half building, left half building, ...
・ Light intensity distribution: flash photography, backlight, ...
And so on.

  In this manner, composition candidates are displayed together with images, and selection of composition candidates is accepted from the operation unit 170, so that image composition candidates can be accurately input by a simple method.

  According to the above embodiment, it is possible to accurately input metadata about an image by a simple method. In addition, it is possible to accurately input the type and position information of the object that is the subject included in the image by a simple method. Further, it is possible to accurately input the type of the object that is the subject included in the image and each part and position information included in the object by a simple method. Even when the position or size of the object that is the subject of the image cannot be specified, the information can be easily input. Even when the position or size of the object that is the subject of the image cannot be specified, it is possible to easily input the information by selecting the reason candidate. In addition, the angle of the subject can be accurately input by a simple method. Even if the angle of the object that is the subject of the image cannot be specified, the information can be easily input. Even when the angle of the object that is the subject of the image cannot be specified, it is possible to easily input the information by selecting a candidate for the reason. In addition, it is possible to input various metadata using an arbitrary character string with respect to an image, a subject, and a subject part by a simple method.

  In the above embodiment, information for identifying a subject portion and other image portions (information for identifying an object) is given as metadata and recorded in association with image data together with position information. Also good.

<Second embodiment>
For example, when a plurality of operators perform input work related to metadata assignment, position information input to the same image and the same object may differ depending on the operator. In addition, there are cases where the same operator inputs position information input to the same image and the same object each time the information is input.

  In view of this, a test image different from a normal metadata application target is displayed on the display screen of the display unit 180 in accordance with an instruction from the control unit 110 once every specific time or once every fixed time. Then, the control unit 110 displays, on the display screen of the display unit 180, a test reception screen that allows the operator to input options for adding metadata.

  The options of metadata input based on the test image are recorded in the HDD 190 together with the input worker and the input time in a location different from the metadata of the image to which normal metadata is added according to the instruction of the control unit 110. To do. The control part 110 accumulate | stores the metadata, and grasp | ascertains an operator's personal habit by detecting the difference between input workers, between input time, or a statistics value. The screen is controlled so as to be re-input by assuming that the metadata input by a person with a large difference or in the time zone has low reliability.

  Alternatively, when there is a tendency to deviate between the initial input value and the converged input value, a correction value is extracted from the tendency (difference). Alternatively, when the position information tends to shift in a certain direction in proportion to time, the control unit 110 records the tendency in the HDD 190 as a correction value. Then, at the time of normal metadata assignment, the control unit 110 modifies metadata such as input position information according to the correction value.

  In this way, by correcting the normal metadata input with the correction value obtained based on the addition of the metadata of the test image data, it becomes possible to perform correction according to the tendency of the stable input value, and more accurately. Metadata can be input.

<Third embodiment>
In the third embodiment, the control unit 110 includes automatic recognition means for automatically recognizing an image and extracting metadata.

  Then, the control unit 110 is provided with a comparison unit that compares the metadata obtained by the selection of options by the operator in the first embodiment and the metadata extracted by the automatic recognition unit.

Further, the control unit 110 is provided with evaluation means for evaluating the metadata extracted by the automatic recognition means using the result of comparison by the comparison means.
Thus, by comparing the metadata input by the operator with the automatically recognized metadata, it becomes possible to evaluate the metadata automatically recognized by the automatic recognition means, and by taking statistics, etc. It becomes possible to grasp the weak points of automatic recognition.

  For example, it is assumed that a rectangular region surrounding a face is added to an image as metadata according to the first embodiment. Here, a rectangular area surrounding the face that is given to the image as metadata is a correct face area (solid line in FIG. 10). On the other hand, as an automatic recognition technique for detecting a face rectangular area, for example, a technique described in Japanese Patent Application Laid-Open No. 2004-234689 can be adopted. A face area obtained by this automatic recognition technique is set as an automatically detected face area (metadata obtained by automatic recognition from image data) (one-dot chain line in FIG. 10).

Then, the comparison unit in the control unit 110 compares the correct face area and the automatically detected face area to calculate the evaluation value detection rate and the number of erroneous detections. The detection rate is calculated according to the following formula for an area that has the largest overlap with the correct answer area among the automatically detected face areas.
Detection rate = (overlap area between correct face area and automatically detected face area) / area of correct answer area,
Further, the number of erroneous detections is calculated by counting the number of areas that do not overlap with the correct answer area among the automatically detected face areas. The index of evaluation shown here is an example, and the present invention is not limited to this. In this case, the automatically detected face area and the evaluation value (evaluation value detection rate and the number of erroneous detections) are added to the image as metadata.

In addition to the rectangular area surrounding the face, when the position of eyebrows and eyes included in the face is added to the image as metadata, the detection rate is similarly obtained for the position.
As described above, when the metadata input by the operator (manual input) and the metadata extracted by the automatic recognition means (automatic recognition) are compared and evaluated, the manually input metadata or automatic At least one of the recognized metadata and the comparison result data (matching degree, similarity degree, etc.) compared by the comparison means provided in the control unit 110 are associated with the image data in the HDD 190 or the external storage device 200. Record it.

In the case of performing the evaluation as described above, it is desirable to record the comparison result data (degree of coincidence, similarity, etc.) in a form that is added to the original metadata.
In addition, when the metadata input by the operator (manual input) and the metadata extracted by the automatic recognition means (automatic recognition) are compared and evaluated as described above, the automatic recognition condition (the program used) Version, learning data used, evaluation date, etc.) should be added to the recorded metadata.

  In addition, when the metadata input by the operator (manual input) and the metadata extracted by the automatic recognition means (automatic recognition) are compared and evaluated as described above, they are evaluated again under different conditions. Therefore, it is desirable to update the evaluation result added to the metadata with the newly evaluated result.

  In addition, when the metadata input by the operator (manual input) and the metadata extracted by the automatic recognition means (automatic recognition) are compared and evaluated as described above, they are evaluated again under different conditions. It is also desirable to add the newly evaluated result while leaving the evaluation result added to the metadata as a history.

  FIG. 11 and FIG. 12 below are examples of a series of metadata described in XML as a markup language. Here, a meta image when “face” is selected as the subject included in the image data aaa.jpg and the positions of the center of the right eyebrow, the center of the left eyebrow, the right eye, and the left eye are specified as the subject part. The state of data description is shown. FIG. 11 is a metadata portion regarding correct answers, and FIG. 12 is a portion including a metadata portion by automatic detection and an evaluation result (detection rate, number of erroneous detections).

  In the third embodiment described above, when information such as gender (male / female) and presence / absence of glasses is added to the metadata at the same time as the correct face area, the evaluation result and information such as gender should be compared. Thus, it becomes possible to investigate the tendency of the image that the algorithm of the automatic recognition means to be evaluated is good at and not good at.

<Fourth embodiment>
In the fourth embodiment, learning data creating means for creating learning data used for automatic recognition for adding metadata from metadata input from an operator via the operation unit 170 as in the first embodiment described above. Is provided in the control unit 110. The control unit 110 further includes automatic recognition learning means for learning using the learning data created by the learning data creating means. Then, with respect to the addition of metadata, creation of learning data related to automatic recognition and learning using the created learning data are performed.

  Here, using the face area obtained in the first embodiment described above, a face area image to be input as learning data of a neural network disclosed in, for example, Japanese Patent Application Laid-Open No. 2004-234689 is created. By doing in this way, it becomes possible to create learning data related to automatic recognition and to learn using the created learning data with regard to metadata assignment.

<Fifth embodiment>
In the fifth embodiment, as in the first embodiment described above, dictionary data creating means for creating dictionary data used for automatic recognition for adding metadata from metadata input from the operator via the operation unit 170. Is provided in the control unit 110. Then, an automatic recognition unit that automatically recognizes the dictionary data created by the dictionary data creation unit is provided in the control unit 110 to perform automatic recognition based on the dictionary data.

  Here, when authentication is performed using a face image, a face image to be authenticated (or a feature amount extracted from the face image) is recorded as dictionary data in advance. The dictionary data is created using the face area obtained in the first embodiment. Since a person name can be given in association with the area, dictionary data can be created.

<Sixth embodiment>
In the sixth embodiment, as in the first embodiment described above, the template creation means for creating a template used for automatic recognition for adding metadata is controlled from the metadata input from the operator via the operation unit 170. The unit 110 is provided. The control unit 110 further includes template matching processing means for performing template matching processing using the template created by the template creation means. Then, with respect to metadata assignment, a template relating to automatic recognition and a template matching process using the created template are performed.

  For example, in the template matching method disclosed in Japanese Patent Laid-Open No. 2002-44489, the skin color region is determined based on whether or not the color feature amount is within a predetermined range. Therefore, when the feature value of the skin color is determined, the feature value of the color in the face area obtained in the first embodiment is extracted, and the feature value range of the skin color is determined.

<Seventh embodiment>
In the seventh embodiment, an image classification means for classifying images according to metadata input from an operator via the operation unit 170 as in the first embodiment described above is provided in the control unit 110 according to an instruction from the control unit 110. Prepare. This makes it possible to execute creation of metadata and image classification processing based on the created metadata.

<Other embodiments>
In each of the above embodiments, an image such as a face photograph or a landscape photograph has been taken as a specific example. However, the present invention is not limited to this type of image. That is, in medical images and the like, it is also possible to apply metadata addition as shown in the first embodiment to the seventh embodiment to various diagnoses.

It is a functional block diagram which shows the structure of the metadata provision apparatus of 1st embodiment of this invention. It is a flowchart which shows operation | movement (the procedure of a metadata provision method, the procedure of a metadata provision program) of the metadata provision apparatus of 1st embodiment of this invention. It is explanatory drawing which shows the mode of the display in the display part in 1st embodiment of this invention using the photograph of a halftone. It is explanatory drawing which shows the mode of the display in the display part in 1st embodiment of this invention using the photograph of a halftone. It is explanatory drawing which shows the mode of the display in the display part in 1st embodiment of this invention using the photograph of a halftone. It is explanatory drawing which shows the mode of the display in the display part in 1st embodiment of this invention using the photograph of a halftone. It is explanatory drawing which shows the mode of the display in the display part in 1st embodiment of this invention using the photograph of a halftone. It is explanatory drawing which shows the mode of the display in the display part in 1st embodiment of this invention using the photograph of a halftone. It is explanatory drawing which shows an example of the description of the metadata provided by the metadata provision apparatus of 1st embodiment of this invention. It is explanatory drawing which shows typically the mode of the difference of the area | region by manual input and automatic recognition in 3rd embodiment of this invention. It is explanatory drawing which shows an example of the description of the metadata provided by the metadata provision apparatus of 3rd embodiment of this invention. It is explanatory drawing which shows an example of the description of the metadata provided by the metadata provision apparatus of 3rd embodiment of this invention.

Explanation of symbols

100 Metadata giving device 101 CPU
102 ROM
103 RAM
110 Control unit 170 Operation unit 180 Display unit 190 HDD

Claims (14)

Display means for displaying an image including a subject according to image data;
An adding unit for inputting metadata corresponding to a selected subject of the image displayed on the display unit;
Metadata recording means for associating the metadata input from the assigning means with the image data and recording it on a recording medium;
A metadata providing apparatus comprising: The display means displays the displayed subject candidates as metadata options,
The assigning unit is configured to select a subject type candidate included in the image displayed on the display unit from the options, and to display an image associated with the option when a subject type is selected from the options. It is possible to accept input of the position or size of
The metadata recording unit records the selected option as metadata, acquires the position or size of the subject on the image designated by the adding unit, and records it as metadata.
The metadata providing apparatus according to claim 1. The display means displays the displayed subject candidates as metadata options,
The adding unit is configured to select a subject type candidate and a subject part candidate included in the image displayed on the display unit from the options, and when a subject or a subject part is selected from the options. It is configured to be able to accept an input of a position or size on an image associated with the option,
The metadata recording unit records the selected option as metadata, acquires the position or size of the subject on the image designated by the adding unit, and records it as metadata.
The metadata providing apparatus according to claim 1. The metadata recording means records that the object could not be acquired when the position or size of the subject could not be acquired;
The metadata providing apparatus according to claim 2 or claim 3, wherein The display means is configured to be able to display a candidate for a reason when the position or size of the subject cannot be acquired,
The granting unit is configured to be able to accept an input of a candidate for a reason displayed on the display unit.
The metadata providing apparatus according to claim 4. In addition to the image being displayed, the display means displays a subject model of the same type as the subject type selected or input by the assigning means,
The assigning means is configured to be able to change an angle on the display screen of the subject model,
The metadata recording means acquires the angle of the subject from the subject model changed by the giving means and records it as metadata;
The metadata providing apparatus according to any one of claims 1 to 5, wherein The metadata recording means records that the information could not be acquired when the angle of the subject cannot be acquired.
The metadata providing apparatus according to claim 6. The display means is configured to be able to display a candidate for a reason when the angle of the subject cannot be acquired,
The granting unit is configured to be able to accept an input of a candidate for a reason displayed on the display unit.
8. The metadata providing apparatus according to claim 7, wherein: The giving means is configured to be able to accept input of an arbitrary character string.
The metadata providing apparatus according to any one of claims 1 to 8, wherein The display means displays composition candidates of the displayed image as metadata options,
The assigning means is configured such that a composition candidate of an image displayed on the display means can be selected from the options,
The metadata adding apparatus according to claim 1, wherein the metadata recording unit records the selected option as metadata. The test image data is held, the test image data is displayed on the display means, and the operation of receiving the metadata input by the adding means is repeatedly executed a plurality of times, and the metadata input a plurality of times When the difference between the first input value and the converged input value is extracted as a correction value, normal image data is displayed on the display means, and the metadata is input to the assigning means, the meta data Control means for correcting data by the correction value;
11. The metadata providing apparatus according to claim 1, wherein Automatic recognition means for automatically recognizing input image data and extracting metadata corresponding to the image or subject;
An evaluation unit that compares the metadata input from the assigning unit with the metadata extracted by the automatic recognition unit, and evaluates the automatic recognition;
11. The metadata providing apparatus according to claim 1, wherein The metadata recording means includes at least one of the metadata input from the assigning means or the metadata extracted by the automatic recognition means, and the comparison result data compared by the comparison means. Record to a recording medium in association with
The metadata providing apparatus according to claim 12, wherein: Using the metadata providing apparatus according to any one of claims 1 to 13,
Display an image including the subject according to the image data,
Accepts metadata input according to the selected subject in the displayed image,
Recording the input metadata on a recording medium in association with the image data;
A method for providing metadata, comprising: