JP2006314010A - Image processing apparatus and image processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily grasp who is photographed in a plurality of image data and to easily extract only image data that a user desires to appreciate. <P>SOLUTION: Face information showing features of the face of a person recognized in the plurality of image is acquired and image data having the same face information are grouped to form processing unit groups, and representative images set by the processing unit groups are displayed as thumbnails. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image processing method for classifying image data for each face information.

  In recent years, digital cameras and camera-equipped mobile phones have become widespread, and captured images are stored in a memory or the like as digital image data. Therefore, there is no need to worry about the remaining amount of film unlike a conventional film type camera, and a large number of images can be easily taken.

Digital image data shot in this way is not only printed and viewed, but also stored on a personal computer such as a personal computer hard disk or PDA, so that the captured image can be displayed on a monitor for easy viewing. be able to. In addition, it is used such that an electronic album is created by recording on a recording medium such as a CD-R or DVD-R, and a photograph is displayed on a computer, a TV monitor, or the like (for example, see Patent Document 1).
JP 2000-293974 A

  Here, for example, considering the case of viewing images taken when traveling with multiple friends, it takes time for a user to view all the images taken, so for the time being, an image of himself It is natural to think that it is only necessary to see images that show the scenery of the travel destination.

  That is, the user does not have much interest even when an image showing only a person other than himself / herself is displayed on the monitor screen, and therefore, the user performs an operation of skipping the currently displayed image to immediately display the next image. it is conceivable that. However, it is very troublesome to perform the skip operation each time.

  In addition to appreciating images, for example, if you want to select and print only photos that you have taken or images of your travel destination, you will have many images. It was troublesome for the user to search only the image desired to be printed from among the images.

  The present invention has been made in view of the above points, and the object of the present invention is to easily grasp who is photographed in a plurality of image data and to obtain only image data that the user desires to view. The purpose is to enable easy extraction.

  In order to achieve the above object, in the present invention, face information of a person included in a plurality of image data is acquired, and the image data is classified based on the face information.

That is, the invention of claim 1 is an image display means for displaying a plurality of image data including a still image;
Image recognition means for recognizing a person's face in the plurality of image data and acquiring face information indicating features of the recognized face;
Grouping means for grouping image data having the same face information among the plurality of image data, and creating a processing unit group for each face information;
Representative image setting means for setting one of the image data included in the processing unit group as a representative image;
And representative image display means for displaying the representative image set for each processing unit group as a thumbnail on the image display means.

  Therefore, according to the present invention, face information indicating the facial features of a person recognized in a plurality of image data is acquired, and image data having the same face information is grouped as a processing unit group. The representative image set to is displayed as a thumbnail on the image display means.

According to a second aspect of the present invention, in the image processing apparatus according to the first aspect,
The representative image display means is configured to display only one overlapping representative image as a thumbnail on the image display means when the same image data is set as a representative image of different processing unit groups. It is characterized by.

  Therefore, according to the present invention, when the same image data is set as a representative image of different processing unit groups, only one representative image that overlaps is displayed on the image display means.

The invention of claim 3 is the image processing apparatus according to claim 1 or 2,
In the case where a plurality of human faces are present in the representative image, an identification display means for displaying the plurality of human faces in a display format that can be distinguished from each other is further provided.

  Therefore, according to the present invention, the faces of a plurality of persons in the representative image are displayed in a display format that can be distinguished from each other.

The invention according to claim 4 is the image processing apparatus according to claim 1,
The representative image setting means is configured to extract an image range including a human face in the image data included in the processing unit group and set the image range as the representative image. To do.

  Therefore, according to the present invention, an image range including a human face in the image data is extracted, and this image range is set as a representative image.

According to a fifth aspect of the present invention, in the image processing device according to any one of the first to fourth aspects,
The image recognition means is configured to perform a recognition operation only on a human face having a predetermined size in the image data.

  Therefore, according to the present invention, the face recognition operation is performed only on the face of a person having a predetermined size in the image data.

The invention according to claim 6 is the image processing apparatus according to any one of claims 1 to 5,
The image processing apparatus further includes a sorting unit that rearranges a display order of the plurality of representative images displayed as thumbnails on the image display unit based on the number of image data included in the processing unit group.

  Therefore, according to the present invention, the display order of the plurality of representative images is rearranged based on the number of image data included in the processing unit group.

The invention of claim 7 is the image processing apparatus according to any one of claims 1 to 6,
When the user selects one of the representative images displayed as thumbnails on the image display means, the image data included in the processing unit group related to the selected representative image is displayed on the image display means. It is characterized by becoming.

  Therefore, according to the present invention, when one of the representative images displayed as thumbnails is selected, the image data included in the processing unit group related to the selected representative image is displayed.

The invention of claim 8 is the image processing apparatus according to any one of claims 1 to 7,
It further comprises a printing means for printing the image data,
When the user selects one of the representative images displayed as thumbnails on the image display means, the image data included in the processing unit group related to the selected representative image is printed by the printing means. It is characterized by.

  Therefore, according to the present invention, when one of the representative images displayed as thumbnails is selected, the image data included in the processing unit group related to the selected representative image is printed.

The invention of claim 9 recognizes a person's face in a plurality of image data, and obtains face information indicating features of the recognized face;
Grouping image data having the same face information among the plurality of image data, and creating a processing unit group for each face information;
A procedure for setting one of the image data included in the processing unit group as a representative image;
And a procedure for displaying the representative image set for each processing unit group as a thumbnail.

  As described above, according to the invention of claim 1, a processing unit group is created by grouping image data having the same face information, and one of the image data included in the processing unit group is used as a representative image. Since this representative image is displayed as a thumbnail on the image display means, it is possible to easily grasp what kind of person is photographed in all the image data. For example, image data when traveling with a plurality of friends is preferable because it is possible to easily recall the members who traveled together at that time simply by looking at the representative images displayed as thumbnails.

  According to the second aspect of the present invention, when overlapping image data is set as the representative image, only one of them is displayed as a thumbnail. It is possible to display with the minimum necessary number of displays.

  Specifically, for example, when there is image data in which Mr. A and Mr. B are captured by two people, the images captured by these two people as representative images of the processing unit group of both Mr. A and Mr. B. The case where data is set is considered. In such a case, it is not necessary to display both thumbnails of the same image data, and if only one image is displayed as a thumbnail, the number of persons included in all the image data with the minimum required number of displays is displayed. Etc. are preferable.

  According to the invention according to claim 3, since the faces of a plurality of persons in the representative image are displayed in a display format that can be distinguished from each other, it is possible to easily grasp which person is displayed in the representative image. it can. Specifically, for example, if an image range including a person's face is surrounded by a frame, or if the image range other than the person's face is changed to an inconspicuous color such as gray, the person's face in the representative image This is preferable because it is easy to grasp the position of the person and who the person is.

  According to the fourth aspect of the present invention, the image range including the human face in the image data is extracted and the image range is set as the representative image. For example, the group photo is set as the representative image. Sometimes, the face of the person in the photo is too small to be seen, but if you extract an image range that includes the face of a specific person from the group photo, an image with that person's face enlarged is displayed as the representative image. It can be set, and it is possible to easily grasp what kind of person is photographed in all the image data.

  According to the invention of claim 5, since the recognition operation is performed only on the face of a person having a predetermined size in the image data, it is advantageous in recognizing only the face of the person traveling together. .

  Specifically, if a person passing by or a person in the distance is captured in the captured image data, and the face recognition operation is performed including those persons, a processing unit group for the person you do not know In order to prevent this, the face recognition operation is not performed on a person who is captured in a predetermined size or less in the image data.

  According to the sixth aspect of the present invention, the display order of the representative images displayed as thumbnails on the image display means is rearranged based on the number of image data included in the processing unit group. If a representative image of a person who frequently appears in the image data is displayed preferentially, it is preferable that the images are displayed in order from the central person in the trip. Further, even if a representative image of a passing person who is not related to the traveled member is displayed, the representative image is displayed in the lower order, so that there is less hindrance to image viewing.

  According to the invention of claim 7, when the user selects one of the representative images displayed as thumbnails, the image data included in the processing unit group related to the representative image is displayed on the image display means. The user does not have to view all the image data when viewing the image data, for example, by selecting the representative image that is captured from the thumbnail displayed representative images. It is preferable that only can be appreciated efficiently.

  According to the eighth aspect of the present invention, when the user selects one of the representative images displayed as thumbnails, the image data included in the processing unit group related to the representative image is printed. Therefore, it is preferable that the user can efficiently print only the image data of the user by selecting, for example, the representative image of himself / herself from the representative images displayed as thumbnails.

  According to the ninth aspect of the present invention, it is possible to easily grasp what kind of person is photographed in all the image data. For example, image data when traveling with a plurality of friends is preferable because it is possible to easily recall the members who have traveled at that time simply by looking at the representative images displayed as thumbnails.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The following description of the preferred embodiments is merely exemplary in nature and is in no way intended to limit the invention, its application, or its application.

<Device configuration>
FIG. 1 is a block diagram showing the overall configuration of an image processing apparatus 100 according to an embodiment of the present invention. As shown in FIG. 1, 10 is a drive unit, 11 is an image processing unit for performing image processing on image data read by the drive unit 10, 12 is an information input unit for inputting information to the image processing unit 11, and 13 is A monitor (image display means) for displaying image data, various menus of image processing software, and the like, and a print unit 14 for printing image data.

  The drive unit 10 acquires (reads) data from a semiconductor medium Ms such as a flash memory in which image data including photographs and moving images taken by a digital camera is stored, and a CD-R or DVD-R. And a disk drive 2 that enables acquisition and writing of data from a plurality of types of large-capacity disk-shaped media Md. The semiconductor drive 1 preferably includes a plurality of insertion portions so as to be compatible with the types of flash memories such as CompactFlash (registered trademark), SmartMedia (registered trademark), and Memory Stick (registered trademark).

  The image processing unit 11 is configured by a computer including a CPU that performs information processing, a memory, and a hard disk. The image processing unit 11 stores image data read by the drive unit 10 in the hard disk and uses the installed image processing software to store the image data. Various image processing can be performed on the image.

  Specifically, image recognition means for recognizing a person's face in the image data and acquiring face information indicating the characteristics of the person's face (for example, the size and position of the eyes and nose relative to the size of the entire face) Grouping means for grouping image data having the same face information to create a processing unit group, representative image setting means for setting one of the image data included in the processing unit group as a representative image, and for each processing unit group A representative image display means for displaying the set representative image as a thumbnail on the monitor 13; when there are a plurality of human faces in the representative image, a display format (for example, the Identification display means for displaying an image range including a face in a frame frame, etc., and a multiple display for displaying thumbnails on the monitor 13 based on the number of image data included in the processing unit group. Has such a sort sorting means the display order of the representative image as a software (program).

  The information input unit 12 includes, for example, a keyboard, a mouse, a remote controller, and the like, and can perform necessary inputs to the image processing unit 11. The input may be performed using the touch panel type monitor 13.

  The monitor 13 is composed of, for example, a CRT or a liquid crystal display device, and displays image data input to the image processing unit 11 and displays various menu screens of image processing software so that the user can check. It is like that.

  The print unit 14 can print the image data by selecting the image data that the user desires to print on the monitor 13 screen and instructing the print.

  In the present embodiment, an example in which a computer is used as an example of the image processing apparatus 100 has been described. However, the present invention is not limited to this embodiment. For example, the face information is recognized by recognizing a human face in the image data. It is also possible to use a playback device (for example, a DVD player) provided with software as a procedure for acquiring image data, classifying image data based on face information, and displaying the representative image as a thumbnail. Further, a program for executing such a procedure may be recorded in an electronic album together with image data, and the image data may be processed based on this program when played back by a DVD player or the like.

<Image processing procedure>
Next, a procedure for classifying image data based on the face information of a person in the image data in the image processing apparatus 100 according to the present embodiment will be described with reference to the flowchart of FIG. As shown in FIG. 2, in step S101, image data captured by a digital camera or the like is input to the image processing unit 11, and the process proceeds to subsequent step S102.

  In step S102, for each of a plurality of image data input to the image processing unit 11, it is determined whether or not a human face is included in the image data.

  If the determination in step S102 is “YES”, the process branches to step S103, and in step S103, the face of the person in the image data is recognized, and facial information indicating the characteristics of the person's face is acquired. Proceed to step S104. If the determination in step S102 is “NO”, the image data is determined to be a landscape image, and the process proceeds to subsequent step S104.

  In step S104, it is determined whether or not human face recognition has been performed for all input image data. If “NO” at step S104, the process branches to step S102, and the same processing is repeated for the next image data.

  If the determination in step S104 is “YES”, the process branches to step S105. In step S105, image data having the same face information is grouped to create a processing unit group, and the process proceeds to step S106.

  In step S106, it is determined whether to add a landscape image to the processing unit group. If “YES” at step S106, the process branches to step S107. At step S107, the landscape image is added to the processing unit group, and the process proceeds to step S108. If “NO” at step S106, the process branches directly to step S108.

  In this way, instead of grouping only image data including a person into a group, the processing unit group includes the image data of the scenery taken at the time of the trip, so that when the processing unit group is reproduced and displayed, the person is included. In addition to image data, a landscape image at the time of travel can also be appreciated, which is advantageous for recalling the scene at the time of travel.

  In step S108, among the image data included in the processing unit group, an image that is considered suitable for representing a person included in the group is set as a representative image, and the process proceeds to subsequent step S109.

  In step S109, the number of image data included in each processing unit group is counted, and the process proceeds to subsequent step S110.

  In step S110, a representative image for each group is displayed as a thumbnail on an image selection menu screen described later displayed on the monitor 13. At this time, the representative images of the group with the large number of image data counted earlier are rearranged and displayed by the sort program of the image processing software so that they are positioned at the top of the menu screen. In this way, it is preferable that the representative image showing the person with the highest appearance frequency among all the image data is the top.

  By performing the processing as described above, the total number of people included in multiple image data can be easily grasped, and who is in the image can be easily grasped with a thumbnail image. It is advantageous in recalling the member composition at that time.

<Display format of representative image>
Next, an example of the display format of the representative image displayed as a thumbnail on the monitor 13 will be described with reference to FIGS. FIG. 3 shows a case where five image data No. 1 to No. 5 are input to the image processing apparatus 100. As shown in FIG. 3, No. 1 is an image of Mr. A, No. 2 is a landscape image 1, No. 3 is an image of Mr. B, No. 4 is two people of A and B It is assumed that the image No. 5 shown in FIG.

  Here, in step S105 in the flowchart of FIG. 2 described above, image data having the same face information is grouped to create a processing unit group. Specifically, the processing unit group of Mr. A is No. .1 and No. 4 images are included, and Mr. B's processing unit group includes No. 3 and No. 4 images. In the following description, it is assumed that the landscape image is not grouped in the subsequent step S106.

  Next, in step S108, a representative image is set for each processing unit group. Here, the No. 1 image is set as the representative image of the group A, and the No. 1 image is set as the representative image of the group B. Assume that three images are set. As a result, as shown in FIG. 4, when an image selection menu described later is displayed, the No. 1 and No. 3 images set earlier are displayed as thumbnails.

  In addition, when displaying the persons (two persons A and B in FIG. 3) included in a plurality of input image data as the minimum number of representative images, the representative images of A and B are respectively No. .4 images are set, and as shown in FIG. 5, in order to identify each face in the image data, each frame frame surrounds an image range including Mr. A and Mr. B faces. 21 is displayed. Thereby, it becomes possible to easily grasp that Mr. A and Mr. B are reflected in the No. 4 image, which is preferable.

  Next, another display format of the representative image will be described with reference to FIGS. FIG. 6 illustrates a case where five image data No. 1 to No. 5 are input to the image processing apparatus 100. As shown in FIG. 6, No. 1 is an image in which A and B are reflected by two people, No. 2 is a landscape image 1, No. 3 is an image of one B, and No. 4 is C. Suppose that No. 5 is an image of Mr. B and Mr. C in two people.

  Here, Mr. A's processing unit group includes the No. 1 image, Mr. B's processing unit group includes the No. 1, No. 3 and No. 5 images, and Mr. C's processing. The unit group includes No. 4 and No. 5 images.

  Then, the No. 1 image is set as the representative image of the group of A, the No. 1 image is set as the representative image of the group of B, and the No. 4 image is set as the representative image of the group of C. It shall be set. As a result, when the image selection menu is displayed as shown in FIG. 7, the images No. 1 and No. 4 set earlier are displayed as thumbnails. Since the representative images of Mr. A and Mr. B are the same No. 1 images, only one duplicate No. 1 image is displayed.

  As shown in FIG. 7, in the No. 1 image, two people, Mr. A and Mr. B, are shown, so in order to identify each face, an image range including the faces of Mr. A and Mr. B is shown. Frame frames 21 are displayed so as to surround each frame. Thereby, it is possible to easily grasp that Mr. A and Mr. B are reflected in the No. 1 image.

  Note that the frame frame 21 is displayed as means for identifying the faces of a plurality of persons in the image data. In addition to this, for example, as shown in FIG. It is preferable to display a region other than the image range including the image with a color that is not easily noticeable, such as gray.

  Next, another display format of the representative image will be described with reference to FIGS. 9 and 10. FIG. 9 shows a case where five image data No. 1 to No. 5 are input to the image processing apparatus 100. As shown in FIG. 9, No. 1 is an image where A and B are reflected by two people, No. 2 is an image where A, B and C are reflected by three people, No. 3 Is an image of Mr. B, No. 4 is an image in which Mr. A and Mr. B are captured by two people, and No. 5 is a landscape image 1.

  Here, Mr. A's processing unit group includes No. 1, No. 2 and No. 4 images, and Mr. B's processing unit group includes No. 1, No. 2, No. 3 and The No. 4 image is included, and Mr. C's processing unit group includes the No. 2 image.

  Next, a representative image is set for each processing unit group. Here, an image range including a person's face in image data included in the group is extracted, and this image range is set as a representative image. I am doing so.

  Specifically, in Mr. A's processing unit group, an image range including Mr. A's face in the No. 1 image is extracted, and an image in which this image range is enlarged is set as a representative image. Yes. Similarly, in Mr. B's processing unit group, an image range including Mr. B's face in the No. 1 image is set as a representative image, and in Mr. C's processing unit group, Mr. C in the No. 2 image. An image range including the face is set as a representative image.

  As a result, as shown in FIG. 10, when the image selection menu is displayed, the representative images of Mr. A, Mr. B and Mr. C set earlier are displayed as thumbnails.

  In this way, even if the image data is a group photo and the person's face is shown small, the image range including the person's face is enlarged on the menu screen, so the thumbnail is displayed. It is possible to easily grasp who is reflected just by looking at the representative image. In addition, since the representative image with the face expanded for each person in the entire image data is displayed as a thumbnail, it is possible to easily grasp how many persons are included in the entire image data.

<Image playback procedure>
Next, a procedure for allowing the user to view a plurality of image data input to the image processing apparatus 100 will be described. The above-described human face recognition and representative image setting operations in the image data are automatically performed when the image data is input to the image processing apparatus 100, and the user inputs the image data. When completed, the image selection menu is activated from the menu screen of the electronic album.

  In the image selection menu displayed on the monitor 13, as shown in FIG. 10, for example, as shown in FIG. 10, representative images showing persons appearing in all input image data are displayed as thumbnails. By selecting any one of the representative images, only the image data included in the processing unit group related to the representative image is displayed on the monitor 13.

  For example, when the representative image of Mr. A is selected in the image selection menu of FIG. 10, the images of No. 1, No. 2, and No. 4 included in the processing unit group of Mr. A shown in FIG. 13 is displayed as a thumbnail or displayed as a slide show one by one.

  As a result, the user does not need to view all the image data recorded in the electronic album, and can selectively view, for example, only the image in which the user is captured. That is, the user-specific electronic album can be reproduced, and the convenience for the user is enhanced.

  In addition to appreciating images, it is also possible to use only printing an image of the user himself / herself by linking with the print unit 14. That is, by selecting one of the representative images in the image selection menu and displaying the image data included in the processing unit group related to the representative image on the monitor 13, these image data are transmitted to the printing unit 14. Only the image data included in the processing unit group can be printed.

  As a result, the user selects only the image in which he / she appears from all the image data (in the embodiment, an example of selecting all of the processing unit groups has been described, but the present invention is not limited to this mode. Furthermore, even if the configuration is such that an arbitrary image is selected, it does not deviate from the technical idea of the present invention.) When it is desired to print, all the image data is checked and the desired image data is sequentially selected. This eliminates the need for such a cumbersome work and improves user convenience.

  As described above, according to the image processing apparatus 100 according to the present embodiment, it is possible to select a person in the image data without performing troublesome operations such as selecting only the image data that the user wants to view from a plurality of image data. Convenience can be improved by displaying based on face information.

  In addition, if the present invention is used when creating an electronic album, it is possible to create an electronic album in which only members' images are recorded for each member who traveled together. Can be created.

  As described above, the present invention provides a highly practical effect of easily grasping who is photographed in a plurality of image data and easily extracting only the image data that the user desires to view. Therefore, it is extremely useful and has high industrial applicability.

1 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. It is a flowchart figure which shows the image processing procedure which concerns on this embodiment. It is a figure explaining the creation procedure of a processing unit group. It is a figure which shows the display format of the representative image displayed on an image selection menu. It is a figure which shows another display format of the representative image displayed on an image selection menu. It is a figure explaining another example of the creation procedure of a processing unit group. It is a figure which shows another display format of the representative image displayed on an image selection menu. It is a figure which shows another display format of the representative image displayed on an image selection menu. It is a figure explaining another example of the creation procedure of a processing unit group. It is a figure which shows another example of an image selection menu.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Drive unit 11 Image processing part 12 Information input part 13 Monitor 14 Print part 100 Image processing apparatus

Claims (9)

Image display means for displaying a plurality of image data including still images;
Image recognition means for recognizing a person's face in the plurality of image data and acquiring face information indicating features of the recognized face;
Grouping means for grouping image data having the same face information among the plurality of image data, and creating a processing unit group for each face information;
Representative image setting means for setting one of the image data included in the processing unit group as a representative image;
An image processing apparatus comprising: representative image display means for displaying the representative image set for each processing unit group as a thumbnail on the image display means. The image processing apparatus according to claim 1,
The representative image display means is configured to display only one overlapping representative image as a thumbnail on the image display means when the same image data is set as a representative image of different processing unit groups. An image processing apparatus. In the image processing apparatus according to claim 1 or 2,
An image processing apparatus, further comprising: an identification display unit configured to display a plurality of human faces in a display format that can be distinguished from each other when the representative image includes a plurality of human faces. The image processing apparatus according to claim 1,
The representative image setting means is configured to extract an image range including a human face in the image data included in the processing unit group and set the image range as the representative image. An image processing apparatus. The image processing apparatus according to any one of claims 1 to 4,
The image processing apparatus, wherein the image recognition means is configured to perform recognition work only on a person's face having a predetermined size in the image data. The image processing apparatus according to any one of claims 1 to 5,
An image processing apparatus, further comprising: a sorting unit that rearranges a display order of the plurality of representative images displayed as thumbnails on the image display unit based on a number of image data included in the processing unit group. The image processing apparatus according to any one of claims 1 to 6,
When the user selects one of the representative images displayed as thumbnails on the image display means, the image data included in the processing unit group related to the selected representative image is displayed on the image display means. An image processing apparatus characterized by comprising: The image processing apparatus according to any one of claims 1 to 7,
It further comprises a printing means for printing the image data,
When the user selects one of the representative images displayed as thumbnails on the image display means, the image data included in the processing unit group related to the selected representative image is printed by the printing means. An image processing apparatus. A procedure for recognizing a person's face in a plurality of image data and acquiring face information indicating the characteristics of the recognized face;
Grouping image data having the same face information among the plurality of image data, and creating a processing unit group for each face information;
A procedure for setting one of the image data included in the processing unit group as a representative image;
And a procedure for displaying the representative image set for each processing unit group as a thumbnail.

Images