JP2007156729A - Retrieval device and retrieval method and camera having retrieval device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a retrieval device and a retrieval method for performing efficient image retrieval by a simple input operation and a camera having the retrieval device. <P>SOLUTION: In a retrieval device, factor information having at least one of image attributes, temporal attributes and positional attributes is derived by an image information part 16 under conditions owned by at least one character and word. Then, at least one image is retrieved from among a plurality of images according to the factor information derived by the image information part 16 by a microcontroller 10. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a search device, a search method, and a camera having the search device, and more particularly, to a search device and a search method technique for searching for a desired image from images taken by a camera taken many times. is there.

  In recent years, the use of small digital cameras and the mounting of cameras on mobile phones and portable music players have increased the number of images that people take every day. Under these circumstances, even when trying to view the captured images later, if there are a large number of captured images, it takes time to find them and often gives up. In addition, Internet site management companies and the like have recently emphasized technology for searching for images and music.

In view of this, a technique for searching by attaching an emotional keyword to an image and a technique for attaching semantic information have been proposed (see, for example, Patent Document 1 and Patent Document 2). Furthermore, a proposal for searching for an image by text is also known (see, for example, Patent Document 3).
JP 2004-5534 A JP 2004-102792 A JP-A-8-83287

  However, the techniques described in Patent Document 1 to Patent Document 3 described above depend on human senses and sensibilities, require a huge database such as dictionary information, and input text such as inputting text. It took a lot of load.

  Accordingly, an object of the present invention is to provide a search device, a search method, and a camera having a search device that can perform an efficient image search by a simple input operation.

  That is, the invention according to claim 1 derives factor information having an image attribute, a temporal attribute, or a positional attribute from the meaning of a plurality of characters or words, and outputs the factor information from the derivation means. And selecting means for selecting at least one image from a plurality of images according to each factor information derived by the image attribute, the time attribute, or the position attribute.

  The invention according to claim 2 is the invention according to claim 1, wherein the image attributes derived from the plurality of characters or words include brightness information, brightness distribution information, color information, color distribution information, Etc. are included.

  According to a third aspect of the present invention, in the first aspect of the present invention, the temporal attribute derived from the plurality of characters or words includes factors such as seasonal information and morning and evening information. It is characterized by.

  According to a fourth aspect of the present invention, in the first aspect of the present invention, the positional attribute derived from the plurality of characters or words includes a factor of latitude / longitude information or indoor / outdoor information. It is characterized by that.

  The invention according to claim 5 further comprises display means for displaying time information and position information of the searched image together with the searched image together with the searched image in the invention of claim 1. It is characterized by that.

  The invention described in claim 6 is characterized in that, in the invention described in claim 1, the image selection is selected from a plurality of displayed characters for search.

  The invention according to claim 7 is the invention according to claim 1, wherein the deriving means derives based on characteristic information of each factor in a plurality of attributes associated with each other according to the character. It is characterized by doing.

  The invention according to claim 8 is the invention according to claim 1, further comprising display means for collectively displaying the images selected by the selection means as a group.

  The invention according to claim 9 is the invention according to claim 1, wherein the selecting means selects a specific image in accordance with information on the frequency of photographing of the photographs taken before and after that and information on changes in the photographing points. It is characterized by that.

  The invention according to claim 10 is derived by the deriving means for deriving factor information having at least one of an image attribute, a temporal attribute and a positional attribute from the condition of at least one character and word, and the deriving means. Search means for searching for at least one image from a plurality of images according to the factor information.

  The invention according to claim 11 is the invention according to claim 10, wherein the image attribute includes at least one of factors of light / dark information, light / dark distribution information, color information, and color distribution information. It is characterized by that.

  The invention described in claim 12 is the invention described in claim 10, wherein the temporal attribute includes at least one factor of season information and morning / day / night information.

  The invention according to claim 13 is the invention according to claim 10, wherein the positional attribute includes at least one factor of latitude / longitude information and indoor / outdoor information. .

  The invention described in claim 14 is the invention described in claim 10, further comprising display means for displaying at least one image searched by the search means.

  According to a fifteenth aspect of the invention, in the invention according to the fourteenth aspect, when there are a plurality of at least one image searched by the search means, a desired one is obtained from the plurality of images displayed on the display means. The image processing apparatus further includes selection means for selecting the image.

  The invention according to claim 16 is the invention according to claim 15, wherein the display means displays information contained in the searched image together with the image searched by the searching means. To do.

  The invention according to claim 17 is the invention according to claim 16, wherein the display means collectively displays the images selected by the selection means as a group.

  According to an eighteenth aspect of the present invention, in the invention according to the sixty-sixth aspect, the selection means performs selection based on information included in an image displayed on the display means.

  The invention according to claim 19 is the invention according to claim 16, wherein the selecting means selects a specific image in accordance with information on the frequency of photographing and the change in photographing location of the photographs taken before and after that. It is characterized by that.

  The invention according to claim 20 is the invention according to claim 10, wherein the deriving means derives based on characteristic information of each factor in a plurality of attributes associated according to an image. It is characterized by that.

  A twenty-first aspect of the invention includes the search device according to the first to twentieth aspects.

  The invention according to claim 22 includes a first step of deriving a factor having an image attribute and a temporal attribute or a positional attribute from the meanings of a plurality of characters or words, and the first step, And a second step of selecting at least one image from a plurality of images according to each factor information derived by the image attribute, the time attribute, or the position attribute.

  According to the present invention, it is possible to provide a search device, a search method, and a camera having a search device that can perform an efficient image search by a simple input operation.

  In addition, according to the present invention, an appropriate image can be selected by using abundant information by simple input, so that the user can easily find what he / she wants to see from old photos when he / she likes. You can enjoy watching.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of a search device according to an embodiment of the present invention.

  In FIG. 1, a microcontroller 10 corresponds to the derivation means and controls the overall control operation of the search device. The microcontroller 10 includes a character input unit 11, a selection unit 12, a display unit 13, an operation unit 14, a memory 15 having an image information unit 16 and an image attachment information unit 17, and a light / dark detection unit 20a. The color detection unit 20b, the person detection unit 20c, and the information generation unit 21 connected to the database (DB) 22 are connected to each other.

  The character input unit 11 is for inputting a keyword or the like of an image that the user wants to see. If the user's memory is very certain, for example, “the image when going to XX on XX ", It is possible to easily search for data. However, in general, the situation that the user wants to see without specifying the date or place, such as what it was like when you traveled before or how Mr. A's wedding was I often remember only vague images. The image may be input in a word-like form from the keyboard. However, in this embodiment, a scene with a large number of shooting opportunities is expressed by several characters and the image is input without preparing a keyboard or the like. To be able to.

  The display unit 13 displays the analysis result of each image and attached information according to the input result of the character input unit 11. The selection unit 12 is for further narrowing down the information output and displayed on the display unit 13. The operation unit 14 is for performing operations such as inputting information such as a shooting date, which will be described later.

  A large number of image data is recorded in the image information section 16 in the memory 15. This image data is selected by the microcontroller 10 from the image information section 16 as desired. On the other hand, the image attachment information section 17 records information attached to the image such as time information and location information.

  The brightness / darkness detection unit 20a and the color detection unit 20b are provided to extract the information because each image recorded in the image information unit 16 has light / dark information, distribution, and color information. . In other words, the light / dark detection unit 20a detects light / dark information, light / dark distribution information, and the like of each image recorded in the image information unit 16, and the color detection unit 20b detects each image recorded in the image information unit 16. It detects the color information, color distribution information, etc. of the image. Further, since a method for detecting the number of persons by detecting a person's face or the like from image data is also known, a person detection unit 20c is provided using such a technique and stored in the image information unit 16. A person is detected from each image.

  The information generating unit 21 is for searching for information by converting the information recorded in the database (DB) 22 into information, and also converts information representing the photographing frequency into search information. Have a role.

  FIG. 2 is a block diagram illustrating a configuration of a camera to which the search device having the configuration illustrated in FIG. 1 is applied.

  In FIG. 2, this camera includes a search device 30 that is a feature of the present invention, a GPS unit 31 that obtains position information of a current location by positioning using a global positioning system (GPS) using satellites, and clock information. It has a built-in clock part 32. The search device 30 is connected to a sequence controller (MPU) 33, an image processing engine 35, a memory 15 having an image information unit 16 and an image attachment information unit 17, and a database 22. The sequence controller 33 is also connected to an operation switch 33a, an analog processing circuit (AFE) 38, and a flash light emitting device 40.

  The analog processing circuit 38 receives image data in which a subject image incident from the photographing lens 36 is formed on the image sensor 37 and converted into an electrical signal. The image data signal input to the analog processing circuit 38 is supplied to the image processing engine 35 and converted into a compressed digital signal. This conversion result is recorded in the memory 15. At this time, if necessary, the flash light emitting device 40 is controlled to emit light, and the shooting date / time information determined by the clock unit 32 and the information of the GPS unit 31 are also attached to the image attachment information in the memory 15 via the sequence controller 33. The image is recorded in the unit 17 in association with the image in the image information unit 16. The database 22 is assumed to be built into the camera. Since the map information and the like is a huge amount of data for each figure, it may be exchangeable for each region.

  For example, in the case of a camera as shown in FIG. 3A, two memory slots are provided on the side surface of the camera 45, and image or data for attached data is input to one memory 15. So that The other memory 22 may be read-only so that only the database is stored.

  In FIG. 3A, a photographing lens 36 and a flash light emitting device 40 are provided on the front surface of the camera 45, and a release switch (operation switch) 33a is provided on the upper surface.

  Further, as shown in FIG. 3B, a large display LCD 13 a configured as the display unit 13 and a cross key 12 a configured as the selection unit 12 are disposed on the back surface of the camera 45. .

  The present invention is a retrieval apparatus that can easily retrieve and display an image to be viewed when viewing a large number of image data recorded by photography or the like later. The image searched here is assumed to be attached with other accompanying information in addition to image recording for shooting or the like according to the flowchart of FIG.

  Hereinafter, with reference to the flowchart of FIG. 4, the photographing operation by the camera having the configuration of FIG. 2 will be described.

First, in step S <b> 1, an image of a subject (not shown) is captured by the image sensor 37 and recorded in the memory 15. At the same time, in steps S2 to S4, time information I _P , GPS information I _G, and mode information I _M are recorded in the same memory in association with the captured image. However, the information is not limited to these, and some information may be omitted.

The time information I _P uses the time of photographing by the clock of the clock unit 32 built in the camera as information. Also, when the GPS information I _G is present, the date and time difference can also be into account the time. Further, on / off and zooming results of the flash light emitting device 40 set automatically or manually at the time of shooting, the distance at the time of shooting, macro mode information, and the like are recorded in association with the image.

Since the information attached to such an image is also recorded in the image attachment information section 17, in the present invention, the image is narrowed down by using these effectively. Further, simply GPS information I _G, because it can not to the location of a particular, the information generating unit 21, information of searching for location information in consideration of the database information. In addition, the amount of time information is small if only one image is attached to each image, but the time when the images recorded adjacent to each other are adjacent to each other, or a unit time such as one day or one hour. If up to the number of shots is used as information at the time of retrieval, the information becomes much more effective in retrieval.

  The microcontroller 10 configured as described above analyzes each image and attached information according to the input result of the character input unit 11, and outputs and displays the result on the display unit 13. The selection unit 12 is used for the display method and further narrowing down.

  The table shown in FIG. 5 illustrates how the image detection with such a configuration narrows down the images to be displayed.

  FIG. 5 is a table showing correspondence between image attributes, time attributes, and position attributes for each character. In the example of FIG. 5, four character examples are shown on the vertical axis. Here, four scenes of “travel”, “wedding”, “entrance ceremony”, and “athletic day” are listed as the scenes of photographs often taken by general camera users, and are displayed together with the character design. I am doing so. Also, on the horizontal axis, the brightness level of the image obtained from the image information unit 16, the characteristics of the light / dark distribution, the color, the characteristics of the color distribution and the presence / absence of a person, the time attribute obtained from the image attachment information unit 17, and the imaged For items such as position attributes, a list of the characteristics of each shooting scene is displayed. For example, the image characteristics are “brightness”, “lightness distribution”, “color”, “color distribution”, “person”, and the time attributes are “season”, “morning / evening”, “anniversary”, “shooting frequency” However, “latitude and longitude” and “indoor / outdoor” are listed as position attributes.

  It is assumed that the image of each scene is characterized by the characteristics shown in the table shown in FIG. 5, and the microcontroller 10 of this embodiment displays images satisfying these conditions on the display unit 13 as candidates.

  The correspondence of various attributes for each character is not limited to the one shown in FIG. That is, the character may be other than the above-described four scenes, and the number of scenes is not limited to four. Further, the respective attributes are not limited to those described above.

  Then, the candidate image is selected by the microcontroller 10 according to the flowchart of FIG.

  First, in step S1, it is determined whether or not an image to be searched for has been input by the character input unit 11 by the user. Here, when there is no character input, it transfers to step S12 and the operation part 14 is operated and the imaging date input by a user is performed. Next, in step S13, it is determined whether or not a shooting date is designated. As a result, if the shooting date is specified, the process proceeds to step S14, and images of data corresponding to the specified date are displayed in a list on the display LCD 13a from the shooting date information attached to the image. . On the other hand, if there is no shooting date designation or character input, the process proceeds to step S15, and the latest shooting data is displayed on the display LCD 13a.

  On the other hand, when an image to be searched is input in the above step S11, the process proceeds to step S16 to start an image search processing operation which is a feature of the present invention. For example, as shown in FIG. 7A, the image is input by selecting the image from the image characters displayed on the display LCD 13a with the cross key switch 14a or the like. .

  In steps S16 to S18, the image data itself is subjected to light / dark analysis, color analysis, and whether or not a person is photographed. Next, in step S19, information on the spot (GPS information) taken from the information attached to each image and information indicating the date and time taken are referenced. At this time, when the date / time information is moved overseas, the time difference is corrected from the GPS information. Accordingly, in the subsequent step S20, the shooting location is first analyzed from the GPS information and the database, and the season and time are referred to by the date and time information corrected in step S21.

  Since the location information obtained from the GPS information indicates whether it is the northern hemisphere or the southern hemisphere, such information is also considered for the season. Further, the anniversary etc. changes depending on the area, but the anniversary analysis performed in step S22 is executed from the date / time information, the location information and the database. In addition, since the user who does not travel abroad does not need the search so far, the GPS information referred to season or anniversary determination may be omitted.

  Further, in step S23, whether or not the image is taken indoors is analyzed based on the location information obtained in step S20, information on the presence or absence of flash emission, and white balance (WB) correction information. This white balance is one of image electronic processing that examines the condition of the wavelength of light that illuminates the subject and shoots white objects in white, and the wavelength distribution of light between sunlight and fluorescent lamps and lamps. Since it is different, this is corrected. By applying this technique, if the wavelength distribution of the light from the artificial light source is used, it may be determined as indoor.

  In step S24, it is analyzed from the shooting date / time information of each image whether or not shooting was frequently performed. In other words, typical scenes that are frequently photographed include, for example, weddings. For example, when the upper right image character (image 47b) is selected from the four images 47a to 47d displayed in the display LCD 13a as shown in FIG. An image that is determined to have been taken is preferentially displayed.

  In this manner, information such as the table shown in FIG. 5 is generated from the image and the data attached thereto, and thus candidate image groups are searched based on the concept shown in FIG. Thereby, as shown in FIG. 7B, the representative screen of the searched image group is displayed on the display LCD 13a. In addition, if the place name can be determined from the place information, the place name may be written together.

  When it is desired to view the contents of the image group constituting the group from the representative screen displayed in this way, the image is selected by operating the cross key 14a or the like. That is, in steps S26 to S28, the images constituting the selected group of images are displayed as shown in FIG.

  At this time, if the date and the time when the image was taken are displayed in addition to the location and the place name, the user can have various memories.

  As described above, according to the present embodiment, it is possible to provide a search device that allows a user to easily enjoy image data corresponding to a user simply by inputting an image.

  In addition, the characteristics of the image itself, the shooting time, location information, and the shooting conditions of the previous and subsequent photos are taken into consideration.

  This search device may be configured by a personal computer or the like, and in that case, it may be referred to via the Internet as a database. In addition, information may be input in a CD or DVD format.

  In the above-described embodiment, only the constituent groups are displayed. However, the present invention is not limited to this. Further, because of the limitation at the time of shooting, it may be possible to search even when the scene of the target image is different from the selected character.

  Furthermore, the conditions of the table shown in FIG. 5 may be further narrowed down or relaxed according to weighting or statistical search conditions.

  By the way, the present invention is not a search in which a complicated search expression is set in detail using a keyboard, but is a method that can be set with a simple keyword or image. Therefore, the camera having the configuration as shown in FIG. When applied, further effects can be obtained.

  As an application of such a camera, map image information may be stored in the database 22 and displayed on the display unit together with other information.

For example, as shown in FIG. 8 (a), together with the captured image 13 _1, and the map 13 _2, place names and shooting date information 13 _3, may be simultaneously displayed in the display LCD13a. In this way, if the shooting location is also displayed on the map, it will be a variety of memories.

Further, as shown in FIG. 8 (b), by entering the number information by instruction section 13b in the screen, from the number information detected at the person determination by the person detecting unit, it is narrowed down appropriate image 13 ₄ Selective search is also possible.

  With such a function, it is possible to provide a camera that allows the user to easily search for an image that the user wants to see.

  Further, in the flowchart of FIG. 6 described above, search information is obtained from the contents of each image at the time of search. On the premise of such search, one image is taken each time when shooting with a camera. Alternatively, the search information may be analyzed and extracted from the image or GPS information and attached to the image sequentially. That way, faster search is possible.

  As described above, according to the present embodiment, it is possible to provide a camera that can search for an image that the user wants to see at high speed.

  Further, even in a system in which place name information is not searched using a database from GPS information, it is possible to assist in image search by calculating a change in position due to GPS information and a movement amount per unit time. it can. Many scenes (events) taken together include, for example, travel, wedding ceremony, and athletic meet. When the characteristics are summarized by the distance traveled by the photographer, the table shown in FIG. 9 is obtained. .

  As shown in FIG. 9, for example, travel is photographed at various places, so that the position changes frequently. On the other hand, since the wedding is photographed exclusively in the same room or near the building, it is assumed that the position change is small. And athletic meet is considered to be an intermediate change between these.

  Hereinafter, as a modification of the embodiment of the present invention, an image search operation according to the table of FIG. 9 will be described with reference to the flowchart of FIG.

  In this flowchart, when one of the three scenes in the table shown in FIG. 9 is selected by the user in steps S31 to S33, a group of imaged images that have undergone position changes in accordance with the selected scenes in steps S34 to S37. In step S34 to S36, the photographing time and GPS information are obtained from the attached information, the position change rate is obtained therefrom, and information such as time frequency is added to the information as in the above-described embodiment. The image group is selected.

  When the image search is started, first, in steps S31 to S33, a scene imaged by the user is input according to the table shown in FIG. These are input when the selection unit 12 and the operation unit 14 are operated by the user on the table of FIG. 9 displayed on the display unit 13. Here, when a travel event is selected, the process proceeds to step S34 and is searched for an image group having a large positional change. If an athletic meet event is selected, the process proceeds to step S35, where it is recognized that the position change is an intermediate image group and searched. That is, it is a group of images in which the number of images taken in one day is taken and the position change is medium. Further, when a wedding event is selected, a set number of images are taken in 2 to 3 hours, and the image is recognized as a group of images with a small positional change and searched.

  Thereafter, in step S37, the images searched in steps S34, S35, and S36 are displayed on the display unit 13.

  Thus, according to the modification of this embodiment, it is possible to quickly search for photos of various events using GPS information and time information.

  As mentioned above, although embodiment of this invention was described, in the range which does not deviate from the summary of this invention other than embodiment mentioned above, this invention can be variously modified.

It is a block diagram which shows the structure of the search device which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the camera to which the search apparatus of the structure shown by FIG. 1 was applied. It is the figure which showed the external appearance structure of the camera shown by FIG. 3 is a flowchart for describing a photographing operation by the camera having the configuration of FIG. 2. It is the table | surface which showed the response | compatibility with the image attribute for every character, a time attribute, and a position attribute. It is a flowchart explaining the operation | movement of the image search by the search device concerning one Embodiment of this invention. It is the figure which showed the example of the image displayed on the display LCD of a camera. It is the figure which showed the example which displayed the image information and other information simultaneously on the display part of a camera as application of one Embodiment of this invention. It is the table | surface which showed the position change attribute as a modification of one Embodiment of this invention. FIG. 10 is a flowchart illustrating an image search operation according to the table of FIG. 9 as a modification of the embodiment of the present invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Microcontroller, 11 ... Character input part, 12 ... Selection part, 13 ... Display part, 15 ... Memory, 16 ... Image information part, 17 ... Image attachment information part, 20a ... Brightness detection part, 20b ... Color detection part, 20c ... person detection unit, 21 ... information generation unit, 22 ... database (DB), 30 ... search device, 31 ... GPS unit, 32 ... clock unit, 33 ... sequence controller (MPU), 35 ... image processing engine, 36 ... Photographic lens, 37... Image sensor, 38... Analog processing circuit (AFE), 40.

Claims (22)

Deriving means for deriving factor information having an image attribute, a temporal attribute or a positional attribute from the meanings of a plurality of characters or words;
Selection means for selecting at least one image from a plurality of images according to each factor information derived from the image attribute, temporal attribute or position attribute from the output of the deriving means;
A search apparatus comprising:   2. The search apparatus according to claim 1, wherein the image attribute derived from the plurality of characters or words includes factors such as light / dark information, light / dark distribution information, color information, and color distribution information.   The search device according to claim 1, wherein the temporal attribute derived from the plurality of characters or words includes factors such as season information and morning and evening information.   2. The search apparatus according to claim 1, wherein the positional attribute derived from the plurality of characters or words includes a factor of latitude / longitude information or indoor / outdoor information.   The search device according to claim 1, further comprising display means for displaying time information and position information of the searched image together with the searched image together with the searched image.   2. The search apparatus according to claim 1, wherein the image is selected from a plurality of displayed search characters.   The search device according to claim 1, wherein the deriving means derives based on characteristic information of each factor in a plurality of attributes associated with each other according to a character.   2. The search apparatus according to claim 1, further comprising display means for collectively displaying the images selected by the selection means as a group.   2. The search apparatus according to claim 1, wherein the selection means selects a specific image according to information on the frequency of taking photographs taken before and after that and information on changes in the photographing points. Deriving means for deriving factor information having at least one of an image attribute, a temporal attribute, and a positional attribute from conditions of at least one character and a word;
Search means for searching for at least one image from a plurality of images according to the factor information derived by the derivation means;
A search apparatus comprising:   11. The search apparatus according to claim 10, wherein the image attribute includes at least one of factors of light / dark information, light / dark distribution information, color information, and color distribution information.   The search device according to claim 10, wherein the temporal attribute includes at least one factor of seasonal information and morning / day / night information.   The search device according to claim 10, wherein the positional attribute includes at least one factor of latitude / longitude information and indoor / outdoor information.   The search device according to claim 10, further comprising display means for displaying at least one image searched by the search means.   15. The apparatus according to claim 14, further comprising selection means for selecting a desired image from the plurality of images displayed on the display means when there are a plurality of at least one image searched by the search means. The described search device.   16. The search device according to claim 15, wherein the display means displays information included in the searched image together with the image searched by the searching means.   The search device according to claim 16, wherein the display means collectively displays the images selected by the selection means as a group.   The search device according to claim 16, wherein the selection means performs selection based on information included in an image displayed on the display means.   The search device according to claim 16, wherein the selection means selects a specific image in accordance with information on the frequency of taking photographs taken before and after that and information on changes in the photographing locations.   11. The search apparatus according to claim 10, wherein the deriving means derives based on characteristic information of each factor in a plurality of attributes associated with each other according to an image.   A camera comprising the search device according to claim 1. A first step of deriving a factor having an image attribute and a temporal attribute or a positional attribute from the meaning of a plurality of characters or words;
A second step of selecting at least one image from a plurality of images according to each factor information derived by the image attribute, the time attribute or the position attribute according to the first step;
A search method comprising:

Images