JP2003271601A - Image management search method, image management search device, program, and recording medium storing the program - Google Patents

Abstract

(57) [Summary] [PROBLEMS] To efficiently search for an image similar to a folder, an image similar to a color designated by a user, and an image similar to a line drawing drawn by a user. SOLUTION: An arbitrary image file 21 in an unregistered folder is selected, an image feature amount is extracted from the selected image file 21, and an image file name is changed to an image file name so that the image file 21 can be specified. The image feature amount is registered in the file 22 and is associated with the image feature amount in the image feature amount file 23. If there is an unregistered image file 21 in the folder, the process returns to the first process, and if not, the registration process ends. Then, a query image for search is input, an image feature value is extracted from the query image, and a search process for searching for an image feature value similar to the image feature value of the query image is executed. An image file name associated with each image feature amount of the search result is obtained from the image file name file 22,
Generate search results for image file names.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image retrieval method for managing and retrieving images on a computer, an apparatus for implementing this method, a program for retrieving and managing images on a computer, and a recording medium storing this program.

[0002]

2. Description of the Related Art As this type of technology, for example, (1) Japanese Patent Laid-Open No. 2000-187731 (2) Japanese Patent Laid-Open No. 2000-285141 (3) Japanese Patent Laid-Open No. 2000-311246 The invention disclosed in US Pat.

Among these, the publication (1) describes which region in a color space each pixel belongs to in advance based on the color of each pixel constituting the prepared image. When a color histogram of an image is generated based on the number of strokes that belong to each region and the generated color histogram is extracted as a feature amount that represents the feature of the image, the color perception that humans have as a color space It is a color space that is suitable for human beings, and uses a color space that is divided into unit areas that match the human sense of color. There is disclosed an invention capable of extracting a feature amount of an image according to a human sense of similarity.

Further, in the publication (2), at least one kind of feature quantity representing a feature of the image is extracted from the inquiry image and a plurality of images to be searched, and the inquiry is made using at least one kind of extracted feature quantity. In an image search device that searches for an image that is the same as or similar to the inquiry image from the plurality of search target images by determining the similarity between the image and the search target image, the similarity between the query image and the search target image is calculated. Correspondence is made between the designating means for designating one inquiring image for each kind of feature amount used when obtaining and the similarity of each inquiring image designated by the designating means for each image to be searched. Retrieval means for obtaining the total similarity of the images to be retrieved by summing the similarities with the respective query images thus obtained, using the characteristic amounts of the types Disclosed is an invention that includes an output unit that outputs a search result based on the total similarity calculated for each image to be searched by the search unit, and that allows the user's intention to be accurately expressed as a search condition. There is.

Further, in the publication (3), in a similar image display method in which a plurality of similar images are displayed in association with each other, a feature amount indicating the feature of the image is extracted from the image data of each image, and the extraction is performed. The feature region space in which each feature amount exists is hierarchically divided based on the feature amount, and the hierarchically divided feature region space is generated into a tree structure that is branched as nodes, and the generated tree is generated. The display space that is the visualization space of the image is divided based on the structure, each image data corresponding to each divided display space is arranged and the image is displayed, and the feature space can be displayed at high speed based on the similarity. The invention as set forth above is disclosed.

[0006]

When managing image files such as those taken by a digital camera, the images are often managed in folders on the computer. However, when the image files are stored in a plurality of folders, It was difficult to search efficiently using the image feature quantity extracted from the image. The above-mentioned conventional techniques each provide a search technique for a large number of images, but do not mention how to perform a search on a folder of a file system of a computer.

The present invention has been made in view of such a background, and an object thereof is to provide an image similar to each folder, an image similar to a color designated by the user, and an image similar to a line drawing drawn by the user. An object of the present invention is to provide an image management search method, an image management search device, an image management search program, and a recording medium storing the program that enable efficient search.

[0008]

In order to achieve the above object, a first means is an image management search method for performing a management search for an image data file in a folder of a file system of a computer, and an image stored for each folder. The image feature amount of each image is extracted in advance from the data, and the similarity between the image feature amount of the input search target image and the extracted image feature amount is compared to search for an image similar to the search target image. It is characterized by

A second means is an image management search method for performing a management search for an image data file in a folder of a file system of a computer, in which the image feature amount of each image is extracted in advance from the image data stored for each folder. Every other time, the image feature quantity of the color designated by the user is compared with the extracted image feature quantity to search for an image similar to the color.

A third means is an image management search method for performing a management search for an image data file in a folder of a file system of a computer, in which the image feature amount of each image is extracted in advance from the image data stored for each folder. Every other time, the image feature amount of the line drawing drawn by the user is compared with the extracted image feature amount, and an image similar to the line image is searched.

A fourth means is characterized in that, in any one of the first to third means, when the user changes the image file in the registered folder, the image feature amount is updated.

A fifth means is characterized in that, in any one of the first to fourth means, all the subordinate folders belonging to the folder designated at the time of the search are targeted for the search.

A sixth means is an image management search device for managing and searching an image data file in a folder of a file system of a computer, and means for previously extracting the image feature amount of each image from the image data stored for each folder. And a means for searching the image similar to the search target image by comparing the similarity between the image feature amount of the input search target image and the extracted image feature amount.

The seventh means is characterized in that a procedure for executing each step of the image management search method according to any one of the first to fifth means is written as a program.

An eighth means is characterized in that the program according to the seventh means is read by a computer and written in a recording medium so that it can be executed.

[0016]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings.

1. First Embodiment 1.1 Configuration FIG. 1 is a block diagram showing the system configuration of an image management search system according to a first embodiment of the present invention. This system comprises a computer 10, a file system 20 in a storage device, and a display 30. An image management search program 11 is stored in the computer 10, and an image management search is executed according to this program.
The image management search program 11 includes an image registration routine 1
2 and an image search routine 15 are included. In the image registration routine 12, the first feature extraction process 13 and the registration process 14 are executed, and in the image search routine 15, the second feature extraction process 16, the search process 17 and the result. Each processing of the display processing 18 is executed. The processing result of the result display processing 18 is output to the display 30 and displayed on the display 30.

The file system 20 is an image file 2
1, the image file name file 22, and the image feature amount file 23 are included.

It is assumed that an image file obtained by, for example, a digital camera is stored in a storage device in the system configured as described above. In such a case, registration processing is performed on the designated folder according to the following procedure.

1) One arbitrary image file 21 in an unregistered folder is selected (file selection). 2) A feature extraction process for extracting an image feature amount from the selected image file 21 is executed (feature extraction process). 3) The image file name is registered in the image file name file 22 so that the image file 21 can be identified, and the registration process of registering the image feature amount in the image feature amount file 23 in association with this is executed (registration process). . 4) If there is an unregistered image file 21 in the folder, return to 1). If not, it ends.

The search process is processed as follows.

1) Input an inquiry image for retrieval.
The input image may be an image existing in a folder other than the relevant folder (search input). 2) A feature extraction process for extracting the image feature amount from the inquiry image is executed (feature extraction process). 3) A search process for searching an image feature amount similar to the image feature amount of the inquiry image is executed. In this search processing, similar images may be searched one by one with all image feature amounts, or a spatial index such as R-TREE may be used. Then, the image file name associated with each image feature amount of the search result is acquired from the image file name file 22, and the search result of the image file name is generated (search process). 4) Image file 21 from the search result of the image file name
A search result display process of displaying a list of the images of and displayed on the display 30 is executed (display process).

1.2 Image Registration Routine The details of the image registration routine 12 are as follows.

File system 20 of computer 10
Folders are three-layer folders 200, 210, 220, 221, 22 as shown in the folder structure of FIG.
It has a hierarchical structure of 2 and a specific folder 2 of the 3rd layer.
Suppose there are a large number of images in 22. As shown in the figure, the image feature amount file 23 and the image file name file 22 may be placed in the folder in which the image exists, or may be placed in another folder associated with the image folder.

The image feature quantity extracted in the first feature extraction processing 13 is described in the prior art as Japanese Patent Laid-Open No. 2000-1877.
It is possible to use the feature amount used in Japanese Patent No. 31, and, for image retrieval, a general color histogram or the like can be used. JP 2000-18773 A
The extraction of the feature amount disclosed in Japanese Patent Laid-Open No. 1-54: Which region in a color space each pixel belongs to in advance is divided into a plurality of regions based on the color of each pixel constituting the prepared image. Is specified, a color histogram of the image is generated based on the number of pixels belonging to each area, and the generated color histogram is extracted as a feature amount representing the feature of the image. Is a color space that matches a human sense of color, and a color space that is divided into unit areas that match a human sense of color is used. These feature amount data are represented as vector data (multidimensional data).

In the registration process 14, the image feature amount and the image file name are associated with each other in a one-to-one relationship as can be seen from the explanatory diagram showing the relationship between the image feature amount and the image file name in FIG. It is registered in the file name file 22. In this way, by registering in association with one-to-one, if either one can be searched, it is possible to search immediately because the corresponding relationship of the other is known. Further, in order to execute the image search at high speed, it is possible to further have an index file of the feature amount in addition to the image feature amount. As the index file, a general multidimensional spatial index R-TREE or the like can be used.

1.3 Image Retrieval Routine In the image retrieval routine 15, the image specified by the second feature amount extraction process 16 is extracted as in the feature amount extraction (first feature amount extraction process 13) during the image registration process. A feature amount is extracted from the. The extracted image feature amount is compared with the image feature amount in the image feature amount file 23 for each similarity. Since the similarity is expressed by the distance in the feature vector space, the Euclidean distance, which is a general distance expression, can be used. When a plurality of feature amounts are used, the user may be allowed to specify weighting for each feature amount. The image feature amounts are sorted in ascending order of distance (larger similarity) between the image feature amount of the inquiry image and the image feature amount in the image feature amount file 23, and the number of images previously designated by the user from the higher order is the result of the image search. And Alternatively, the image feature amount that falls within the threshold value of the distance (similarity) designated by the user in advance is set as the search result. The image file name associated with the image feature amount thus obtained is acquired from the image file name file 22, each image file 21 is loaded, and the image is displayed on the display 30. An example of the search screen and the search result screen is shown in FIG. FIG. 4 is a front view showing a screen of the display 30.

In FIG. 4, an operation display area 31a and a search result display area 31b are set on the similar image search screen 31, the operation display area 31a is provided on the upper side, and the search result display area 31b is provided on the lower side. ing.

The search target image 32 is displayed on the upper part of the operation display area 31a, and the "entire color" and "color distribution" are displayed on the right side thereof.
A slider bar 33 for designating weighting parameters of "outline" and "pattern" is displayed. Below the search target image 32, a search folder specification input unit 34 is provided.
However, below the slide input unit 33, a display number input unit 35 for inputting the maximum number of search displays is provided. On this screen, the search folder is C \ TEMP \ DA
In TA, the maximum display number is 20. From the search folder specification input unit 34, it is possible to specify an image folder and whether to search a subordinate folder. When these inputs and designations are completed and the search start key 50 provided below the display number input section 35 is selected, the search is started based on the above-mentioned settings.

The searched similar image is displayed in the search result display area 31b. In this example, the maximum display number is 20
Since it is set to, 20 images are displayed. The user may select an intended image from these.

1.4 Images in the image feature amount update folder may be edited, and image files may be added. Therefore, when the user designates a folder and executes the image registration check, it is checked whether the image file 21 in the designated folder is registered in the image file name file 22 and the image file name file 22 is re-registered in some cases. The processing is shown below.

(1) Select an arbitrary image file from the images in the folder. (2) If the selected image file 21 does not exist in the image file name file 22, the image file 21 is newly registered. (3) Even if the image file 21 exists in the image file name file 22, if the update date and time of the image file 21 is newer than the registration date and time, it is re-registered. In order to realize this, it is necessary to record the registration date and time in the image file name file 22 at the time of registration. (4) If there is an unprocessed image file, return to 1). (5) Image file 2 in image file name file 22
In 1, the image feature amount that does not exist in the folder is deleted from the image feature amount file 23 and the image file name file 22.

According to the first embodiment, the image feature amount is extracted from the image for each folder in advance and registered in the image feature amount file 23 and the image file name file 22 in a one-to-one correspondence. Every time the search is performed, it is possible to search in units of folders by searching using the image feature amount. Further, even when the user changes the image file 21 in the already registered folder, the image feature amount can be updated. Furthermore, according to the designation of the folder search from the folder designation input unit 34, all the subordinate folders belonging to the folder designated at the time of the search can be targeted for the search.

2. Second Embodiment This embodiment is an example configured so that an image stored in a distributed manner in each folder can be searched.
Hereinafter, the same reference numerals will be given to the respective units equivalent to those of the above-described first embodiment, and the duplicate description will be appropriately omitted.

2.1 Configuration The system configuration is the same as that of the first embodiment shown in FIG.

2.2 Image Registration Routine The details of the image registration routine 12 in the second embodiment are as follows.

File system 20 of computer 10
Folder is 3 as shown in the explanatory diagram of the folder structure in FIG.
Layer folders 200, 210, 220, 221, 222
Images are placed in a plurality of folders, and for example, image feature amount files 23b and 23a and image file names in which images of the folders are registered in the second layer folder 210 and the third layer folder 222, respectively. File 23
b and 21a are arranged.

The user executes the registration process by designating the folder 210 or 222, and the images in the designated folder are stored in the image feature amount file 23b or 2 in the folder.
3a and image file name file 22b or 22a
Register with. The processing is similar to that of the first embodiment described above.

2.3 Image Retrieval Routine The user designates a folder to be retrieved from the folder designation input section 34. It is possible to search all subordinate folders as well as the specified folder as follows.
In the case where the image feature amount file and the image file name file exist in the designated folder (corresponding to the folder 210-image feature amount file step 23b, the image file name file 22b, and the image file 21b), the first embodiment The same process as the search process is performed to obtain an image file list of search results.

If another folder exists in the folders (corresponding to the folder 220-folder 222-image feature amount file 23a, image file name file 22a, and image file 21a), the image feature amount file and the image file are similarly set. If the name file exists, the image file list of the process search result is obtained and merged with the corresponding image file list as in the search process of the first embodiment. Similarly, the lower folders are sequentially traced and the same processing is repeated to sequentially merge the image file lists of the search results.
Then, the image file list is sorted by the distance (similarity), and the number of images designated by the user in the upper rank designated by the user is set as the result of the image search. Alternatively, the image feature amount that falls within the threshold of the distance (similarity) designated by the user in advance is set as the search result. The image file name associated with the image feature amount thus obtained is acquired from the image file name file, each image file is loaded, and the image is displayed on the display 30. The display screen at this time is the same as that of the first embodiment shown in FIG.

In addition, each part not particularly described is the same as the first part described above.
Is configured and functions in the same manner as the embodiment of FIG.

According to the second embodiment, it is possible to perform the same search as that in the first embodiment for the images distributed and stored in each folder.

3. Third Embodiment The present embodiment is an example configured so that the user can specify a color and search for an image having that color. Hereinafter, the same reference numerals will be given to the respective units equivalent to those of the above-described first and second embodiments, and the duplicated description will be appropriately omitted.

3.1 Configuration The system configuration is the same as that of the first embodiment shown in FIG.

3.2 Image Registration Routine This is similar to the first and second embodiments.

3.3 Image Retrieval Routine FIG. 6 is a view showing the entire color retrieval screen 36 according to the third embodiment. An operation display area 36a and a search result display area 36b are set on the overall color search screen 36 of the display 30, and the operation display area 36a is provided on the upper side and the search result display area 36b is provided on the lower side.

A color palette (color picker) 27 is displayed above the operation display area 36a, a brightness slider bar 38 is below the color palette 27, and a selection color display section 39 is to the right of the slider bar 38. It is provided.
Further, below the lightness slider bar 38, a search folder designation input unit 34 is provided, and below the slide input unit 33, a display number input unit 35 for inputting the maximum number of search displays is provided. The searched image of the similar color is displayed in the search result display area 36b. In this example, since the maximum display number is set to 20, 20 images are displayed. A search start key 50 is provided below the display number input section 35.

When the user selects a color from the color palette 37 of the overall color search screen 36 displayed as shown in FIG. 6 and specifies the lightness with the slider bar 38, the specified color is displayed in the selected color display section 39. . In this third embodiment,
Unlike the first and second embodiments described above, the color histogram is generated from the selected single color. The method of generation is simple, setting a value in the bin of the color histogram corresponding to the selected color. A value of 100 is set in the color histogram that is normalized with the total of 100, and the search is performed by the same processing as that of the first and second embodiments described above with the feature amount of the color histogram. In this embodiment, it is assumed that the feature amount of the color histogram is extracted as the image feature amount.

The other parts not particularly explained are the same as those in the first section.
Alternatively, it is configured in the same manner as the second embodiment and functions in the same manner.

According to the third embodiment, in addition to the effects of the first and second embodiments, it is possible to retrieve an image having a color designated by the user.

4. Fourth Embodiment The present embodiment is an example configured so that the user can specify a line image and search for an image similar to the line image. Hereinafter, the same reference numerals are given to the respective units equivalent to those of the above-described first to third embodiments, and the duplicated description will be appropriately omitted.

4.1 Configuration The system configuration is the same as that of the first embodiment shown in FIG.

4.2 Image Registration Routine This is similar to the first and second embodiments.

4.3 Image Retrieval Routine FIG. 7 is a view showing a sketch retrieval screen 40 according to the fourth embodiment. Operation display area 4 on the sketch search screen 40
0a and the search result display area 40b are set, the operation display area 40a is provided on the upper side, and the search result display area 40b is provided on the lower side.

A sketch canvas 41 is displayed on the left side of the operation display area 40a, and a pencil button 4 is displayed on the right side thereof.
2. An eraser button 43 and a reset button 44 are provided below the search folder specification input unit 45 and a display number input unit 35 for inputting the maximum number of search displays.
An image similar to the searched line drawing is displayed in the search result display area 40b. In this example, since the maximum display number is set to 20, 20 images are displayed. A search start key 50 is provided below the display number input section 35.

The user can freely write or erase a line drawing on the sketch canvas 41 as shown in FIG. In the search process, the edge feature amount is extracted from the designated line image. The first or second depending on this feature amount
The search process is performed in the same manner as in the above embodiment. However, the feature amount of the edge needs to be extracted in advance as the feature amount. The extraction processing of the edge feature amount can be performed by a known method, for example, the method disclosed in the above-mentioned Japanese Patent Laid-Open No. 2000-285141.

In addition, each part not particularly explained is the above-mentioned first part.
Alternatively, it is configured in the same manner as the second embodiment and functions in the same manner.

According to the fourth embodiment, in addition to the effects of the first and second embodiments, it is possible to retrieve an image similar to the line drawing drawn by the user.

[0059]

As described above, according to the present invention, a similar image, an image similar to a color designated by the user, and an image similar to a line drawing drawn by the user can be efficiently searched for in each folder. .

[Brief description of drawings]

FIG. 1 is a block diagram showing a system configuration of an image management search system according to a first embodiment of the present invention.

FIG. 2 is an explanatory diagram showing a folder structure of the file system of FIG. 1 (first embodiment).

FIG. 3 is a diagram showing a correspondence relationship between an image file name stored in an image file name file and an image feature amount file of the file system in FIG. 1 and an image feature amount.

FIG. 4 is a diagram showing an example of a search screen and a search result screen in the first embodiment.

FIG. 5 is an explanatory diagram showing a folder structure in the third embodiment.

FIG. 6 is a diagram showing an example of a search screen and a search result screen in the third embodiment.

FIG. 7 is a diagram showing an example of a search screen and a search result screen in the fourth embodiment.

[Explanation of symbols]

10 computers 11 Image management search program 12 Image registration routine 13 First feature extraction process 14 Registration process 15 Image search routine 16 Second feature extraction process 17 Search processing 18 Result display processing 20 file system 21,21a, 21b image files 22,22a, 22b Image file name file 23, 23a, 23b Image feature amount file 31 Similar image search screen 36 Global Color Search Screen 40 Sketch search screen 50 Search start key 200,210,220,221,222 folders

Continued front page    F-term (reference) 5B050 CA07 DA09 EA04 EA08 EA09                       EA18 FA02 FA08 FA12 FA19                       GA08                 5B075 ND06 PR06                 5L096 AA02 AA06 BA18 FA35 JA03                       JA11

Claims (8)

[Claims] 1. An image management search method for performing a management search for an image data file in a folder of a file system of a computer, the method comprising: preliminarily extracting an image feature amount of each image from image data stored for each folder; Comparing the image feature amount of the search target image with the similarity of the image feature amount extracted in the extracting step to search for an image similar to the search target image. An image management search method characterized by performing a high-speed search by comparing image feature amounts. 2. An image management search method for performing a management search for an image data file in a folder of a file system of a computer, wherein a step of previously extracting an image feature amount of each image from the image data stored for each folder, Comparing the image feature amount of the designated color with the extracted image feature amount to search for an image similar to the color, and comparing the images in the folder with the image feature amount. An image management search method characterized by high-speed search. 3. An image management search method for performing a management search for an image data file in a folder of a file system of a computer, wherein a step of previously extracting an image feature amount of each image from the image data stored for each folder, Comparing the image feature amount of the drawn line drawing with the extracted image feature amount to search for an image similar to the line drawing, and comparing the images in the folder with the image feature amount. An image management search method characterized by high-speed search. 4. The method according to claim 1, further comprising a step of updating an image feature amount when a user changes an image file in a registered folder. Image search management method described. 5. The image management search method according to claim 1, further comprising a step of searching all lower folders belonging to a folder designated at the time of search. . 6. An image management search device for performing a management search for an image data file in a folder of a file system of a computer, comprising means for extracting in advance image feature amount of each image from image data stored for each folder. An image management search device, comprising: means for comparing the image feature amount of the search target image with the similarity of the extracted image feature amount to search for an image similar to the search target image. 7. A program in which a procedure for executing each step of the image management search method according to any one of claims 1 to 5 is written. 8. A recording medium in which the program according to claim 7 is read by a computer and written so as to be executable.