JP2004164331A - Image search method, image search device, and image search program - Google Patents

Abstract

An object of the present invention is to enable a user to efficiently retrieve desired information from a large amount of image data.
An image search program according to the present invention determines an image group to be searched based on search conditions input by a user, converts the image group to an icon that visually represents the image to be searched, Alternatively, the feature amount for determining where to place the icon on the display screen is extracted from the image information of the original image, and based on the feature amount, the image to be searched or the icon is displayed on the display screen. A large number of images are determined by deciding which sheet is to be arranged in which position, and arranging sheets in which images or icons to be searched are arranged in the virtual three-dimensional space of the display device in accordance with the image feature amount. It is possible to efficiently search for information desired by the user from the data.
[Selection diagram] Fig. 1

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image search, and more particularly to a technique for efficiently searching a large amount of image data for information desired by a user, such as a product image search in online shopping and a material image search for content creation.
[0002]
[Prior art]
The conventional information search service uses full-text search technology targeting text information to search for information that matches the keyword entered by the user, and lists the titles and summaries of the information in order of matching degree. Was generally presented. In this case, if the keyword is appropriate and the number of matching information is small, the listed information is small and the information can be easily found. However, even if the keyword is inappropriate or the keyword is appropriate, if there is a large amount of information that matches the keyword, the list of information to be presented is enormous, and the order of displayed information is large. Does not always match the user's intention. Further, information to which text information is not assigned cannot be searched.
[0003]
In recent years, in limited fields such as apparel-related auctions, when a user specifies an image to be a key, an image search system searches for images similar to the image and displays a list of images in order of similarity. The similar image search technique described above (for example, see Non-Patent Document 1) has begun to be used. However, it is often difficult for a general user to specify an appropriate key image, and what can be used as a similarity between images is limited to image features such as image tint. At present, it is not always possible to set a key image that matches a user's search intention.
[0004]
A problem common to the above-described conventional search methods is "difficulty in specifying an appropriate key for narrowing down information". As a method for solving this problem, there is an approach in which as much information as possible is presented to a user, and information desired by the user is searched from the information (for example, see Non-Patent Document 2). Then, since images are excellent in viewability for humans, by using images as search targets, users can relatively easily search for desired information from a large number of displayed images.
[0005]
However, in the similar image search technology, it is not possible to determine whether the user is approaching the intended image by repeating similar image presentation, selection, and search. Therefore, from each image in the image database, statistical processing is performed using various feature amounts such as a color distribution and an edge distribution, and the components of the image feature amount are converted into three feature amounts, which are regarded as three-dimensional coordinates. Thus, a technology for generating computer graphics in which image icons are arranged at corresponding three-dimensional positions has been disclosed (for example, see Patent Document 1).
[0006]
Also, when using text attribute data such as a name or description document included in the image information as a method for extracting the feature amount from the image information in a related technology, it is effective to preliminarily characterize the text. A possible word set is selected, the relative importance of each word is measured using the tf-idf method (term frequency-inverse document frequency), and the tf-idf value of the word is calculated from each text by enumerating the relative importance. A method for extracting a feature vector as an element (for example, see Non-Patent Document 3), a method for extracting a feature amount of an image based on a color feature, a texture feature, a frequency feature (Fourier feature or DCT feature), a shape feature, and the like ( For example, see Non-Patent Document 4.) From the image data in the image information, the image is captured from the image. There is a method of separating an object from a background and extracting a feature related to color from the object (for example, see Non-Patent Document 5). In the case of a text feature or an image feature, since the dimension is generally high, the feature amount is extracted after reducing the dimension of the feature vector by KL expansion or the like (for example, Non-Patent Document 6). reference.).
[0007]
[Patent Document 1]
JP-A-11-175534 (pages 2-3)
[0008]
[Non-patent document 1]
"Similar Image Search", [online], Fujitsu) Intelligent Systems Research Department, [October 25, 2002 Search], Internet <URL: http: // venus. netlaboratory. com / salon / chiteki / mur / imgsearch. html>
[0009]
[Non-patent document 2]
Shigemi Nagata and 4 others, “Multimedia Information Retrieval System“ MIRACLES ””, Information Processing Society of Japan Humanities and Computer Symposium 2001, p. 267-274 (2001)
[0010]
[Non-Patent Document 3]
Shojiro Nishio and 5 others, "Iwanami Lecture Multimedia Informatics (Vol. 8) Information Structuring and Retrieval", Iwanami Shoten, 2000, p. 113-119
[0011]
[Non-patent document 4]
Vittorio Castelli and Lawrence D. Bergman, "Image Databases", John, Wiley & Sons, 2002, p. 285-372
[0012]
[Non-Patent Document 5]
Shuichi Shiiya and 4 others, "Product search service using color features of images", Information Processing Society of Japan 42nd Groupware and Network Services Workshop p. 267-274 (2001)
[0013]
[Non-Patent Document 6]
Kenichiro Ishii and 3 others, "Easy-to-understand pattern recognition", Ohmsha, 1998, p. 106-114
[0014]
[Problems to be solved by the invention]
As described above, the "approach of presenting as many images as possible to the user and having the user search for the desired image from among them" is a method for visually searching for a desired image from a large number of images. This eliminates the need for setting key information and is a very effective method. However, displaying a large number of images on one screen has the following problems.
(A) When a large number of images are to be displayed on one screen, each image becomes small, making it difficult to determine. In order to avoid this, there are methods of scrolling the screen and dividing pages within the range that can fit on one screen, but with these methods, it is possible to arrange only one axis (feature amount) such as name order, date order etc. However, it is difficult to grasp the distribution of what kind of images and how many as a whole.
(B) If a large number of images are randomly displayed on one screen, it is difficult to search for a desired image from the images.
(C) When the search process is executed on the server and the processing result is displayed on the display device of the client via the communication network, a long transfer wait due to transfer of a large amount of image information occurs. Further, in the client, in order to display the image data on the display device, processing for each image such as reduction of the image data is required, and the CPU load is greatly increased.
[0015]
The present invention has been proposed in view of the above circumstances, and has as its object to enable a user to efficiently retrieve desired information from a large amount of image data.
[0016]
[Means for Solving the Problems]
FIG. 1 shows an overall configuration diagram of a first embodiment in which the present invention is realized by a single computer. The image search program 10 of the present invention is a program executed on a computer as the image search device 1, and includes search conditions input by the user such as a directory of the image database 21, keywords and key screens for information desired by the user. Based on the search request receiving means 11 for determining an image group to be searched, and a feature value for determining at which position on the display device 3 the image to be searched is arranged. A feature extracting unit 13 for extracting from the image information, an arranging unit 14 for generating, based on the feature amount, an arrangement information for indicating on which sheet on which position the image to be searched is arranged on the display device 3; The search target image is displayed in a virtual three-dimensional space on the display device 3 in accordance with the arrangement information. It functions as a virtual space display means 15 and, as shown in FIG. 7, a sheet in which a plurality of images to be searched are arranged in accordance with a feature amount is arranged in a virtual three-dimensional space on the display device 3. This makes it possible to efficiently search for information desired by the user from a large amount of image data.
[0017]
The image search program 10 further includes an iconifying unit 12 that converts the image to be searched determined by the search request receiving unit 11 into an icon that is represented only by a color, a shape, or a combination thereof. Extracting a feature amount for determining where to place the search target image or the icon on the display screen from the image information of the search target image or the original image of the icon; Generating, on the basis of the feature amount, arrangement information indicating on which sheet on the display screen the image or icon to be searched is arranged, and the virtual space display means performs the search in accordance with the arrangement information. By displaying a target image or a sheet on which the icons are arranged in a virtual three-dimensional space on the display device, as shown in FIG. Person in the seat other than the seat being viewed, it becomes possible to keep displaying the icon visually represent the content of the image instead of the image. Further, as shown in FIG. 9, by sequentially replacing the icons with images with priority given to the sheet before and after the sheet currently being viewed, which is likely to be viewed next by the user, a large amount of image data can be obtained. It is possible to retrieve information desired by the user at high speed.
[0018]
Note that the above-mentioned “converting an image to be searched determined by the search request receiving unit 11 to an icon that is expressed only by a color, shape, or a combination thereof” means that a standard image is converted into a color, shape, or Generate an icon that is expressed only by its combination, etc., store the generated icon in association with identification information, and convert it to identification information of the icon corresponding to the image to be searched instead of converting it to an icon Shall be included.
[0019]
FIG. 4 shows an overall configuration diagram of the second embodiment in which the present invention is realized by linking the client 4 and the image search server 8. The search request unit 41 of the client 4 makes a search request to the image search server 8 via the communication network 7 based on the search condition input by the user. The same image search processing as that performed is performed, the result output unit 85 transmits the search result to the client 4 via the communication network 7, and the result acquisition unit 42 of the client 4 receives the search result from the image search server 8, The virtual space display means 43 displays an image search screen on the display device 6 based on the received search result.
[0020]
In the second embodiment, since image information is transferred via the communication network 7, icons having a small amount of information are arranged on sheets other than the sheet being viewed by the user, so that the icons are displayed on the display device 6 of the client 4. It is possible to avoid a long transfer wait due to transfer of image information that occurs when displaying the search processing result. In addition, by providing the image search function on the server side capable of storing a large amount of data and having a high processing speed, it is possible to store a very large amount of image information which is difficult for the client to store, or By cross-searching for other related various information, not only display of image data and explanation of image data, but also supplementary explanation of encyclopedia and provision of various value-added information can be provided.
[0021]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 1 is an overall configuration diagram of Embodiment 1 of the present invention. The image search program 10 of the present invention is a program executed on a computer acting as the image search device 1. The search request receiving means 11 sets the image group to be searched from the image database 21 based on search conditions input by the user, such as a directory, a keyword for information desired by the user, and a key screen. Is determined. The iconifying means 12 converts the image information of the image to be searched into an icon that visually represents the image, stores the converted icon in the icon file 22 in association with the identification information of the icon, and identifies the converted icon. The information is stored in the management file 23 in association with the image ID of the original image.
[0022]
The feature extracting unit 13 calculates a feature amount for determining where to place the search target image or the icon on the display device 3 from the image information of the search target image or the original image of the icon. The extracted feature amount is stored in the management file 23 in association with the image ID of the original image. The arranging means 14 generates, based on the feature amount, arrangement information indicating the position of the sheet on the display device 3 where the image or the icon to be searched is arranged, and assigns the arrangement information to the sheet ID, The relative position is stored in the management file 23 in association with the image ID of the original image. The virtual space display unit 15 displays a sheet in which a plurality of search target images and icons are arranged according to the feature amount as shown in FIG. They are placed one on top of the other in dimensional space.
[0023]
The external storage device 2 includes an image database 21 storing various image information, an icon file 22 storing image information of an icon that visually represents an image to be searched in association with identification information, A management file 23 storing feature amounts, position information, and identification information of icons is stored in association with the image ID of the image that has become. The display device 3 allows the user to specify search conditions, Used to display the search results of the image search program and the like.
[0024]
FIG. 2 is an example of a management file for controlling a search screen displayed by the image search program 10. In the first embodiment, it is located on the external storage device 2, but may be deployed on the memory of the image search device 1. The image ID is information for identifying image information stored in the search target image database. In this example, the search target image is a mobile terminal device, the image with image ID 1 is a black folding type mobile phone, the image with image ID 2 is a gray straight type mobile phone, and the image with image ID 3 is It is assumed to be a gray PDA (Personal Digital Assistance).
[0025]
In this example, the feature amount 1 is used for icon generation, and is a numerical value of a color. 0 indicates black and 8 indicates gray. The feature quantities 2 to n are feature vector values extracted from text attribute data, feature vector values extracted from shape features, and the like, and are used to determine an arrangement when an image or icon to be searched is displayed on the display device 3. Numerical information used for The sheet ID and the relative position are arrangement information for displaying the search target image or icon on the display device 3. The sheet ID is position information indicating which sheet to display, and the relative position is information indicating which position in the sheet to display. The icon information is identification information of the icon visually represented, for example, when the icon information is 0, representing a black square of 1 cm square.
[0026]
FIG. 3 is a flowchart illustrating a flow of processing of the image search program 10 executed on the image search device 1 according to the first embodiment. In the present invention, first, a search condition input by the user from the display device 3 is received (S301), and an image group composed of image data to be searched and attribute data of the image that matches the search condition from the image database 21. Is determined, and a list of image IDs of the search target images is output to the management file 23 (S302). In this example, the details of the processing are omitted because the prior art is used, but there are two methods for specifying the search conditions. One is a method for specifying the directory in which these image information groups are stored. There is another method of specifying an image information group obtained as a result of a search by an existing information search tool (for example, a keyword search engine).
[0027]
In addition, for example, an image (key image) closest to the one that the user is searching is designated from among a plurality of images randomly extracted from the image database, and a similar image search technique is used, for example. Search for images with color features close to the color features extracted from the image, display a list of image file names that have some association with the specified key image, or use the drawing tool to search for By inputting a key image by drawing an existing image, using a similar image search technique, for example, searching for an image having a color feature close to the color feature extracted from the key image, There is also a method of displaying a list of related image IDs and specifying an image group.
[0028]
When the image group to be searched is determined, an icon of an image belonging to the image group is generated, the generated icon is stored in the icon file 22 in association with the identification information, and the image ID is associated in the management file 23. The identification information of the generated icon is stored in the icon information (S303). In order to generate the icon, the content of the image information is first analyzed using a conventional technique. If there is numerical attribute data in the image information, the numerical value is paired with the attribute name and extracted. If there is text attribute data in the image information, words (important words) characterizing the main image information are extracted from the text attribute data.
[0029]
Specifically, a method using the mutual information amount, a method using the chi-square value, or using the tf-idf value while referring to the text data in all the image information to be searched is necessary. The important words are extracted by using a technical term dictionary and rules accordingly. Further, from the image data in the image information, the object and the background appearing in the image are separated, features relating to color are extracted from the object, and icon image information is generated based on the information.
[0030]
Conventionally, an image search screen could not be displayed unless all reduced images to be searched were generated. However, when the present invention is applied, the sheet currently being viewed by the user (when displayed first, the first sheet is displayed). For sheets other than those described in (1) and (2), only icons need to be arranged, so that the CPU processing time and image data access time involved in image reduction processing are greatly reduced, and a screen display for image search can be performed at high speed.
[0031]
In this embodiment, an icon is generated in step S303. However, an icon that represents a standard image only in color, shape, or a combination thereof is generated in advance, and the generated icon is identified as identification information. The corresponding icon may be stored in the icon file 22 and the optimum icon may be selected from the registered icons.
[0032]
Next, a feature amount for determining an arrangement position of an image or an icon is extracted from each piece of image information to be searched, and the extracted feature amount is stored in the management file 23 in association with the image ID (S304). This process also uses the conventional technology, and if there is numerical attribute data in the image information, the numerical value is used as a feature as it is, and in the case of text data, it seems to be effective to preliminarily characterize the text A word set is selected, the relative importance of each word is measured using the tf-idf method, and the word is enumerated to extract a feature vector having the tf-idf value of the word as an element from each text. For image data, color features, texture features, frequency features (Fourier features and DCT features), and shape features are extracted.
[0033]
Next, the arrangement of the images or icons on the display screen is determined based on the extracted features, and the arrangement information is stored in the management file 23 as the sheet ID and the relative position in the sheet in association with the image ID (S305). Also in this process, in the case of numerical attribute data, icons are arranged in line priority from the upper left of the plane by using the order relation of the values, or arranged in the depth direction in order in the case of the numerical attribute data using the conventional technology. It is possible. In the case of a text feature or an image feature, the dimension is generally high. First, the dimension of the feature vector is reduced by KL expansion or the like, and icons are arranged two-dimensionally by a self-organizing map using the obtained low-dimensional vector. Then, it is possible to arrange such that similar image information gathers nearby.
[0034]
Next, based on the arrangement information of the search target image or icon obtained in S305 via the management file 23, the image is extracted from the image database 21 and the icon is extracted from the icon file 22 and the image search screen is displayed on the display device. 3 (S306). As described above, the user browses the search screen in the virtual three-dimensional space with excellent browsability without setting detailed key information, thereby quickly and efficiently obtaining a desired image from a large number of images. It becomes possible to search.
[0035]
FIG. 4 shows an overall configuration diagram of the second embodiment in which the present invention is realized by linking the client 4 and the image search server 8. The search request unit 41 of the client 4 makes a search request to the image search server 8 via the communication network 7 based on the search condition input by the user. The same image search processing as that performed is performed, the result output unit 85 transmits the search result to the client 4 via the communication network 7, and the result acquisition unit 42 of the client 4 receives the search result from the image search server 8, The virtual space display means 43 displays an image search screen on the display device 6 based on the received search result. Note that the management files 53 and 92 may not be stored on the external storage devices 5 and 9 but may be developed on the memory of the client 4 or the image search server 8, respectively, as in the first embodiment. .
[0036]
FIG. 5 is a flowchart showing the flow of processing of the image search request program of the client 4 according to the second embodiment. Upon receiving the search condition from the user (S501), the image search request program 40 of the client 4 makes a search request to the image search server 1 in accordance with the received search condition (S502), waits, and performs a search result from the image search server 8 (S502). (Here, it is a general term of information necessary for displaying a screen for a user to perform an image search, and indicates each information stored in the management file 92 and image data of an image to be searched.) Upon receipt (S503), an image search screen is output to the display device 6 based on the search result (S504).
[0037]
FIG. 6 is a flowchart showing a flow of processing of the image search program 80 of the image search server 8 according to the second embodiment. Upon receiving a search request specifying search conditions from the client 4, the image search server 8 performs a search process for outputting a screen for image search similar to the image search program in the first embodiment. The processing from S601 to S605 corresponds to S301 to S305 in FIG. 3, and the same applies to the processing contents, and thus the description is omitted. Step S606 is processing for transmitting a search result. The content of the search result to be transmitted is based on image information in which the image ID of the search target image is associated with the image data, the sheet ID associated with the image ID of the search target image stored in the management file 92, and the relative position in the sheet. This is image management information including the configured location information and icon identification information.
[0038]
In the present embodiment, the image search server 8 transmits the search result of the image data of the original image to be searched in response to the search request from the client 4. Of the original image as the search result, and transmits the reduced image data of the original image instead of the image data of the original image, and transmits the original image data to the client 4 only when the client 4 requests the original image data. You may do so. The iconizing means 82, the feature extracting means 83, and the arranging means 84 provided in the image search program correspond to the load of the image search server 8 and the capabilities of the client 4 and the communication network 7, and the image search server 8, client 4 Regardless of which side is installed, it is possible to provide the user with a considerable function.
[0039]
FIG. 7 is a display example of an image search screen output to display device 3 in Embodiment 1 or display device 6 in Embodiment 2 to which the present invention is applied. In this example, four sheets in which 10 × 10 reduced images to be searched are arranged on one sheet are displayed in an overlapping manner, so that a large number of images to be searched can be distinguished on a display screen of a limited size. It is possible to display in an appropriate size. Therefore, it is possible to grasp the entire distribution of the search target image by simultaneously observing a plurality of sheets while obliquely shifting the viewpoint without scrolling or switching the screen.
[0040]
Further, by applying the present invention, it is possible to arrange a reduced image according to the feature amount of the image. For example, by arranging hierarchically, such as a large classification, a medium classification, and a small classification from a sheet in front, for example, Search that is suitable for classification of plants and plants, and by placing an image similar to that image directly below an image on the sheet, it is possible for users to search continuously, and color characteristics from images Can be extracted and arranged so that images of similar colors are gathered near each other, and the search efficiency of the user can be improved.
[0041]
FIG. 8 is a display example of an image search screen output to display device 3 in Embodiment 1 or display device 6 in Embodiment 2 to which the present invention is applied. In this example, a reduced image of the search target image is displayed only on the sheet that the user is browsing, and the other sheets are displayed with icons that visually represent the search target image in color. . As in the second embodiment, when the image database 91 is provided on the image search server 8 side and image data having a large amount of information is transmitted to the client 4 via the communication network 7, the client 4 needs to display a search screen. If the image data is displayed after all the image data to be searched is received, a long transfer waiting time occurs.
[0042]
As in this example, the image search server 8 transmits image data only to the sheet viewed by the user, and preferentially prioritizes the next or next sheet that is likely to be viewed by the user while the user is viewing. By transmitting the icons to the reduced image of the received image data as shown in FIG. 9, the client 4 reduces the transfer waiting time of the image data, converts the image to the reduced image, and the like. CPU load can be suppressed, and a high-speed image search can be performed for the user.
[0043]
In addition, on the sheets other than the sheet that the user is browsing, features related to color and shape of those that are reflected in the original image are displayed by icons, so it is possible to grasp what kind of image and how much as a whole In addition, it is possible to determine which sheet should be viewed next, and an efficient image search can be performed.
FIG. 9 is a display example of an image search screen output to display device 3 in Embodiment 1 or display device 6 in Embodiment 2 to which the present invention is applied. While the user is browsing the first sheet, the image search server 8 transmits the image data to be displayed on the next sheet having the highest probability of the next browsing by the user to the client 4 via the communication network 7. In this example, image data of the next sheet received while the user is browsing the first sheet is converted into reduced image data, and icons are sequentially replaced with reduced images. In addition, as another example of the display method, the same applies to the case where the reduced image on the sheet is switched at regular time intervals to sequentially display another image, or the image is sequentially displayed by moving the sheet itself. The effect is obtained.
[0044]
(Supplementary Note 1) A step of determining an image group that is a set of images to be searched based on a predetermined search condition input by a user;
Converting the image of the search target into an icon that visually represents,
Extracting the feature amount for determining where to place the image or the icon of the search target on the display screen from the image information of the original image of the search target image or the icon,
Based on the feature amount, a step of generating arrangement information indicating which position of which sheet on the display screen to place the image or icon of the search target,
Displaying, in accordance with the arrangement information, a sheet in which the images to be searched or the icons are arranged in a virtual three-dimensional space on a display device;
An image search method comprising:
[0045]
(Supplementary Note 2) Search request receiving means for determining an image group that is a set of images to be searched based on a predetermined search condition input by a user;
A feature extraction unit configured to extract a feature amount for determining a position on the display screen at which the search target image is to be arranged, from image information of an original image of the search target image;
Placement means for generating placement information indicating which position of which sheet on the display screen the image to be searched is placed based on the feature amount;
Virtual space display means for displaying a sheet in which the images to be searched are arranged in a virtual three-dimensional space on a display device in accordance with the arrangement information;
An image search device comprising:
[0046]
(Supplementary Note 3) The image search device further includes an iconization unit that converts the search target image into an icon that visually represents the search target, and the feature extraction unit displays the search target image or the icon. A feature value for determining a position on the screen to be placed is extracted from the image information of the image to be searched or the original image of the icon. And the virtual space display means generates, in accordance with the placement information, a sheet on which the images or icons to be searched are arranged. 2. The image search device according to claim 2, wherein the image search device is displayed in a virtual three-dimensional space on the display device.
[0047]
(Supplementary Note 4) Search request receiving means for determining an image group that is a set of images to be searched based on a predetermined search condition input by a user;
A feature extraction unit configured to extract a feature amount for determining a position on the display screen at which the search target image is to be arranged, from image information of an original image of the search target image;
Placement means for generating placement information indicating which position of which sheet on the display screen the image to be searched is placed based on the feature amount;
Virtual space display means for displaying a sheet in which the images to be searched are arranged in a virtual three-dimensional space on a display device in accordance with the arrangement information;
Image search program to make your computer function.
[0048]
(Supplementary Note 5) The image search device further includes an iconization unit that converts the search target image into an icon that visually represents the search target, and the feature extraction unit displays the search target image or the icon. A feature amount for deciding which position on the screen is to be extracted is extracted from the image information of the image to be searched or the original image of the icon, and the arranging unit is configured to determine the position of the search object based on the feature amount. Generating arrangement information indicating which position of which sheet on the display screen the image or icon is to be arranged, the virtual space display means, according to the arrangement information, the search target image or the sheet on which the icon is arranged The image search program according to claim 4, wherein the image search program is displayed in a virtual three-dimensional space on a display device.
[0049]
(Supplementary Note 6) Search request receiving means for determining an image group, which is a set of images to be searched, based on predetermined search conditions received from the client;
A feature extraction unit configured to extract a feature amount for determining a position on the display screen at which the search target image is to be arranged, from image information of an original image of the search target image;
Placement means for generating placement information indicating which position of which sheet on the display screen the image to be searched is placed based on the feature amount;
A result output unit that transmits to the client a search result including image identification information of the image to be searched, image data associated with the image identification information, and placement information;
An image search server comprising:
[0050]
(Supplementary Note 7) The image search server further includes an iconifying unit that converts the search target image into an icon that visually represents the search target, and the feature extraction unit displays the search target image or the icon. A feature value for determining a position on the screen to be placed is extracted from the image information of the image to be searched or the original image of the icon. It is determined which position of which sheet or image on the display screen to place the image or icon, the result output means, the image identification information of the image to be searched, and image data associated with the image identification information, 7. The image search server according to claim 6, wherein a search result including the image identification information and the arrangement information of the icon is transmitted to the client.
[0051]
(Supplementary Note 8) Search request receiving means for determining an image group, which is a set of images to be searched, based on predetermined search conditions received from the client;
A feature extraction unit configured to extract a feature amount for determining a position on the display screen at which the search target image is to be arranged, from image information of an original image of the search target image;
Placement means for generating placement information indicating which position of which sheet on the display screen the image to be searched is placed based on the feature amount;
A result output unit for transmitting, to a client, a search result including image identification information of the image to be searched, image data associated with the image identification information, and placement information;
Image search program to make your computer function.
[0052]
(Supplementary Note 9) The image search program further includes an iconification unit that converts the search target image into an icon that visually represents the search target, and the feature extraction unit displays the search target image or the icon. A feature value for determining a position on the screen to be placed is extracted from the image information of the image to be searched or the original image of the icon. The image output unit determines the position of the image or icon on which sheet on the display screen, and the result output unit associates the image or the icon with the image identification information of the image group and the image identification information of the image to be searched. The image search program according to claim 8, wherein a search result including the image data, the image identification information of the icon, and the arrangement information is transmitted to the client.
[0053]
(Supplementary Note 10) Search request means for transmitting search request information of an image group, which is a set of images to be searched, to an image search server based on predetermined search conditions input by a user;
From the image search server, image identification information of the image to be searched, and image data associated with the image identification information, a result acquisition unit that receives a search result including placement information,
Virtual space display means for displaying a sheet in which the images to be searched are arranged in a virtual three-dimensional space on a display device in accordance with the arrangement information;
A client comprising:
[0054]
(Supplementary Note 11) The result obtaining unit receives, from the image search server, image identification information of the image to be searched, and a search result including image data, icon identification information, and arrangement information associated with the image identification information. 10. The client according to claim 9, wherein a sheet in which the images to be searched and the icons are arranged in a virtual three-dimensional space on a display device is displayed in accordance with the arrangement information.
[0055]
(Supplementary Note 12) Search request means for transmitting search request information of an image group, which is a set of images to be searched, to an image search server based on a predetermined search condition input by a user;
From the image search server, image identification information of the image to be searched, and image data associated with the image identification information, a result acquisition unit that receives a search result including placement information,
Virtual space display means for displaying a sheet in which the images to be searched are arranged in a virtual three-dimensional space on a display device in accordance with the arrangement information;
An image search request program that causes a computer to function.
[0056]
(Supplementary Note 13) The result obtaining unit receives, from the image search server, image identification information of the image to be searched, and a search result including image data, icon identification information, and arrangement information associated with the image identification information. 12. The image search request program according to claim 11, wherein a sheet in which the images to be searched and the icons are arranged according to the arrangement information is displayed in a virtual three-dimensional space on a display device.
[0057]
【The invention's effect】
It is possible to efficiently search for information desired by the user from a large amount of image data.
[Brief description of the drawings]
FIG. 1 is an overall configuration diagram of a first embodiment.
FIG. 2 shows an example of a data format of a management file.
FIG. 3 shows processing of an image search program of the image search device according to the first embodiment.
FIG. 4 is an overall configuration diagram of a second embodiment.
FIG. 5 is a diagram illustrating a process of a client image search request program according to the second embodiment;
FIG. 6 shows processing of an image search program of an image search server according to the second embodiment.
FIG. 7 shows an example in which sheets are arranged in a virtual three-dimensional space.
FIG. 8 shows an example in which icons are arranged on sheets other than the sheet being viewed.
FIG. 9 is an example in which icons are replaced with images from those for which transfer has been completed.
[Explanation of symbols]
1 Image retrieval device
2 External storage device
3 Display device
4 clients
5 External storage device
6 Display device
7 Communication network
8 Image search server
9 External storage device
10. Image search program
11 Search request receiving means
12 Iconizing means
13 Feature extraction means
14 Placement means
15 Virtual space display means
21 Image Database
22 Icon file
23 Management files
40 Image search request program
41 Search request means
42 Result acquisition means
43 Virtual space display means
51 Image files
52 Icon file
53 Management file
80 Image Search Program
81 Search request receiving means
82 Iconizing Means
83 Feature extraction means
84 Placement means
85 Result output means
91 Image Database
92 Management file

Claims (5)

Determining an image group, which is a set of images to be searched, based on a predetermined search condition input by a user;
Converting the image of the search target into an icon that visually represents,
Extracting the feature amount for determining where to place the image or the icon of the search target on the display screen from the image information of the original image of the search target image or the icon,
Based on the feature amount, a step of generating arrangement information indicating which position of which sheet on the display screen to place the image or icon of the search target,
Displaying a sheet in which the search target images or the icons are arranged in a virtual three-dimensional space on a display device according to the arrangement information. Search request receiving means for determining an image group, which is a set of images to be searched, based on predetermined search conditions input by a user;
A feature extraction unit configured to extract a feature amount for determining a position on the display screen at which the search target image is to be arranged, from image information of an original image of the search target image;
Placement means for generating placement information indicating which position of which sheet on the display screen the image to be searched is placed based on the feature amount;
An image retrieval apparatus comprising: a virtual space display unit configured to display a sheet in which the images to be searched are arranged in a virtual three-dimensional space on a display device in accordance with the arrangement information. The image search device further includes an iconifying unit that converts the image of the search target into an icon that visually represents the image, and the feature extraction unit displays the image of the search target or the icon on the display screen. A feature amount for determining whether to be arranged at a position is extracted from the image information of the image to be searched or the original image of the icon, and the arranging means, based on the feature amount, the image or icon to be searched. Is generated on the display screen, and the virtual space display means displays a sheet on which the search target images or the icons are arranged according to the layout information. 3. The image search device according to claim 2, wherein the image search device is displayed so as to overlap the virtual three-dimensional space. Search request receiving means for determining an image group, which is a set of images to be searched, based on predetermined search conditions input by a user;
A feature extraction unit configured to extract a feature amount for determining a position on the display screen at which the search target image is to be arranged, from image information of an original image of the search target image;
Placement means for generating placement information indicating which position of which sheet on the display screen the image to be searched is placed based on the feature amount;
An image search program for causing a computer to function as virtual space display means for displaying a sheet in which the images to be searched are arranged in a virtual three-dimensional space on a display device in accordance with the arrangement information. The image search device further includes an iconifying unit that converts the image of the search target into an icon that visually represents the image, and the feature extraction unit displays the image of the search target or the icon on the display screen. A feature amount for determining whether to be arranged at a position is extracted from the image information of the image of the search target or the original image of the icon, and the arranging unit determines the image or the icon of the search target based on the feature amount. Generating arrangement information indicating which position of which sheet on the display screen is to be arranged, the virtual space display means displays, on a display device, the sheet in which the images to be searched or the icons are arranged according to the arrangement information. 5. The image search program according to claim 4, wherein the image search program is displayed so as to be superimposed in the virtual three-dimensional space.

Images