JP2011257868A - Image server, imaging apparatus, and control program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To eliminate a barrier such as limitation and labor imposed on an user when the user acquires his/her own favorite contents quickly, though the user has used the same contents many times and has acquired adequate contents from a server by inputting the user's hobby and preference to an apparatus.SOLUTION: In order to solve the above problem, an image server comprises: a reception part for receiving an image data from a terminal device via a network; a taste extraction part for analyzing the image data received by the reception part and extracting user's preference information in the terminal device; and a transmission part for selecting information related to the preference information extracted by the taste extraction part from a database and transmitting the information to the terminal device via the network.

Description

  The present invention relates to an image server, an imaging device, and a control program.

For example, a technique for classifying content in advance so that a server that stores music or the like can distribute content reflecting a user's hobbies and preferences to devices connected to a network is known.
[Prior art documents]
[Patent Literature]
[Patent Document 1] Japanese Unexamined Patent Application Publication No. 2007-279788

  The user has obtained appropriate content from the server by using the same content many times or by inputting his / her hobbies and preferences to the device. However, in any case, the restrictions and labors imposed on the user have become barriers for the user to quickly acquire favorite content.

  In order to solve the above-described problem, an image server according to a first aspect of the present invention analyzes a reception unit that receives image data from a terminal device via a network, and the image data received by the reception unit. A preference extraction unit that extracts preference information related to the user's preference, and a transmission unit that selects related information related to the preference information extracted by the preference extraction unit from the database and transmits the selected information to the terminal device via the network.

  Moreover, in order to solve the said subject, the imaging device in the 2nd aspect of this invention communicates with the said image server.

  In order to solve the above-described problem, a control program according to a third aspect of the present invention includes a reception step of receiving image data from a terminal device via a network, and analyzing the image data received in the reception step to obtain a terminal A preference extraction step for extracting preference information related to the preference of the user of the device, and a transmission step for selecting related information related to the preference information extracted in the preference extraction step from the database and transmitting it to the terminal device via the network. Let the computer run.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

It is a conceptual diagram explaining the concept of the content supply using the image server which concerns on this embodiment. It is a schematic block diagram of an image server. It is a figure which shows the file structure of an image file. It is a figure which shows the detailed structure of a preference extraction part. It is a conceptual diagram which shows the adjacent relationship between groups. It is an example of the adjacent information table which shows the adjacent information between groups. It is an example of the link table which shows the affiliation group of image data. It is an example of the user table which shows a user's preference order. It is a figure which shows a mode that the content as a user's preference information was displayed on the camera. It is an example of the processing flow of an image server.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

FIG. 1 is a conceptual diagram illustrating the concept of content supply using the image server 50 according to the present embodiment. Users Y ₁ , Y ₂ , and Y ₃ are users of the cameras 11, 12, and 13, respectively. The user Y ₁ sequentially sends image data captured by the camera 11 from the camera 11 to the image server 50 via the network 20. Similarly, the users Y ₁ and Y ₂ send image data from the respective cameras 12 and 13 to the image server 50. The image server 50 is a server managed by a camera manufacturer, for example, and sequentially accumulates image data sent from the user. A content server 30 is further connected to the network, and the image server 50 acquires and accumulates content from the content server 30.

  A camera carried by a user is a digital camera that can capture a subject image using an image sensor such as a CCD or CMOS. In addition to a so-called interchangeable lens digital camera and a compact digital camera, a mobile phone having a digital camera function, a notebook PC, portable game device, etc. Here, these devices are collectively referred to as a camera. In many cases, the camera includes a display unit such as an LCD, and the user can observe various information in addition to the captured image. In this embodiment, the user can acquire the content that suits his / her hobbies and preferences from the image server 50 by sending the taken images to the image server 50 one after another and enjoy it on the display unit of the camera.

  The image server 50 analyzes the image data sent from each user and determines the preference for each user. Then, content such as other image data and related information is selected from the preference group classified into the determined preference category and transmitted to the user's camera. Note that related information and the like belonging to the preference group is periodically acquired from the content server 30 and stored in the image server 50.

  FIG. 2 is a schematic configuration diagram of the image server 50. The image receiving unit 51 receives image data from a user sent via the network 20. The received image data is analyzed by the preference extraction unit 52 to determine the preference of the user who transmitted the image data. The specific configuration of the preference extraction unit 52 and preference determination will be described later.

  When the preference determination of the image data is completed, the group classification unit 53 executes group classification of the image data. The group classification is a task of assigning affiliation information as to which preference group the image data belongs to, and more specifically, a link table update operation that links the image data with a preset preference group.

  The image data after the group classification is stored in the image database 61. The image database 61 records and accumulates image data continuously transmitted by various users in this way. Note that the accumulated image data is deleted by the accumulated image update unit 71, for example, by the elapsed time since recording or by an explicit instruction via the network. Accordingly, the image data stored in the image database 61 is maintained in an appropriately updated state for use.

  The user can only transmit image data to the image server 50, but can also request content together with the transmission of image data. When the user's camera requests content along with the transmission of image data, the user's camera may add a content request flag as tag information that is incidental information of the image data. May be sent to

  When the content request transmitted from the user's camera is received by the content request reception unit 54, the image server 50 extracts the camera information attached to the content request by the camera information acquisition unit 55. In particular, the camera information to be extracted includes memory capacity information relating to how much content information the user's camera can record. The memory capacity of the content information recorded by the camera may be an empty capacity in the camera memory space or a capacity specified by the user. In the case where the content request is made together with the transmission of the image data, if the information about the memory capacity is described in the tag information of the image data, this can be used.

  As the images transmitted by the respective users to the image server 50 are accumulated, the preference for each user is established. Specifically, when the image data is classified into groups by the group classification unit 53 according to the user preference determined by the preference extraction unit 52, the group classification unit 53 also updates the user table indicating the preference order for each user. Run together. The user table is managed by the table management unit 62, and is read by the group classification unit 53 in accordance with the update. Therefore, when the table management unit 62 manages and updates the user table, the image server 50 can construct information about what hobbies and preferences each user has.

  The content selection unit 56 reads the user table managed by the table management unit 62 and extracts the preference of the user who is transmitting the content request. Then, the content stored in the content database 63 is read according to the preference order described in the user table. The content stored in the content database 63 is periodically acquired and updated from the content server 30 via the content update unit 73. Further, the contents stored in the image database 61 are classified for each of the above-described preference groups in which the image data is classified. Therefore, the content selection unit 56 may read content classified into the preference group described in the user table.

  Note that the content database 63 does not store the content itself, but may store only link information to the content. In this case, the link information is classified and managed for each preference group. With this configuration, the storage capacity of the content database 63 can be suppressed.

  The content selection unit 56 reads out the content in the content database 63 or, instead of reading out the content in the content database 63, the image data belonging to the preference group from the image database 61 as information related to the user's preference. It may be extracted. In this case, it is preferable to extract the image data transmitted by another user and recorded in the image database 61 instead of the image transmitted by the user requesting the content. The user who is requesting the content can know what kind of photograph the other user having the same preference is taking, and can also be used as a reference for his own shooting plan. Data can be valid content.

  The content selection unit 56 determines the transmission targets in order from the content with the highest priority so as to fit in the memory capacity of the content information allowed by the camera. The content thus determined is sent to the transmission unit 57, and the transmission unit 57 transmits the content to the user's camera. The user can use the content transmitted via the display unit included in the camera.

  Next, the file structure of image data stored in the image database 61 will be described. FIG. 3 is a diagram illustrating a file structure of an image file. The image file mainly includes main image data as compressed pixel data, reduced and resized thumbnail image data used for display on a display unit, and tag information that is related information of the main image data. Has a file structure. When the JPG file format is adopted as the file format of the image file, the tag information is defined as EXIF information. As shown, the tag information is classified for each category and various information is described. The main information is described below.

  Image information is described in the file information category. Specifically, the number of dots in the x direction and the y direction of the main image data and the number of bits that are the number of colors of each dot are described. The shooting date and time are described in the shooting information category. The accumulated image update unit 71 deletes the old image data by monitoring the date information. The shooting information category describes exposure information such as exposure time, F value, ISO sensitivity, focal length of the shooting lens, and shooting mode set at the time of shooting. Such information can be an index for classifying preferences. For example, it can be inferred that a user who frequently uses a short exposure time often uses a moving object as a subject, and this is a clue that it is a hobby to shoot sports photographs, vehicle photographs, and the like. Similarly, if the frequency of selecting the landscape mode as the shooting mode is high, it is a clue to determine that taking a landscape photograph is a hobby. Moreover, if the image analysis is performed on the surrounding area of the coordinates based on the description of the distance measuring point center coordinates, it is possible to guess what the user wanted to photograph. It can be an effective material for judging the target of the.

  In the GPS information category, shooting location information such as latitude, longitude, and altitude is described. By comparing this information with the map information, the image server 50 can acquire information regarding the shooting location. For example, if “Kyoto autumn leaves” can be inferred in combination with date information, the group of “Kyoto” and “Autumn leaves” can be listed as candidates for preference classification.

  In the category of attribute information, a user ID, storage target availability, and disclosure target availability are described. The image server 50 determines a user based on the user ID, and constructs a user table based on the user ID. Further, the image server 50 determines whether or not to record the image data in the image database 61 based on a flag indicating whether or not to accumulate. If the accumulation target availability flag is Y, the preference is extracted and recorded in the image database 61. If it is N, the preference is not extracted and is not recorded in the image database 61. The flag indicating whether or not to be stored can be selected from the menu item of the user's camera, but if the N flag is selected, the image data can be excluded from the target of preference determination. Therefore, when transmitting a large amount of captured image data to the image server 50, the user can transmit the image data all at once without selecting images one by one.

  Further, the image server 50 determines whether or not to allow the image data to be transmitted to other users as information related to the user's preference based on the flag indicating whether or not to be disclosed. When the user transmits his / her image data to the image server 50, if the N flag is selected, the user can set the image as a non-publication target. If the Y flag is selected, the release is permitted, and the image can be introduced to others.

  Next, how the preference extraction unit determines the user's preference will be further described. FIG. 4 is a diagram illustrating a detailed structure of the preference extraction unit 52. As shown in the figure, the preference extraction unit 52 includes a tag analysis unit 81 that analyzes the above tag information, and an image analysis unit 91 that performs image analysis using the main image data or thumbnail image data.

  The tag analysis unit 81 further includes a GPS information analysis unit 82, a distance measurement point extraction unit 83, an exposure information extraction unit 84, and the like. As described above, the GPS information analysis unit 82 reads shooting locations such as latitude, longitude, and altitude from the GPS information and compares them with map information acquired via the network 20 or the like to obtain shooting location information. Further, it is determined whether the shooting location information is a spot area classified as a so-called sightseeing spot. Spot areas are preset and listed, and each spot area is classified in association with a preference group directly or indirectly. For example, a “zoo” with a spot area is given a “family” classification as a preference group, and a “history” classification is given for a “shrine and shrine”.

  The distance measuring point extraction unit 83 extracts subject coordinates on which the camera has adjusted the focus as described above. The distance measurement point coordinates are delivered to the image analysis unit 91 as a region of interest for the user's subject. The exposure information extraction unit extracts exposure information from the tag information. Then, for example, it is estimated whether the user tends to prefer moving body photography or outdoor photography depending on whether the exposure time is shorter than a preset time. These estimates are determined in combination with shooting date / time information, GPS information, and the like.

  The image analysis unit 91 further includes a face recognition unit 92, a color information analysis unit 93, a focus state analysis unit 94, and the like. The face recognition unit 92 recognizes a face area existing on the subject from the main image data or the thumbnail image data. If the ratio of face recognition in the transmitted image data is high, the tendency of portrait photographs and snap photographs is estimated as user preferences. In particular, if it is configured to be able to recognize the face of a specific registered individual, it can be estimated whether it is a family photo, and a “family” classification can also be given.

  The color information analysis unit 93 can estimate, for example, a blue sky, a sunset, a night sky, etc. by analyzing information of a specific color that occupies a large proportion of the subject area, and can give classifications such as “landscape” and “night view”. it can. The focus state analysis unit 94 determines whether the image is a blurred image by extracting high-frequency components of the main image data or the thumbnail image data. When it is determined that the image is a blurred image, processing such as excluding from preference determination can be executed.

  Next, the correlation between preference groups will be described. FIG. 5 is a conceptual diagram showing the adjacency relationship between groups. The preference group is classified into a category classification as a major classification and an item classification as a minor classification subdivided within the category classification. For example, category classification A includes item classifications A-1, A-2, A-3, and the like. In the figure, B and C are further shown as the category classification. As the category classification, for example, different categories such as “shooting place” and “shooting target” are assigned. As the item classification, “Kyoto”, “Yokohama”, “Mt. Fuji”, “Amusement park”, etc. are assigned as items for subdividing the category classification “shooting place”. “Landscape”, “Family”, “Sports”, etc. are assigned to the category classification “shooting object”.

  When classified in this way, a correlation is created between the category classification and the individual item classification due to the relevance of the preference. For example, it can be said that “amusement park” and “family” are closely related. Therefore, adjacent information as an adjacent group is given to each item classification in advance with respect to a highly correlated category classification and item classification. For example, according to FIG. 5, A-1 is given adjacent information to B-1 and C-1. Adjacent information may be provided for other item classifications within the same category classification, or one item classification may be provided for other entire category classifications. For example, according to the figure, A-2 has adjacent information for A-3 in the same category classification, and B-2 has adjacent information for the entire category classification C.

The image data transmitted to the image server 50 is classified into preference groups in which adjacent information is defined in this way. For example, the image data I ₀₁ is classified into item classification A-1 of category classification A. An image may belong to several item categories. For example, according to the figure, the image data I ₀₂ belongs to B-2 as well as A-2.

  The above relationship is recorded in the table management unit 62 as an adjacent information table and a link table. FIG. 6 is an example of an adjacency information table indicating adjacency information between groups. As illustrated, target groups are listed in the left column, and adjacent groups are listed in the right column so as to correspond to target groups in the row direction. The adjacent information table is preset and recorded, and is referred to by the group classification unit 53 and the content selection unit 56 as appropriate. The adjacent information table is appropriately updated by the administrator of the image server 50.

  FIG. 7 is an example of a link table indicating the group to which image data belongs. As shown in the figure, image data recorded in the image database 61 is listed in the left column, and a group to which target image data in each row direction belongs is listed in the right column. The link table is updated every time the image server 50 receives image data and the group classification unit 53 executes group classification. When image data is selected from the content selection unit 56, the target image data is specified by referring to the link table.

  The table management unit 62 further records a user table. FIG. 8 is an example of a user table indicating the user preference order. As shown in the figure, in the user table, users are listed in the leftmost column, and preference groups corresponding to users in the respective row directions are listed in the right column. The preference groups are listed in order of preference, for example, from the first priority to the third priority. The enumeration of users is performed corresponding to the user IDs described in the tag information. Therefore, if a plurality of users can be registered on the user's camera side, the user can be registered in the user table as a plurality of users even with one camera by switching the user every time shooting is performed.

  The preference ranking is determined by the number of image data belonging to each preference group and the relevance of adjacent groups that each preference group has. For example, if 1000 pieces of image data belong to C-1 as a preference group and belong to 10 pieces of image data C-2, they have adjacent information to C-1 rather than the preference order of C-2. Priority is given to the preference order of A-1. Therefore, in this case, C-1 is defined as the first priority, A-1 is defined as the second priority, and C-2 is defined as the third priority. Since the content selection unit 56 refers to the user table and selects the content, even if the image data transmitted by the user is not the preference group to which the image data directly belongs, the content selection unit 56 can be a target group for content selection if it is an adjacent preference group. . It should be noted that how much importance is given to the relevance of adjacent groups is determined by a predetermined standard. Alternatively, a degree of correlation may be provided in advance between adjacent groups and described in the adjacent information table, and a target group for content selection may be determined in order from a preference group having a higher degree of correlation.

  The user table is reviewed each time the preference is extracted by the preference extraction unit 52 based on the image data transmitted by each user to the image server 50, and is updated if the preference ranking varies. Therefore, the more the image data is accumulated in the image database 61, the more accurately the preference of each user is reflected.

  The content information transmitted from the image server 50 to the user's camera as described above is displayed on the display unit of the camera. FIG. 9 is a diagram illustrating a state in which content as user preference information is displayed on the camera 100. As shown in the figure, a cross SW 102 as an operation member is provided in the vicinity of the display unit 101 provided on the back of the camera, and the user can move the selection range up, down, left, and right by operating the cross SW 102. it can. An enter button 103 is provided at the center of the cross switch 102, and the user presses the enter button 103 to confirm the selection by the cross switch 102.

  The display unit 101 displays content information received from the image server 50. For example, as shown in the title 111, shooting spot information is displayed. Since the content is selected and generated according to the user's preference from those stored in the content database that is sequentially updated, the content is displayed on the display unit 101 as timely and unique information. Further, if content related to a preference group with a lower preference order is also recorded in the memory of the camera, the user can switch the current display by operating the operation member and display the content.

  For example, in the figure, “Shinjuku Gyoen” is displayed in the recommended window 112 and “Tama Zoo” is displayed in the recommended window 113 as shooting spot information. Each text is a hyperlink, and the user can acquire detailed information by selecting the text. Also, incidental information 114 and 115 are displayed in the recommended windows 112 and 113, respectively. Information displayed as supplementary information also employs hyperlinks, and the user can select text when he wants to know detailed information.

  Further, related images 116 and 117 are displayed in the recommendation windows 112 and 113. As the related image, for example, image data recently taken by another user at “Shinjuku Gyoen” is displayed. Such image data is image data that the content selection unit 56 selects from the image database 61 as image data belonging to the user preference group. In addition, when there are many recommended windows, it can be configured to be switched and displayed by operating the scroll bar 118.

  Next, the processing flow in the image server 50 will be described again. FIG. 10 is an example of the processing flow of the image server. When the image receiving unit 51 receives image data from the user's camera in step S101, the preference extracting unit 52 extracts and determines the user's preference from the image data in step S102. In step S102, the group classification unit 53 executes group classification of the outside image data.

  In step S104, the group classification unit 53 checks the accumulation target flag in the tag information of the image data. If the accumulation target flag is to be accumulated, the group classification unit 53 proceeds to step S105 and records and accumulates the image data in the image database 61. If it is not an accumulation target, the image data is discarded as it is. Note that the image receiving unit 51 determines whether or not the image is to be accumulated. If the image is not to be accumulated, the image data may be discarded without performing preference extraction or the like.

  In step S106, the content request receiving unit 54 checks whether or not a content request signal is attached to the image data. If no content request signal is attached, the series of processing ends. If a content request signal is attached, the process proceeds to step S107.

  In step S107, the camera information acquisition unit 55 acquires camera information, particularly information on the memory capacity of the camera. In step S108, the user ID is acquired and the user is confirmed.

  In step S109, the content selection unit 56 reads the user table, the adjacent information table, and the link table from the table management unit 62 and refers to them. In step S110, the content selection unit 56 acquires content and image data from the image database 61 and the content database 63 according to the read information on each tail. Further, the content selection unit 56 determines the content to be transmitted in order from the content with the highest priority so as to fit in the memory capacity of the content information allowed by the camera, and the content thus determined is determined as the transmission unit in step S111. 57 to the user's camera. Then, a series of processing ends.

  In the above embodiment, the camera has been described as an example of the user terminal. However, the user terminal is not limited to the camera as long as the terminal handles image data. For example, if the user collects favorite photographed image data in a digital photo frame, the user can request content by connecting the digital photo frame as a user terminal to the image server 50. If the user transmits favorite image data to the image server 50 from the jacket image data attached to the music content collected by the portable music player, the image server 50 analyzes the image data and carries the portable data. You can send content to a music player. In this case, the portable music player becomes the user terminal. In the case of handling image data that does not have tag information that the captured image data has, the preference extraction unit 52 performs, for example, pattern matching with a template image that is prepared in advance, and the like, such as contour information of the image data. From the user's preference is determined.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

11, 12, 13 Camera, 20 Network, 30 Content server, 50 Image server, 51 Image reception unit, 52 Preference extraction unit, 53 Group classification unit, 54 Content request reception unit, 55 Camera information acquisition unit, 56 Content selection unit, 57 transmission unit, 61 image database, 62 table management unit, 63 content database, 71 stored image update unit, 73 content update unit, 81 tag analysis unit, 82 GPS information analysis unit, 83 ranging point extraction unit, 84 exposure information extraction , 91 Image analysis unit, 92 Face recognition unit, 93 Color information analysis unit, 94 Focus state analysis unit, 100 Camera, 101 Display unit, 102 Cross switch, 103 Decision button, 111 Title, 112, 113 Recommended window, 114, 115 Additional information, 116, 117 Related images , 118 Scroll bar

Claims (10)

A reception unit for receiving image data from a terminal device via a network;
A preference extraction unit that analyzes the image data received by the reception unit and extracts preference information related to the user preference of the terminal device;
An image server provided with the transmission part which selects the relevant information relevant to the said preference information extracted by the said preference extraction part from a database, and transmits to the said terminal device via the said network.   The image server according to claim 1, wherein the terminal device is an imaging device, and the image data is captured image data captured by the user with the imaging device. A group classification unit that classifies the image data into a plurality of groups based on the preference information extracted by the preference extraction unit;
The image server according to claim 1, wherein the transmission unit selects, from the database, the related information associated with a group to which the image data belongs. The plurality of groups have neighboring information based on the preference correlation;
The image server according to claim 3, wherein the transmission unit selects, from the database, the group to which the image data belongs and the related information associated with the group having the adjacent information.   The image server according to claim 4, wherein the transmission unit sequentially selects the related information associated with the group having the adjacent information having a high degree of correlation based on a memory capacity included in the terminal device. The database stores the image data received by the receiving unit,
The transmission unit selects, as the related information, image data having the same preference information as the preference information extracted by the preference extraction unit, which is received by the reception unit from another terminal device and accumulated in the database. The image server according to claim 1.   The image server according to claim 6, wherein the transmission unit does not select the image data received from the other terminal device and having a tag that is not to be disclosed.   The image server according to any one of claims 1 to 7, wherein the preference extraction unit does not extract the preference information for the image data to which a tag not to be analyzed is attached.   The imaging device which communicates with the image server of any one of Claim 1 to 8. An accepting step for accepting image data from a terminal device via a network;
Analyzing the image data received in the receiving step and extracting preference information relating to the user's preference of the terminal device; and
The control program which makes a computer perform the transmission step which selects the relevant information relevant to the said preference information extracted by the said preference extraction step from a database, and transmits to the said terminal device via the said network.

Images