JP2017151632A - Server device, search method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a server device capable of readily obtaining content including an image of a desired person.SOLUTION: A server device 1 comprises: a first storage unit 111 that stores one or more contents, i.e. images transmitted by a plurality of terminal devices 2; a first reception unit 121 that receives search instructions including person specifying information, i.e. information on specifying a person from one terminal device 2 among the plurality of terminal devices 2; a search unit 131 that searches for one or more contents including a face image, i.e. an image of the face of a person corresponding to the person specifying information included in the search instructions received by the first reception unit 121; and a transmission unit 14 that transmits one or more contents retrieved by the search unit 131 to the one terminal device 2. The server device allows a user to readily obtain contents including an image of a desired person.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a server device that searches and transmits contents such as images.

  In a conventional server device, identification information in which a person name, a place name, etc. are described is associated with an image (photo or video). For example, when a person name is input, the identification information including the person name is searched. Thus, an image list screen associated with the corresponding identification information is transmitted (see Non-Patent Document 1).

“Google”, [online], [Search July 6, 2015], Internet [URL: www.google.co.jp]

  By the way, in the exhibition of photographs taken at an event where a large number of people participate, such as athletic meet, to obtain a photograph of a desired person, for example, my child, from among a lot of photographs Not easy.

  On the other hand, in this type of event, many people are taking pictures with their own cameras and smartphones, and images taken by other people may contain images that show the desired person by chance. There is not much nature.

  Therefore, each person may upload an image taken by himself / herself from a terminal device such as a personal computer or a smartphone to the server device, while allowing an image of a desired person to be downloaded from the server device to the terminal device. It is done.

  However, even when trying to achieve this using a conventional server device, the photos and videos taken by each person are usually not associated with identification information describing the person name, and all uploaded It is not realistic to create such identification information for the photos and videos because the burden on each person becomes heavy.

  Therefore, an object of the present invention is to provide a server device or the like that can easily obtain content in which a desired person is shown.

  A server device according to the present invention includes a first storage unit that stores one or more contents that are images (photos or moving images) transmitted from a plurality of terminal devices, and person designation information that is information relating to person designation. Corresponds to a first receiving unit that receives a search instruction from one terminal device among the plurality of terminal devices, and person designation information included in the search instruction received by the first receiving unit among the one or more contents A search unit that searches for one or more contents including a face image that is an image of a person's face, a transmission unit that transmits the one or more contents searched by the search unit to the one terminal device, Is a server device.

  With this configuration, when a desired person is designated on the one terminal device, one or more contents (contents here) in which the person is reflected are searched in the server device, and one or more searched contents are found. Since it is transmitted to the one terminal device, it is possible to easily obtain content in which the person is reflected from among a large number of content.

  In the above configuration, the person designation information may be constituted by a designation operation for a person in an image output from the one terminal device.

  With such a configuration, simply by specifying a person in the image, the content in which the person is shown is searched, which increases convenience.

  In the above configuration, the designation operation may be an operation of designating one person among a plurality of persons in an image output from the one terminal device.

  With such a configuration, the user can search for content in which only one person is specified from among a plurality of persons in the image, so that convenience is improved.

  In the above-described configuration, the information processing apparatus further includes a second storage unit that stores one or more pieces of person information including a face image of a person and a person identifier for identifying the person, and the person designation information includes a person identifier. The search unit acquires a face image of a person who is paired with the person identifier included in the person designation information from the second storage unit, and includes a part of an image having a predetermined relationship with the face image. One or more contents may be searched.

  With such a configuration, simply specifying the person identifier of the desired person, the face image of the person is acquired, and the content including the face image is searched, so that convenience is improved.

  In the above configuration, one or more contents in the first storage unit are associated with time information related to time, the search instruction includes time information, and the search unit includes a time included in the search instruction. One or more contents corresponding to information are determined, and the transmission unit is a screen having a union of one or more contents searched by the search unit and one or more contents determined by the search unit, Screen information that is information on a screen for selecting or specifying a person may be transmitted. Note that the time information included in the search instruction may be information in a mode different from the time information associated with the content (for example, time zone information indicating a time zone).

  With this configuration, a screen including not only content determined by person search but also content determined by narrowing down by time information is output from the terminal device, and content can be selected or a person can be designated on this screen. That is, since it is possible to use both a person search based on a face image and a refinement based on time information, the possibility that a content showing a desired person can be obtained increases.

  Further, in the above configuration, the one or more contents in the first storage unit are associated with position information regarding a position, the search instruction includes position information, and the search unit includes a time included in the search instruction. One or more contents corresponding to the information and the position information may be determined. Note that the position information included in the search instruction may be information in a mode different from the time information associated with the content (for example, area information indicating an area).

  With this configuration, content determined by narrowing down based on position information is also included on the screen, so that narrowing based on time information can be used together, and there is a possibility that content containing a desired person can be obtained. Increase more.

  In the above configuration, the time information associated with the one or more contents in the first storage unit indicates the time when the content was captured by the camera, and is associated with the one or more contents in the first storage unit. The position information indicates the position of the camera that captured the content, and the one or more contents in the first storage unit further include optical information related to at least one of an angle of view and a focal length of the camera that captured the content. And a calculation unit that calculates the position of each person in the content using position information and optical information associated with the one or more contents in the first storage unit, The search unit calculates the person calculated by the calculation unit when determining one or more contents corresponding to the time information and the position information included in the search instruction. The location may be compared with the position indicated by the positional information included in the search instruction.

  With this configuration, the position information of the person shown in the content is calculated, not the position information of the camera itself, from the position information and optical information of the camera corresponding to the content. Since the position information of the person is used, not only the position of the camera but also the narrowing down by the position of the person is possible.

  Further, in the above configuration, the search unit includes a similarity between a face image corresponding to the person designation information and each face image included in one or more contents stored in the first storage unit. Is compared with a threshold value to determine whether each content includes a face image corresponding to the person designation information, and when at least one of time information and position information is used for the search, The threshold may be changed when neither information nor position information is used.

  With this configuration, a significant search result can be easily obtained even if the recognition rate of the face image is low.

  Further, in the above configuration, the apparatus further includes a third storage unit that stores at least period information indicating a period and event information indicating an event performed in the period, and the one or more contents of the first storage unit Is associated with time information related to time, and when the transmission unit transmits one or more contents searched by the search unit, the time information associated with the content and the third storage unit Is compared with each period information stored in the event information to identify event information indicating an event that occurred during the period including the time when the content was shot, and the identified event information is transmitted in association with the content. May be.

  With this configuration, the event is identified using the time information associated with the content and the period information and event information stored in the third storage unit, so what event the retrieved content was captured at I know what it is. As a result, for example, it is possible to easily acquire content shot at a desired event or classify the acquired content according to the event.

  In the above configuration, the transmission unit includes one or more representative images respectively associated with one or more contents searched or determined by the search unit for transmission in the screen information, and In the terminal device, when a representative image is selected on the screen output based on the screen information, content associated with the selected representative image may be transmitted to the one terminal device.

  With this configuration, the terminal device can receive the content corresponding to the representative image from the server device simply by selecting the representative image on the screen, so that convenience is improved.

  In the above configuration, a fourth storage unit that stores a pair of a person identifier and a password, a person identifier stored in the fourth storage unit, and a person corresponding to the person identifier have a predetermined relationship. A fifth storage unit that stores a pair of person identifiers of one or more persons, and a third reception unit that receives a login request specifying a person identifier and a password via the terminal device, The transmission unit accepts a selection of content on the screen when a pair of a person identifier and a password specified in the login request is stored in the fourth storage unit, and A person corresponding to the person identifier specified in the login request, or a person identifier stored in the fifth storage unit in association with the person identifier specified in the login request. The content may be transmitted when a person who respond, the face image is included.

  With such a configuration, download is possible only for content in which a logged-in person or a person (family member, etc.) who has a predetermined relationship with the logged-in person is shown, so that both privacy protection and easy content acquisition can be achieved. It is done.

  In the above configuration, the sixth storage unit that stores price information regarding the price of the content, and the price corresponding to the content downloaded by the transmission unit to the storage unit of the terminal device with reference to the price information And a charging unit that charges the user of the terminal device.

  With this configuration, the content selected on the screen can be purchased.

  The above configuration may further include a creating unit that creates an album by arranging one or more contents selected on the screen on each page.

  With such a configuration, an album is automatically created simply by selecting content on the screen.

  In the above configuration, a second receiving unit that receives content transmitted from each of the plurality of terminal devices, a storage unit that stores the content received by the second receiving unit in the first storage unit, By extracting one or more face images from the content received by the second receiving unit and comparing the one or more face images with one or more face images stored in the second storage unit, The image processing apparatus may further include a processing unit that acquires at least one person identification information paired with the face image and holds the acquired one or more person identification information as a pair with the content.

  With this configuration, the content stored in the storage unit of each terminal device is uploaded to the server device, while the desired content is downloaded from the server device 1 to each terminal device, so that the content can be shared. In addition, each time content is uploaded, one or more person identification information paired with one or more face images included in the content is acquired and held in pairs with the content, The processing load when performing a person search is reduced.

  The search method according to the present invention is a search method performed by a first storage unit, a first reception unit, a search unit, and a transmission unit, and is an image transmitted to the first storage unit from a plurality of terminal devices. A first storage step in which one or more contents are stored, and the first reception unit receives a search instruction including person designation information, which is information relating to person designation, from one terminal device among the plurality of terminal devices. A receiving step, and the search unit searches for one or more contents including a face image that is a face image of a person corresponding to the person specifying information included in the search instruction received by the first receiving unit. A search method comprising: a search step; and a transmission step in which the transmission unit transmits one or more contents searched by the search unit to the one terminal device.

  With such a configuration, it is possible to easily obtain content in which the person is reflected from among a large number of content.

  A program according to the present invention provides a computer with a search instruction including a first storage unit that stores one or more contents that are images transmitted from a plurality of terminal devices, and person designation information that is information relating to person designation. A face image that is an image of a person's face corresponding to the person designation information included in the search instruction received by the first receiving unit and the search instruction received by the first receiving unit. A program for causing a search unit to search for one or more contents included therein and a transmission unit to transmit the one or more contents searched by the search unit to the one terminal device.

  With such a configuration, it is possible to easily obtain content in which the person is reflected from among a large number of content.

  According to the present invention, content in which a desired person is shown can be easily obtained.

Conceptual diagram of content sharing system in an embodiment Block diagram of the server device 1 Block diagram of the terminal device 2 A flowchart for explaining the operation of the server device 1 A flowchart for explaining the operation of the server device 1 A flowchart for explaining the operation of the server device 1 A flowchart for explaining the operation of the server device 1 Flow chart for explaining the operation of the terminal device 2 Flow chart for explaining the operation of the terminal device 2 Flow chart for explaining the operation of the terminal device 2 Diagram showing the schedule information The figure which shows the same person information and related person information Diagram showing the content Figure showing the same person position management table The figure which shows the screen based on the same screen information A figure showing a person being specified from the photo

  Hereinafter, embodiments of a server device and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

(Embodiment)
First, an outline of the present embodiment will be described. In the present embodiment, the server device 1 receives designation of a person via the terminal device 2, searches for content in which the person is reflected, and transmits the search result to the terminal device 2. The designation is performed by designating a person in the photograph output by the terminal device 2 or designating a person identifier. The content refers to content including a face image that is an image of the person's face, and can be uploaded from the terminal device 2 to the server device 1.

  In particular, for example, the server apparatus 1 narrows down the time information, and transmits a screen having one or more narrowed contents to the terminal apparatus 2. The content selection operation and the person designation operation are performed on the screen of the terminal device 2. The narrowing down may be performed with position information, or may be performed with time information and position information.

  Further, the server device 1 calculates the position of the person shown in the content from the position of the camera that shot the content, the shooting direction, the angle of view, the focal length, the height of the person shown in the content, and the like. Narrowing by position information is performed using position information indicating a person position. Information relating to the optical performance of the camera, such as the angle of view and the focal length, may be referred to as “optical information”.

  In addition, the server device 1 identifies an event that has been performed when the content is photographed, and transmits event information to the terminal device 2 in association with the content.

  In addition, the server device 1 transmits the content selected on the screen of the terminal device 2 to the terminal device 2. However, for example, only the content showing the logged-in person or a person in a predetermined relationship (for example, a parent-child relationship, a family relationship, a friend relationship, etc.) can be downloaded from the server device 1 to the terminal device 2. Then, the server device 1 charges the person for an amount corresponding to the transmitted content.

  The server device 1 can create an album with the selected content.

  Next, details of the present embodiment will be described. FIG. 1 is a conceptual diagram of a content sharing system according to the present embodiment. The content sharing system includes a server device 1 and a plurality of terminal devices 2.

  The server device 1 and the terminal device 2 are realized by a computer system, and include an MPU, a memory, a hard disk drive, a memory card slot, a communication module, an input device such as a keyboard, and an output device such as a display and a speaker. Have. Note that the hardware listed here is merely an example.

  The server device 1 is communicably connected to each terminal device 2 via the Internet, and realizes content sharing among users who use each terminal device 2. The content here is an image (photograph, video, etc.) taken by the user. As a result, each user can upload the content he / she owns from the terminal device 2 to the server device 1, while downloading the content showing a desired person from the server device 1 to the terminal device 2.

  The server device 1 will be described. FIG. 2 is a block diagram of the server device 1 in the present embodiment. The server device 1 includes a storage unit 11, a reception unit 12, a processing unit 13, and a transmission unit 14.

  The storage unit 11 includes a first storage unit 111, a second storage unit 112, a third storage unit 113, a fourth storage unit 114, a fifth storage unit 115, and a sixth storage unit 116. The receiving unit 12 includes a first receiving unit 121, a second receiving unit 122, and a third receiving unit 123. The processing unit 13 includes a search unit 131, a calculation unit 132, a billing unit 133, a creation unit 134, and a storage unit 135.

  The storage unit 11 stores contents, various information, a management table (described later), and the like. The content is an image uploaded from each terminal device 2. Further, the content may be obtained from outside the content sharing system. Various types of information and management tables are used to realize content sharing.

  The storage unit 11 can be realized by a non-volatile recording medium or a volatile recording medium. Moreover, the process in which content etc. are memorize | stored in the storage part 11 is not ask | required. For example, content or the like may be stored via a detachable recording medium, may be stored from the terminal device 2 via the communication line, or the like, or may be stored via an input device. Also good.

  The first storage unit 111 stores one or more contents that are images transmitted from the plurality of terminal devices 2. The content is associated with either time information indicating the time when the image was captured or position information indicating the position where the image was captured (in other words, the position of the camera that captured the image). .

  The second storage unit 112 stores one or more pieces of person information including a person face image and a person identifier for identifying the person. The face image is, for example, an image including a face, and an image of a portion determined to be a face is extracted from an image obtained by photographing a person through processes such as binarization and contour line extraction. The person identifier may be of any type as long as it is data that can identify a person, such as an account or an e-mail address.

  The third storage unit 113 stores at least period information indicating a period and event information indicating an event performed during the period in association with each other. This information is called schedule information. In addition to the period information and event information, the schedule information may further include location information indicating a location where the event is performed, entity information indicating an entity performing the event, and the like.

  The fourth storage unit 114 stores a person identifier and a password in pairs. Specifically, a person who uses the content sharing system for the first time inputs a personal identifier (for example, an account) related to himself and an arbitrarily determined password via the terminal device 2. The pair of the person identifier and password input in this way is stored in the fourth storage unit 114 as login information for the person to log in to the content sharing system.

  The fifth storage unit 115 stores a pair of the person identifier stored in the fourth storage unit 114 and the person identifier of one or more other persons. The other one or more persons are one or more persons having a predetermined relationship with a person (person) corresponding to the person identifier stored in the fourth storage unit 114. The predetermined relationship is, for example, a family relationship or a friend relationship. The fifth storage unit 115 stores a pair of a person identifier of the logged-in person and a person identifier related to the family and friends.

  The sixth storage unit 116 stores price information related to the price of the content. The price information describes the price of each content or a unit price common to the contents. A unit price that varies depending on the size of content, a discount rate according to the number of content, and the like may be described.

  The receiving unit 12 receives commands, data, requests (for example, search instructions, content, login requests) transmitted from the terminal device 2. In addition, although the receiving part 12 is implement | achieved by the communication module, the module which receives broadcast may be sufficient. That is, the command transmitted from the terminal device 2 is unicast toward the server device 1 but may be broadcast without specifying the transmission destination, and the receiving unit 12 receives the command transmitted in any manner. Is possible.

  In particular, the first receiving unit 121 receives a search instruction including person designation information that is information related to person designation. The person designation information is configured (created) in the terminal device 2 and is information for designating a person such as a person ID, a face image, or feature information thereof. In the person designation information, when the image has a plurality of persons (for example, a plurality of face images), the person is constituted (created) by a designation operation for one person among the plurality of persons. Alternatively, the person designation information may be configured (created) by a designation operation for designating a person ID in the terminal device 2. The feature information refers to information on facial features, for example, the relative position, size, shape, color, etc. of each part such as eyes, nose, and jaw.

  In addition, the second receiving unit 122 receives content transmitted from each terminal device 2.

  In addition, the third receiving unit 123 receives the login request transmitted from the terminal device 2. In the login request, a person identifier and a password are specified. The designation of the person identifier and the password is performed via the input unit of the terminal device 2. The third receiving unit 123 can be realized by a device driver of input means such as a keyboard, control software for a menu screen, or the like.

  The processing unit 13 executes various processes for realizing content sharing, such as search processing, person position calculation processing, billing processing, album creation processing, content storage processing, and the like. The processing unit 13 is realized by an MPU, a memory, and the like, and the processing procedure is realized by software, and the software is recorded on a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  In particular, the search unit 131 searches for one or more contents in which a person corresponding to the person designation information included in the search instruction received by the first receiving unit 121 is shown. Here, it is determined by face recognition that the face image of the person is included. In the search, face recognition may be performed in advance to identify persons appearing in each content, or face recognition may be performed in real time to sequentially identify persons appearing in each content.

  When the person designation information has a person identifier, the search unit 131 acquires a face image of a person who is paired with the person identifier included in the person designation information from the second storage unit 112, and is determined as a face image in advance. Search for content that partially includes images that have a relationship.

  Further, when the time information related to the time and the position information related to the position are associated with the content, and the search instruction includes the time information and the position information, the search unit 131 determines the content corresponding to the time information and the position information included in the search instruction. To do.

  In addition, the search unit 131 compares the similarity between the face image corresponding to the person designation information and each face image included in one or more contents stored in the first storage unit 111 with a threshold value. Thus, it is determined whether or not each content includes a face image corresponding to the person designation information and the time information and / or position information (or both) are used for the search, and the time information is also included in the search. The threshold value is changed when the position information is not used.

  For example, when time information and / or position information (or both) are used for the search, the threshold value may be relatively increased, while when neither time information nor position information is used for the search, the threshold value may be relatively small. If the threshold value is increased, it becomes difficult to determine whether each content includes a face image corresponding to the person designation information, and the accuracy of the person search is improved, but the face image corresponding to the person designation information is included. Although the number of contents to be determined is reduced, by using a refinement search based on time information or location information (a sum set of a person search result and a refinement search result is used as a search result of the search unit 131), This is because a search leak can be compensated.

  If the threshold value is decreased, it is not easy to determine whether each content includes a face image corresponding to the person designation information, and the number of contents determined to include a face image corresponding to the person designation information increases. Therefore, it is not necessary to use a refinement search based on time information or position information. However, if the number of contents judged to include a face image corresponding to the person designation information is increased too much by making the threshold smaller, the product set of the person search result and the refined search result is searched. The search result of the unit 131 may be used to eliminate an error in person search.

  For the one or more contents stored in the first storage unit 111, the calculation unit 132 calculates the position of each person shown in the content. That is, the time information associated with each content indicates the time when the content is captured by the camera, and the position information similarly indicates the position of the camera that captured the content. Each content is further associated with optical information regarding at least one of the angle of view and the focal length of the camera that captured the content, and the position of each person in the content is associated with each content. Calculated using position information and optical information. In this case, when determining one or more contents corresponding to the time information and the position information included in the search instruction, the search unit 131 uses the person position calculated by the calculation unit 132 as the position information included in the search instruction. Compare with the indicated position.

  The charging unit 133 refers to the price information stored in the sixth storage unit 116 and charges the user of the terminal device 2 for the amount corresponding to the content downloaded by the transmitting unit 14 to the terminal device 2. .

  The creation unit 134 creates an album using one or more contents selected on the screen. An album includes, for example, one or more pages, and one or more contents are arranged on each page. When content is arranged in a page, the content may be simply arranged in the upper left, upper right, lower left, and lower right, or may be arranged while adjusting the size and orientation according to predetermined layout information.

  For example, the creation unit 134 may sort one or more contents using time information corresponding to each content, and arrange the contents in the album in the sorted order. It is also possible to arrange one or more contents for each event by comparing time information corresponding to each content with time information included in the schedule information. Further, there is a method in which an image of a map is pasted on a page of an album, and one or more contents are arranged on the map using position information corresponding to each content.

  The accumulation unit 135 accumulates the content received by the second reception unit 122 in the first storage unit 111. As a result, the content uploaded from each terminal device 2 is stored in the first storage unit 111.

  The transmission unit 14 transmits various data such as content and screen information to the terminal device 2. The screen information is information relating to a screen for selecting content and specifying a person. In addition, although the transmission part 14 is implement | achieved by the communication means, you may implement | achieve by a broadcast means.

  In particular, the transmission unit 14 transmits one or more contents searched by the search unit 131. When the content includes shooting information, thumbnails, and the like in addition to the image data of the image shot by the camera, the transmission unit 14 may transmit only the image data.

  When the search unit 131 determines one or more contents corresponding to the time information (or time information and position information) included in the search instruction, the transmission unit 14 determines one or more contents determined by the search unit 131. Screen information, which is screen information for selecting content or specifying a person, is transmitted.

  Further, when time information related to time is associated with each content, when the transmission unit 14 transmits one or more contents searched by the search unit 131, Three pieces of period information stored in the storage unit 113 are compared to identify event information corresponding to the period information including the time when the content was photographed, and the identified event information is associated with the content. Send.

  In addition, when content is selected on the screen output based on the screen information in the terminal device 2, the transmission unit 14 downloads the selected content to the terminal storage unit 21 of the terminal device 2.

  However, the transmission unit 14 accepts the selection of the content on the screen when the pair of the person identifier and the password specified in the login request is stored in the fourth storage unit 114, and the content is received by the login request. The person corresponding to the specified person identifier or the person corresponding to the person identifier stored in the fifth storage unit 115 in association with the person identifier specified in the login request is shown (the face of the person). If the image is included), download the content.

  Next, the terminal device 2 will be described. FIG. 3 is a block diagram of the terminal device 2 in the present embodiment. The terminal device 2 includes a terminal storage unit 21, a terminal reception unit 22, a terminal processing unit 23, and a terminal transmission unit 24.

  The terminal storage unit 21 stores, for example, content transmitted (uploaded) to the server device 1, content received (downloaded) from the server device 1, screen information received from the server device 1, and the like. The terminal receiving unit 22 receives content, screen information, and the like transmitted from the server device 1.

  The terminal processing unit 23 outputs a screen based on the screen information, accepts a content selection operation or a person designation operation performed on the screen, or configures (creates) person designation information based on the accepted person designation operation. To do. Note that the screen output is output to a display or transmission to an external display or the like. The reception is a concept including reception of information input from an input unit such as a keyboard and a touch panel, reception of information transmitted via a communication line, reception of information read from a recording medium, and the like. . The terminal processing unit 23 is realized by, for example, an MPU.

  The terminal transmission unit 24 transmits the content stored in the terminal storage unit 21, a search instruction including person designation information, a login request to the content sharing system, and the like to the server device 1.

  Hereinafter, the operation of the content sharing system in the present embodiment will be described with specific examples. In addition, although the specific example given here is sharing of the image | photographed by the parents of the student at the athletic meet of an elementary school, it is only an illustration. The server apparatus 1 is installed in an operating entity of the content sharing system (for example, an elementary school (or an entrusted company or the like)) and operated by the staff. The terminal device 2 is a smartphone or the like, and is operated by student parents.

  First, operation | movement of the server apparatus 1 is demonstrated using the flowchart of FIGS.

  After the athletic meet, the staff inputs to the server device 1 schedule information indicating the athletic meet schedule and price information regarding the price of the content. The schedule information includes period information indicating a period and event information indicating an event performed during the period.

  (Step S401) First, the receiving unit 12 receives an input of schedule information via an input device. As shown in FIG. 11, the input schedule information is assigned a schedule ID by the processing unit 13 and is written in the third storage unit 113 in a format in which period information and event information are paired. The schedule ID is, for example, a record in the recording medium, and is referred to when necessary information is read from the third storage unit 113.

  (Step S402) Subsequently, the reception unit 12 receives input of price information. The price information includes, for example, information such as a common unit price for content, a unit price that varies depending on the size of the content, and a discount rate according to the quantity of content to be downloaded. The input price information is written into the sixth storage unit 116 by the processing unit 13. When the input of the schedule information and the price information is completed, the server device 1 becomes accessible from the terminal device 2. Parents who use the content sharing system for the first time access the server device 1 from the terminal device 2 via the Internet and register login information. Specifically, when the parents input their personal ID and password, a login information registration request is transmitted from the terminal device 2 to the server device 1.

  (Step S403) The processing unit 13 determines whether or not the receiving unit 12 has received the login information registration request transmitted from the terminal device 2. In the login information registration request, the terminal identification information of the terminal device 2 that is the transmission source, the person ID and password pair input via the input unit of the terminal device 2, and the person ID and password pair are logged in And a command requesting registration as information. If a login information registration request is received, the process proceeds to step S404. If not received, the process proceeds to step S405.

  (Step S <b> 404) In response to the login information registration request, the processing unit 13 writes the person ID and password pair included in the fourth storage unit 114. By performing the writing once or more, the fourth storage unit 114 stores one or more pairs of person ID and password. Thereafter, the process returns to step S403. The parents log in to the content sharing system by inputting the registered login information into the terminal device 2. The input login information is transmitted from the terminal device 2 to the server device 1.

  (Step S405) The processing unit 13 determines whether or not the third receiving unit 123 has received the login request transmitted from the terminal device 2. The login request is a command for requesting login of a person corresponding to the person ID corresponding to the terminal identification information relating to the terminal device 2 that is the transmission source, a pair of person ID and password input via the input unit of the terminal device 2 Including. If a login request is received, the process proceeds to step S406. If not received, the process proceeds to step S408.

  (Step S <b> 406) In response to the login request, the processing unit 13 determines whether login information corresponding to the pair of person ID and password included therein is stored in the fourth storage unit 114. If the corresponding login information is stored, it is determined that login of the person corresponding to the person ID (that is, the person who performed the login operation) is permitted, and the process proceeds to step 407. If the corresponding login information is not stored, it is determined that login is not permitted, and the process returns to step S403. When returning to step S403, for example, an error notification that the person ID or password is incorrect may be transmitted from the transmission unit 14 to the terminal device 2.

  (Step S <b> 407) The processing unit 13 updates the login management table in response to allowing the login of the person who performed the login operation. The login management table is a table for managing the person who is logged in to the content sharing system and the terminal device 2 operated by the person. The person ID of the person and the terminal identification information of the terminal device 2 Are described in pairs. The update performed by the processing unit 13 is a process of writing the person ID and password pair included in the login request to the login management table.

  As a result, the pair of the parent's person ID and the IP address of the terminal device 2 is written in the login management table as the person ID of the logged-in person and the terminal identification information.

  Next, the parents register personal information about themselves via the input means of the terminal device 2. Specifically, when the parents specify their face images from the content stored in the terminal storage unit 21, a personal information registration request is transmitted from the terminal device 2 to the server device 1.

  (Step S <b> 408) The processing unit 13 determines whether or not the receiving unit 12 has received the personal information registration request transmitted from the terminal device 2. The person information registration request includes the terminal identification information related to the terminal device 2 that is the transmission source, the person ID input via the input unit of the terminal device 2, and a face image (or a face image captured from the photograph of the person) (or A pair of face image feature information) and a command requesting registration of the person ID and face image pair as person information. If a personal information registration request is received, the process proceeds to step S409, and if not received, the process proceeds to step S410.

  (Step S409) In response to the person information registration request, the processing unit 13 writes the pair of the person ID and the face image included therein in the second storage unit 112. As a result, a pair of the parent's personal ID and the parent's face image (or feature information) is stored in the second storage unit 112. By performing the writing once or more, as shown in FIG. 12A, the second storage unit 112 stores one or more pairs of person ID and face image. Thereafter, the process returns to step S403.

  Next, the parents register personal information (related person information) related to a person (my child who is a student at an elementary school) who has a predetermined relationship with him. Specifically, when the parents specify the face image of the child from the contents stored in the terminal storage unit 21 and input the person ID (related person ID) of the child, the terminal device 2 The related person information registration request is transmitted from the server apparatus 1 to the server apparatus 1.

  (Step S410) The processing unit 13 determines whether or not the receiving unit 12 has received the related person information registration request transmitted from the terminal device 2. In the related person information registration request, terminal identification information related to the terminal device 2 that is the transmission source, the person ID of the person who operates the terminal device 2, and one or more persons who have a predetermined relationship with the person are input. And a command requesting registration of the related person ID and face image pair as related person information corresponding to the person ID of the person. If a related person information registration request is received, the process proceeds to step S409, and if not received, the process proceeds to step S410.

  (Step S411) In response to the related person information registration request, the processing unit 13 first determines whether or not the personal information of the person is registered. If the person ID and face image pair included in the related person information registration request is stored in the second storage unit 112, it is determined that the person information is registered, and the process proceeds to step S412. . If the pair of the person's person ID and face image is not stored in the second storage unit 112, it is determined that the person's person information is not registered, and the process returns to step S403. When returning to step S403, for example, a notification that prompts the user to register his / her personal information may be transmitted from the transmission unit 14 to the terminal device 2.

  (Step S412) Next, the processing unit 13 uses the pair of one or more related person IDs and face images included in the related person information registration request as the related person information corresponding to the person ID of the fifth storage unit. 115 is written. For example, a pair of a related person ID and a face image corresponding to the related person ID is written in the fifth storage unit 115 with respect to the parent person ID. By performing the writing once or more, the fifth storage unit 115 stores the related person ID and the person ID corresponding to the person ID stored in the second storage unit 112 as shown in FIG. The result is that a pair of face images is stored. Thereafter, the process returns to step S403.

  In the fifth storage unit 115, for each person ID stored in the second storage unit 112, instead of the pair of the related person ID and the face image, the related person ID and the feature information of the face image are stored. Pairs may be stored. Alternatively, in addition to the pair of the person ID and the face image (or its feature information), the second storage unit 112 further stores a pair of the related person ID and the face image (or its feature information). The unit 115 may store a pair of a person ID and a related person ID.

  The fifth storage unit 115 includes a pair of a related person ID and a face image for a family (a spouse, a parent, a brother, etc.), a friend, etc., as well as a parent's child (our child who attends elementary school). It may be stored. When writing the related person ID and the face image in the fifth storage unit 115, the processing unit 13 has a predetermined relationship (for example, a parent-child relationship, for example) between the person logged into the system and the person that the person is trying to register. Family confirmation, friendship, etc.) may be confirmed. The confirmation process can be performed, for example, by collation with personal information registered in advance.

  This completes registration of information necessary for using the content sharing system. Thereafter, the parents upload the content stored in the terminal storage unit 21 of the terminal device 2 to the server device 1, or their children are selected from the content stored in the first storage unit 111 of the server device 1. It is possible to search the content that is shown and download it to the terminal storage unit 21 of the terminal device 2.

  (Step S413) The processing unit 13 determines whether or not the second receiving unit 122 has received the content transmitted from the terminal device 2. The content includes terminal identification information related to the terminal device 2 that is the transmission source, and the person ID of the person who performed the upload operation of the content. If the content is received, the process proceeds to step S414, and if not received, the process proceeds to step S418.

  (Step S414) The storage unit 135 stores the received content in the first storage unit 111 in association with the content ID for identifying the content. As a result of such accumulation being performed once or more, as shown in FIG. 13, one or more pairs of content ID and content are stored in the first storage unit 111.

  The content includes, for example, image data of an image (photo or video) taken by a camera, shooting information that is information on shooting conditions of the camera that shot the image, and a representative image (for example, a thumbnail) that represents the image. May be an image file. However, the content does not necessarily have the image file format, and may be, for example, image data itself. When the content is image data itself, shooting information, thumbnails, and the like may be added to the content. The thumbnail does not necessarily have to be a reduced image depending on the size of the image data, the size of the screen, the number of contents in the screen, and the like.

  The shooting information includes time information indicating the time when a photograph or the like is taken by the camera, position information indicating the position of the camera, optical information regarding the angle of view and focal length of the camera when the photograph or the like is taken. The position information is information acquired from, for example, GPS, and describes information indicating position coordinates expressed as (latitude, longitude) and shooting directions expressed as southeast and north-northwest as an example. However, the position information only needs to be information that can specify the position and shooting direction of the camera, and the expression format is not limited.

  (Step S415) The processing unit 13 specifies a person shown in the content. Specifically, for example, by performing processing such as binarization and contour detection on image data such as a photograph, one or more portions of image data determined as a face image are extracted from the image data, A similarity is obtained between the extracted image data of each part and each face image data stored in the second storage unit 112.

  If there is one or more combinations whose similarity is higher than the threshold value, the image data is regarded as including the face image data stored in the second storage unit 112, and from the second storage unit 112, One or more person IDs corresponding to the face image data are acquired. In this way, identification of one or more persons appearing in the content is specified.

  (Step S416) The calculation unit 132 calculates the specified person position using the position information and optical information associated with the content. Specifically, the position information describes the position and shooting direction of the camera that shot the content, and the optical information describes the size of the image sensor, the focal length of the lens, and the like.

  First, the size of the person image formed on the image sensor is obtained from the size of the image sensor and the ratio of the size of the person image (the area determined to be) to the frame size of the content.

  Next, the distance from the camera to the person is obtained from the focal length, the height of the person, and the size of the person image formed on the image sensor. The average height is used for the height of the person. For example, if the focal length is 300 mm and the average height of a person is 140 cm (= 1.4 m), and the size of the person image formed on the image sensor is 6 mm, the distance from the camera to the person is 1. 4 × 300/6 = 70 m. Therefore, the position moved by 70 m from the camera position in the shooting direction is the person position. When the angle of view is described in the optical information instead of the focal length, the focal length is obtained by a known calculation formula based on the angle of view and the size of the image sensor, and then the same calculation as above is performed. Just do it.

  (Step S417) The processing unit 13 updates the person position management table. The person position management table is a table for managing the position of a person shown in the content for one or more contents stored in the first storage unit 111. An example of the person position management table is shown in FIG. In the person position management table, for each content, the person ID of the person shown and the position of each of the one or more persons are described as a pair. In the update of the object position management table, the content ID of the content accumulated in step S414, the person ID specified in step S415, and the position of the person calculated in step S416 are written into the person position management table. It is processing. Thereafter, the process returns to step S403.

  In this way, each time the parents upload content from the terminal device 2, the pair of the identified person's person ID and the calculated person's position is added to the person position management table for the uploaded content. .

  Similarly, the other parents upload the content from their terminal device 2, so that the first storage unit 111 of the server device 1 stores a large number of content shot at the athletic meet and is stored in the person position management table. Is a result of describing a pair of one or more person IDs and one or more person positions shown in each content.

  In this content sharing system, there is a possibility that my child appears in the content uploaded by other parents. The parents perform a search instruction operation that specifies the person ID of my child via the input means of the terminal device 2, so that the content stored in the first storage unit 111 of the server device 1 You can search for content that has children. For example, when the parents perform a search instruction operation specifying the person ID of the child, a search instruction specifying the person ID is transmitted from the terminal device 2 to the server device 1.

  (Step S418) The processing unit 13 determines whether or not the first receiving unit 121 has received a search instruction specifying a person ID. The search instruction specifying the person ID will be described with reference to the flowchart of the terminal device 2. If a search instruction specifying a person ID is received, the process proceeds to step S419. If not received, the process proceeds to step S426.

  (Step S419) The processing unit 13 acquires a face image corresponding to the designated person ID from the second storage unit 112 or the fifth storage unit 115. When the related person is designated, the processing unit 13 acquires a face image corresponding to the related person ID from the fifth storage unit 115. For example, when the person ID of the parents is specified, the face image corresponding to the person ID is acquired from the second storage unit 112. Although not shown, if the designated person ID is not stored in the second storage unit 112 or the fifth storage unit 115, a message to that effect is transmitted from the transmission unit 14 After the original transmission, the process may return to step S401.

  (Step S420) The search unit 131 searches for one content partly including an image having a predetermined relationship with the acquired face image. Specifically, first, the person position management table (see FIG. 14) is searched, and one or more contents exist that include the designated person ID in the “person ID of the person in the picture”. Then, the search result for the search request is determined as the one or more contents. For example, in response to a search request designating a parent child's child person ID (related person ID), two contents “1” and “1” including the related person ID in the “person ID of the person in the picture”. Since “2” exists, the two contents “1” and “2” are determined as search results.

  Next, the same processing as that executed in step S415 is sequentially performed on the image data of the content stored in the first storage unit 111. This is because step S415 is a process that is performed on the content when new content is received, and the person ID of the person who registered the person information later may not be described in the person position management table. .

  That is, by performing processing such as binarization and contour detection on the image data of the content “1”, image data of one or more portions determined as face images is extracted from the image data, and each extracted A similarity is obtained between the image data of the part and the acquired image data of the face image. Similarly, for the content “2” and after, the similarity is obtained, and when there is one or more contents including a portion whose similarity is higher than the threshold, the search unit 131 displays the search result for the search request as the one or more contents. Decide on content. Note that one or more contents determined here are also subjected to the same person position calculation as in step S416 and reflected in the person position management table.

  By the way, in the person search using the face image as described above, for example, when the recognition rate of the face image is low, there is a possibility that a useful search result cannot be obtained. Therefore, in this content sharing system, narrowing down by time and / or position is used together. That is, the parents can specify the time and / or position when performing the search instruction as described above. When specifying the time, it is accepted that a value such as “9:30” is input via an input means such as a touch panel. When designating the position, for example, a map screen of an athletic field may be output on the display of the terminal device 2 and an operation for designating a desired point or range on the map screen may be received on the touch panel. When the time is specified, the terminal device 2 transmits the time information related to the specified time included in the search instruction. When the position is specified, the terminal device 2 transmits the time position related to the specified position in the search instruction.

  (Step S421) The processing unit 13 determines whether or not the search instruction includes time information. If the search instruction includes time information, the process proceeds to step S422; otherwise, the process proceeds to step S423.

  (Step S422) The search unit 131 narrows down the time information included in the search instruction. This narrowing is a process of determining one or more contents corresponding to the time information included in the search instruction from the contents stored in the first storage unit 111. The content corresponding to the time information included in the search instruction means that the time indicated by the time information included in the content coincides with or is close to the time indicated by the time information included in the search instruction (the difference between the two times is used as a threshold). Content is not satisfied. Thereafter, the process proceeds to step S423.

  (Step S423) The processing unit 13 determines whether or not the search instruction includes position information. If the search instruction includes position information, the process proceeds to step S424, and if not, the process proceeds to step S425.

  (Step S424) The search unit 131 narrows down the position information included in the search instruction. Corresponding to this position information means that the position indicated by the position information coincides with or is close to the position indicated by the position information included in the search instruction (the difference between the two positions is less than the threshold value). Further, this narrowing down is performed from among the contents stored in the first storage unit 111 when step S422 is skipped, or from the contents narrowed down by time information when step S422 is executed. Process for determining content corresponding to position information included in search instruction (process for determining content corresponding to time information and position information included in search instruction from contents stored in first storage unit 111 ). For example, when the position “north latitude, east longitude” is designated, the content photographed within an arbitrary size or shape centered on the designated position “north latitude, east longitude” with reference to the position information included in each content. Determined by search results.

  The position specified above is a position related to the camera that captured the content, but a position related to the person captured by the camera can also be specified. In this case, the search unit 131 refers to the “position of a person in the person” in the person position management table and determines one or more contents in which a person at or near the specified position is shown. Thereafter, the process proceeds to step S425.

  (Step S425) The transmission unit 14 is a screen having the content determined by the search unit 131, and the screen information that is the screen information for selecting content or specifying a person is used as the terminal device 2 that is the transmission source of the search instruction. Send to. The content here is a union of the content determined in step S420, the content determined in step S422, and the content determined in step S424.

  The union is used to obtain useful search results even when the recognition rate of the face image is low. However, a product set may be adopted, and for example, it is possible to search for a photograph in which “my child who runs through the goal in a race” is shown. Further, each time a search is performed, either a union set or a product set may be selected and adopted.

  When at least one of time information and position information is included in the search instruction, compared to a case where the time information and position information are not included in the search instruction, a person search using a face image is performed. You may raise a threshold value. For example, by performing this comparison and raising the threshold value, a significant search result may be obtained using time information or position information even if the recognition rate of the face image is low.

  In general, the search unit 131 changes the threshold value depending on whether or not at least one of time information and position information is used for the search. Thereby, even if the recognition rate of a face image is low, it becomes easy to obtain a significant search result.

  The screen information transmitted from the transmission unit 14 is described in a markup language, and thumbnail image data corresponding to the determined content is embedded. The content here includes thumbnail image data, and the thumbnail image data is acquired from the first storage unit 111.

  Preferably, the transmission unit 14 acquires event information corresponding to the determined content from the third storage unit 113, and controls the acquired event information to be displayed in association with the content on the screen. Send control information in the screen information. The event information corresponding to the content is, for example, information on events that have been performed during a period including the time when the content was captured. The event information can be specified by comparing the time information included in the content with each period information stored in the third storage unit 113. After the screen information is transmitted, the process returns to step S403.

  In the terminal device 2, a screen as shown in FIG. 15 is displayed on the display based on the screen information received from the server device 1. On this screen, the content (thumbnail) determined by the search unit 131, schedule information (period information (9:30 to 10:00) and event information (opening ceremony)) displayed in association with each content, Person ID input field, time input field, position input field, search button for instructing search execution, album creation button for instructing album creation, and download button for instructing download Including. The content includes content corresponding to a specified person ID or a specified face image, content corresponding to a specified time, content corresponding to a specified position, and the like. In addition, since schedule information is displayed in association with the content, it is possible to know what time the content was captured during the event.

  The parents can select content from the screen shown in FIG. 15 and download it to the terminal storage unit 21 of the terminal device 2 or create an album. It is also possible to search again by designating a face image from the selected content. It is also possible to search by inputting another person ID. When performing a search, time and position can be specified as a narrowing condition. When content is selected, a selection result notification in which the terminal identification information related to the terminal device 2 itself and the content ID of the content are described is created.

  When the selected content is to be downloaded, the parents perform a download instruction operation following the content selection. In response, the terminal device 2 transmits a selection result notification including a download instruction to the server device 1. In addition, when it is desired to create an album with the selected content, the parents issue an album creation instruction following the content selection. In response to this, the terminal device 2 transmits a selection result notification including an album creation instruction to the server device 1.

  When it is desired to search for an arbitrary person among one or more persons shown in the selected content, the parents perform a search operation specifying the face image of the person following the content selection. In response to this, a selection result notification including a search instruction specifying a face image is transmitted from the terminal device 2 to the server device 1.

  (Step S426) The processing unit 13 determines whether or not the receiving unit 12 has received the content selection notification transmitted from the terminal device 2. The content selection notification includes terminal identification information related to the terminal device 2 that is the transmission source, a content ID indicating the content selected on the screen, and instruction information that is an instruction regarding the handling of the content. The instruction information that can be included in the content selection notification is instruction information indicating either a download instruction, an album creation instruction, or a search instruction that specifies a face image. If a content selection notification is received, the process proceeds to step S427. If not received, the process returns to step S403.

  (Step S427) The processing unit 13 determines whether or not the received content selection notification includes a download instruction. If the content selection notification includes a download instruction, the process proceeds to step S428. If the content selection notification does not include a download instruction, the process proceeds to step S432.

  (Step S428) The processing unit 13 determines whether or not the selected content, that is, the content that is the target of the download instruction, is the content that shows the person who selected the content or the person concerned.

  Specifically, first, the login management table is searched with the terminal identification information included in the content selection notification, and the person ID of the logged-in person, that is, the person who selected the content is acquired. Next, the fifth storage unit 115 is searched to obtain a related person ID corresponding to a related person who has a predetermined relationship with the person concerned. Further, the person position management table is searched with the content ID included in the content selection notification, and at least the acquired person ID or related person ID is included in the “person ID of the person shown” corresponding to the content ID. It is determined whether or not one is included. The content in which at least one of the person ID or the related person ID is included in the “person ID of the person in the picture” is regarded as the content in which the person or the related person is reflected, and the person Content that does not contain either an ID or a related person ID is regarded as content that does not show the person or related person.

  If all of the selected content is content in which the person or related person is shown, the process proceeds to step S429, and if at least one of the selected contents is content in which the person or the related person is not shown, Then, the process proceeds to step S431.

  (Step S429) The transmission unit 14 downloads the selected content to the terminal storage unit 21 of the terminal device 2. For example, if at least one of the parents and the child is reflected in the content selected by the parents, the content is downloaded from the server device 1 to the terminal storage unit 21. The content is not downloaded. This makes it possible to share content in consideration of privacy.

  (Step S430) The charging unit 133 calculates the price of the downloaded content based on the price information stored in the sixth storage unit 116, and charges the price of the person who has selected the content. Note that the download may be performed simultaneously with the settlement or after the settlement. Thereafter, the process returns to step S403.

  (Step S431) The transmission unit 14 transmits an error notification indicating that a content that does not include the person or the related person is specified to the terminal device 2 that is the transmission source of the content selection notification. Thereafter, the process returns to step S403.

  (Step S432) The processing unit 13 determines whether or not the received content selection notification includes an album creation instruction. If the content selection notification includes an album creation instruction, the process proceeds to step S433. If the content selection notification does not include a download instruction, the process proceeds to step S435.

  (Step S433) The creation unit 134 creates an album by arranging the selected content on each page. The storage unit 11 stores layout information that defines areas in which content can be placed in each page. The creation unit 134 places content in a page according to the layout information (adjusting the size and orientation). I will do it. This makes it possible to create a good-looking album. The content may be simply arranged without using layout information. Thereby, an album can be created easily. Alternatively, a plurality of layouts may be presented via the terminal device 2 and an album may be created using the layout selected by the user. As a result, an album having a layout suitable for the user's preference can be created.

  (Step S434) The transmission unit 14 transmits album information, which is information on the album created by the creation unit 134, to the terminal device 2 that is the transmission source of the content selection notification. The album information includes, for example, image data of the selected content, image data of the background of each page, and layout information describing which content is arranged in which area of each page. In the terminal device 2, an album is configured based on such album information, and an album screen is output. The album information may be a bitmap image of the album created by the creation unit 134. In this case, the terminal device 2 outputs an album screen based on the album information. After transmitting the album information, the process returns to step S403. If album creation is to be performed for a fee, after the album information is transmitted, the process may return to step S403 through the billing process in step S403. In the billing process in this case, the price of the album may be calculated based on the number of used contents, for example, or may be calculated according to the number of pages of the album, and the price calculation method is not limited.

  (Step S435) The processing unit 13 determines whether or not the first receiving unit 121 has received a search instruction specifying a face image. The designation of the face image is performed in the terminal device 2 that has received the screen information transmitted in step S425. In the terminal device 2, a screen based on the screen information is output, and when a face image of an arbitrary person is specified from the content in the screen, a search instruction specifying the face image is transmitted to the server device 1. The search instruction specifying the face image will be described with reference to the flowchart of the terminal device 2. If a search instruction designating a face image is received, the process proceeds to step S420, and if not received, the process returns to step S403. In step S420, the content including the face image is searched, and after further narrowing down in steps S421 to S424, the screen information of the screen including one or more determined contents is transmitted.

  In the flowcharts of FIGS. 4 to 7, the process is terminated by power-off or a process termination interrupt.

  Next, operation | movement of the terminal device 2 is demonstrated using the flowchart of FIGS. In addition, the matter demonstrated accompanying operation | movement description of the server apparatus 1 is abbreviate | omitted.

  (Step S801) The terminal processing unit 23 determines whether or not a login information registration operation has been performed. The login information registration operation means that a person who wants to set login information of the content sharing system inputs a personal ID related to himself and a password determined by himself via an input device, and sets the personal ID and password. This is an operation for instructing the pair to be registered as login information. If the login information registration operation is performed, the process proceeds to step S802, and if not, the process proceeds to step S803.

  (Step S <b> 802) The terminal transmission unit 24 transmits a login information registration request to the server device 1. In the login information registration request, a command for requesting registration of the terminal identification information of the terminal device 2 itself, a person ID and password pair input via the input device, and the person ID and password pair as login information And are included. Thereafter, the process returns to step S801.

  (Step S803) The terminal processing unit 23 determines whether or not a login operation has been performed. The login operation is an operation in which a person who wants to log in to the content sharing system inputs a person ID and password pair registered in advance and requests login. If the login request operation is performed, the process proceeds to step S804, and if not, the process proceeds to step S805.

  (Step S804) The terminal transmission unit 24 transmits a login request to the server device 1. The login request includes the terminal identification information of the terminal device 2 itself, the input person ID and password pair, and a command for requesting login of the person corresponding to the person ID. Thereafter, the process returns to step S801.

  (Step S805) The terminal processing unit 23 determines whether or not a personal information registration operation has been performed. The person information registration operation refers to, for example, a person who logs in for the first time designates his / her face image from content stored in the terminal storage unit 21 using an input device such as a touch panel, and his / her person ID and designated face. This is an operation for requesting registration of a pair of images as person information. If the personal information registration operation is performed, the process proceeds to step S806, and if not, the process proceeds to step S807.

  (Step S806) The terminal transmission unit 24 transmits a personal information registration request to the server device 1. In the person information registration request, the terminal identification information of the terminal device 2 itself, the person ID input at the time of login and the face image pair specified as described above, and the person ID and face image pair are used as person information. Command to request registration. Thereafter, the process returns to step S801.

  (Step S807) The terminal processing unit 23 determines whether or not a related person information registration operation has been performed. The related person information registration operation refers to, for example, face images of persons (family members, close friends, etc.) who are in a predetermined relationship with the logged-in person from content stored in the terminal storage unit 21, for example. This is an operation of inputting a person ID (related person ID) of a person while designating and requesting registration of a pair of related person ID and face image related to the person as his / her related person information. If the related person information registration operation is performed, the process proceeds to step S808; otherwise, the process proceeds to step S809.

  (Step S808) The terminal transmission unit 24 transmits a related person information registration request to the server apparatus 1. In the related person information registration request, the terminal identification information of the terminal device 2 itself, the person ID of the logged-in person, and the related person ID and face image input regarding one or more persons having a predetermined relationship with the person A pair and a command for requesting registration of the related person ID and face image pair as related person information corresponding to the person ID of the person are included. Thereafter, the process returns to step S801.

  (Step S809) The terminal processing unit 23 determines whether or not a content upload operation has been performed. The content upload operation is an operation for uploading content stored in the terminal storage unit 21 to the first storage unit 111 of the server device 1. If a content upload operation is performed, the process proceeds to step S810; otherwise, the process proceeds to step S811.

  (Step S810) The terminal transmission unit 24 transmits the content to the server device 1. The content includes the terminal identification information of the terminal device 2 itself and the person ID of the logged-in person. Thereafter, the process returns to step S801.

  (Step S811) The terminal processing unit 23 determines whether or not a search instruction operation specifying a person ID has been performed. The search instruction operation in which the person ID is designated is an operation for instructing the execution of the search by inputting the person ID of the person to be searched with the input device. If a search instruction operation specifying a person ID is performed, the process proceeds to step S812, and if not, the process proceeds to step S818.

  (Step S812) The terminal processing unit 23 creates a search instruction. This search instruction includes terminal identification information related to the terminal device 2 itself, person designation information including the input person ID, and a search execution instruction. Thereafter, the process proceeds to step S813.

  (Step S813) The terminal processing unit 23 determines whether or not a time designation operation has been performed. The time designation operation is an operation for designating time as a condition for narrowing down contents. If the time designation operation is performed, the process proceeds to step S814; otherwise, the process proceeds to step S815.

  (Step S814) The terminal transmission unit 24 adds time information indicating the specified time to the created search instruction. Thereafter, the process proceeds to step S815.

  (Step S815) The terminal processing unit 23 determines whether or not a position specifying operation has been performed. The position specifying operation is an operation for specifying a position as a condition for narrowing down contents. If the position designation operation is performed, the process proceeds to step S816, and if not, the process proceeds to step S817.

  (Step S816) The terminal transmission unit 24 adds position information indicating the specified position to the created search instruction. Thereafter, the process proceeds to step S817.

  (Step S817) The terminal transmission unit 24 transmits the created search instruction. Search instructions that specify a person ID include those with time information added, those with position information added, those with time information and position information added, and those with neither time information nor position information added There are four types. There are also four types of search instructions that specify a face image. Thereafter, the process returns to step S801.

  (Step S818) The terminal processing unit 23 determines whether the terminal receiving unit 22 has received the screen information transmitted from the server device 1. If the screen information is received, the process proceeds to step S819. If not received, the process returns to step S801.

  (Step S819) The terminal processing unit 23 outputs a screen based on the received screen information to an output device such as a display. In this screen, the content determined by the search unit 131 of the server device 1 in accordance with the search instruction transmitted in step S817 is arranged. In addition, event information such as “Opening Ceremony” is displayed on the screen in association with each content. The content includes content corresponding to a specified person ID or a specified face image, content corresponding to a specified time, content corresponding to a specified position, and the like. By displaying the event information in association with the content, it is possible to know what event was taken during the event.

  (Step S820) The terminal processing unit 23 determines whether or not a content selection operation has been performed. The content selection operation is an operation for selecting an arbitrary content from one or more contents on the screen via an input device such as a touch panel. If the content selection operation is performed, the process proceeds to step S821, and if not, the process returns to step S801.

  (Step S821) The terminal processing unit 23 determines whether or not a download instruction operation has been performed. The download instruction operation is an operation for instructing the selected content to be downloaded from the server device 1 to the terminal storage unit 21. If a download instruction operation is performed, the process proceeds to step S822, and if not, the process proceeds to step S823.

  (Step S822) The terminal transmission unit 24 transmits a selection result notification including a download instruction to the server device 1. In the selection result notification including the download instruction, terminal identification information regarding the terminal device 2 itself and the content ID of the selected content are described. Thereafter, the process returns to step S801.

  (Step S823) The terminal processing unit 23 determines whether or not a search instruction operation specifying a face image has been performed. The search instruction operation in which a face image is designated is an operation for instructing execution of a search by designating a face image of a person to be searched from the content selected by the content selection operation using an input device. Specifically, a screen based on the screen information received in step S818 is output to a display or the like, and an operation for designating a desired person's face image from the content in the screen is accepted by the input device. When there are a plurality of face images in the content, an operation for designating a face image among them is accepted. If a search instruction operation specifying a face image is performed, the process proceeds to step S824; otherwise, the process proceeds to step S825.

  (Step S824) The terminal processing unit 23 creates a search instruction. The search instruction here includes person designation information including the designated face image (or its feature information) and a search execution instruction. Specifically, the terminal processing unit 23 extracts a face image of a designated person from the content, and creates a search instruction including the extracted face image and a search execution instruction. Note that feature information indicating the feature of the face image may be created from the extracted face image, and a search instruction including the created feature information and a search execution instruction may be created. Thereafter, the process proceeds to step S813. The search instruction that specifies the face image created in this way is transmitted to the server apparatus 2 in step S817 after time information or position information is added as necessary in step S813 and subsequent steps S814 to S816. .

  (Step S825) The terminal processing unit 23 determines whether or not an album creation instruction operation has been performed. The album creation instruction operation is an operation for instructing to create an album using the content selected by the content selection operation. If an album creation instruction operation is performed, the process proceeds to step S826, and if not, the process proceeds to step S827.

  (Step S826) The terminal transmission unit 24 transmits a selection result notification including an album creation instruction to the server device 1. In the selection result notification including the album creation instruction, terminal identification information regarding the terminal device 2 itself and the content ID of the selected content are described. Thereafter, the process returns to step S801.

  (Step S827) The terminal processing unit 23 determines whether or not the terminal receiving unit 22 has received the album information transmitted from the server device 1. If album information is received, the process proceeds to step S828, and if not received, the process returns to step S801.

  (Step S828) The terminal processing unit 23 outputs an album screen based on the received album information to an output device such as a display. Specifically, the album information includes image data of the selected content, background image data of each page, and layout information describing which content is arranged in which area of each page. The terminal processing unit 23 configures an album image based on such album information, and outputs the album image to an output device.

  In the flowcharts of FIGS. 8 to 10, the process is terminated by powering off or interruption for aborting the process.

  As described above, according to the present embodiment, when a desired person is designated, the server device 1 retrieves content including a face image that is an image of the person's face, and the retrieved one or more contents Since it is transmitted, the content in which the person is reflected can be easily obtained from a large number of contents.

  Further, in the present embodiment, simply by specifying a person ID of a desired person via the terminal device 2, the server apparatus 1 acquires the face image of the person and searches for content including the face image. In addition, the person designation information may be configured by a designation operation for a person in the image output from the terminal device 2, and therefore, even if a desired person in the photograph is designated, the content in which the person is reflected can be searched. Is done. The designation operation may be an operation for designating one person from a plurality of persons in the image. For this reason, convenience is improved.

  In the present embodiment, a screen including the content determined by the search by the face image, the content determined by the narrowing by the time information, the content determined by the narrowing by the time information, and the like is output from the terminal device 2, and this Since content can be selected or a person can be designated on the screen, the possibility that desired content can be obtained increases.

  Further, in the present embodiment, the position information of the person shown in the content is calculated from the position information of the camera corresponding to the content, the optical information, and the like. Therefore, it is possible to narrow down by the position of the person as well as the position of the camera.

  In addition, according to the present embodiment, an event is specified by using time information associated with the content, period information and event information stored in the schedule storage unit, so what event the searched content is You can see if it was taken with. As a result, for example, it is possible to easily acquire content shot at a desired event or classify the acquired content according to the event.

  In the present embodiment, the terminal device 2 can select and download desired content on the screen to download the content from the server device 1 and automatically create an album. Increases nature. And since the contents that can be downloaded are limited to the contents of the logged-in person or their family members, privacy is maintained and security is improved.

  Moreover, in this embodiment, while the content stored in each terminal device 2 is uploaded to the server device 1, while the desired content is downloaded from the server device 1 to each terminal device 2, the content can be shared. In addition, each time a content is uploaded, a person ID that is paired with a face image included in the content is acquired and stored in pairs with the content, thereby performing processing when performing a person search The load is reduced.

  In this embodiment, the operation of designating a desired person in the photograph at the terminal device 2 at the time of the search instruction is the content selected from the screen based on the screen information received from the server device 1 (see FIG. 15). (See Steps S823 and S824), but is not limited to this, and may be performed on content selected from the terminal storage unit 21, for example.

  Specifically, in the terminal device 2, for example, when a photo is selected from the terminal storage unit 21, the terminal processing unit 23 outputs the photo to the display. Next, as shown in FIG. 16, when a person is designated from the photograph displayed on the display via the input device, the terminal processing unit 23 extracts the face image of the designated person from the photograph image. To do.

  Then, the face image extracted from the image of the photograph is compared with the face image stored in the second storage unit 112 or the related person storage unit 115 to obtain a person ID corresponding to the designated person. If the person position management table (see FIG. 14) is searched using the person ID thus obtained, the ID of the content in which the person corresponding to the person ID is shown can be obtained.

  Alternatively, it is also possible to sequentially extract images of portions determined to be faces from each content stored in the first storage unit 111, and compare each face image extracted from the image of the photograph one by one. It is possible to obtain the content ID of the content in which the person corresponding to is shown.

  Also, each time content is uploaded from the terminal device 2, the person ID paired with the face image included in the content is acquired from the second storage unit 112 and stored in the person position storage table as a pair with the content. By doing so, the processing load when performing a person search is reduced.

In addition, the processing in the server device 1 in this embodiment, the processing in the plurality of terminal devices 2, and the communication processing between the server device 1 and the plurality of terminal devices 2 via the Internet are realized by software. Also good. The software may be distributed by downloading or the like, or recorded on a recording medium and distributed. This also applies to other embodiments in this specification.
The software that realizes the processing in the server device 1, the processing in the plurality of terminal devices 2, and the communication processing between the server device 1 and the plurality of terminal devices 2 via the Internet in the present embodiment is as follows. It is built with a simple program.

  As shown in FIGS. 1 and 2, the software according to the present embodiment is a program for executing communication between the server device 1 and the plurality of terminal devices 2 via the Internet. The server device 1 is connected to the storage unit 11. It is a program that causes the reception unit 12, the processing unit 13, and the transmission unit 14 to function, and is a program that causes the terminal device 2 to function as the terminal storage unit 21, the terminal reception unit 22, the terminal processing unit 23, and the terminal transmission unit 24.

  In the above program, in a transmission step for transmitting information, a reception step for receiving information, etc., processing performed by hardware, for example, processing performed by a communication module such as a modem or an interface card (hardware in the transmission step) Processing that can only be done in) is not included. Further, the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  Further, in each of the above embodiments, it goes without saying that a plurality of communication means (for example, the terminal transmission unit 24, the terminal reception unit 22, etc.) existing in one device may be physically realized by one medium. .

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the server device according to the present invention has an effect of easily obtaining content in which a desired person is shown, and is useful as a server device.

DESCRIPTION OF SYMBOLS 1 Server apparatus 2 Terminal apparatus 11 Storage part 12 Reception part 13 Processing part 14 Transmission part 111 1st storage part 112 2nd storage part 113 3rd storage part 114 4th storage part 115 5th storage part 116 6th storage part 121 6th One receiving unit 122 Second receiving unit 123 Third receiving unit 131 Search unit 132 Calculation unit 133 Charging unit 134 Creation unit 135 Storage unit 141 Transmission unit

Claims (16)

A first storage unit that stores one or more contents that are images transmitted from a plurality of terminal devices;
A first reception unit that receives a search instruction including person designation information, which is information relating to person designation, from one terminal device among the plurality of terminal devices;
A search unit for searching for one or more contents including a face image that is an image of a person's face corresponding to the person specifying information included in the search instruction received by the first receiving unit from among the one or more contents. When,
A server device comprising: a transmission unit that transmits one or more contents searched by the search unit to the one terminal device.   The server device according to claim 1, wherein the person designation information is configured by a designation operation for a person in an image output from the one terminal device.   The server device according to claim 2, wherein the specifying operation is an operation of specifying one person among a plurality of persons in the image. A second storage unit that stores one or more pieces of person information including a person face image and a person identifier for identifying the person;
The person designation information has a person identifier,
The search unit
A face image of a person who is paired with a person identifier included in the person designation information is acquired from the second storage unit, and one or more contents partially including an image having a predetermined relationship with the face image are searched. The server device according to claim 1. One or more contents stored in the first storage unit are associated with time information related to time,
The search instruction includes time information,
The search unit determines one or more contents corresponding to time information included in the search instruction,
The transmitter is
A screen having a union of one or more contents searched by the search unit and one or more contents determined by the search unit, and transmitting screen information that is information on a screen for selecting content or specifying a person The server device according to any one of claims 1 to 4. The one or more contents stored in the first storage unit are associated with position information regarding the position,
The search instruction includes position information,
The search unit
The server device according to claim 5, wherein one or more contents corresponding to time information and position information included in the search instruction are determined. The time information associated with one or more contents in the first storage unit indicates the time when the contents were taken by the camera,
The position information associated with one or more contents in the first storage unit indicates the position of the camera that captured the contents,
The one or more contents in the first storage unit are further associated with optical information related to at least one of an angle of view and a focal length of the camera that captured the content,
Using a positional information and optical information associated with one or more contents of the first storage unit, and further calculating a position of each person shown in the content,
When the search unit determines one or more contents corresponding to the time information and the position information included in the search instruction, the search unit indicates the person position calculated by the calculation unit in the position information included in the search instruction The server device according to claim 6, wherein the server device is compared with a position. The search unit
By comparing the similarity between the face image corresponding to the person designation information and each face image included in the one or more contents stored in the first storage unit with a threshold value, It is determined whether or not a face image corresponding to the person designation information is included, and when the time information or the position information is used for the search, and when the time information or the position information is not used for the search, the threshold value is set. The server device according to any one of claims 5 to 7, which is changed. A third storage unit that stores at least period information indicating a period and event information indicating an event performed in the period in association with each other;
One or more contents of the first storage unit are associated with time information related to time,
The transmitter is
When transmitting one or more contents searched by the search unit, the time information associated with the content is compared with each period information stored in the third storage unit, and the content is The event information indicating an event that has been performed during a period including the time when the image was taken is specified, and the specified event information is transmitted in association with the content. Server device. The transmitter is
One or more representative images respectively associated with one or more contents searched or determined by the search unit are included in the screen information and transmitted, and are output based on the screen information in the one terminal device. The server according to any one of claims 5 to 9, wherein when a representative image is selected on the selected screen, content associated with the selected representative image is transmitted to the one terminal device. apparatus. A fourth storage unit in which a person identifier and a password are stored in pairs;
A fifth storage unit that stores a pair of a person identifier stored in the fourth storage unit and one or more person identifiers having a predetermined relationship with the person corresponding to the person identifier;
A third receiving unit for receiving a login request designating a person identifier and a password via the terminal device;
The transmitter is
When the pair of the person identifier and password specified in the login request is stored in the fourth storage unit, the selection of content is accepted on the screen, and the person specified in the login request is received in the content If the face image of the person corresponding to the identifier or the person corresponding to the person identifier stored in association with the person identifier specified in the login request in the fifth storage unit is included, the content is The server device according to claim 10 which transmits. A sixth storage for storing price information related to the price of the content;
The charging device further comprises a billing unit that refers to the price information and charges the user of the terminal device for an amount corresponding to the content downloaded by the transmitting unit to the storage unit of the terminal device. The server device according to claim 11.   The server device according to any one of claims 5 to 7, further comprising a creation unit that creates an album by arranging one or more contents selected on the screen on each page. A second receiving unit for receiving content transmitted from each of the plurality of terminal devices;
An accumulation unit that accumulates the content received by the second reception unit in the first storage unit;
By extracting one or more face images from the content received by the second receiving unit and comparing the one or more face images with one or more face images stored in the second storage unit, 5. The processing apparatus according to claim 4, further comprising: a processing unit that acquires one or more person identification information that is paired with one or more face images, and holds the acquired one or more person identification information as a pair with the content. The server device according to any one of 13. A search method performed by a first storage unit, a first reception unit, a search unit, and a transmission unit,
A first storage step in which one or more contents that are images transmitted from a plurality of terminal devices are stored in the first storage unit;
The first receiving unit receives a search instruction including person designation information, which is information relating to person designation, from one terminal device among the plurality of terminal devices; and
A search step in which the search unit searches for one or more contents including a face image that is an image of a person's face corresponding to the person designation information included in the search instruction received by the first receiving unit;
A transmission step in which the transmission unit transmits one or more contents searched by the search unit to the one terminal device. Computer
A first storage unit that stores one or more contents that are images transmitted from a plurality of terminal devices;
A first receiving unit that receives a search instruction including person designation information, which is information relating to person designation, from one terminal device among the plurality of terminal devices;
A search unit for searching for one or more contents including a face image that is an image of a person's face corresponding to the person specifying information included in the search instruction received by the first receiving unit;
The program for functioning as a transmission part which transmits the one or more content searched by the said search part to said one terminal device.

Images