JP6839366B2 - Information processing equipment, information processing system, its control method, and program - Google Patents

Description

The present invention relates to a concession processing device, an information processing system, a control method and a program thereof.

For example, when a large number of photographs are taken for a plurality of persons such as a school trip or an event, the persons shown in the photographs (image data) may be biased.

When creating an album with the photos, the album editor adjusts the number of all subjects in the photos in the album to be about the same so that the album is fair to all subjects. It is necessary to do this, and the adjustment work requires a great deal of labor.

On the other hand, the number of face images included in the image data for creating an album is counted for each person and presented to the user, and the number of faces of the counted person is restricted so as not to be selected more than a predetermined number. (For example, Patent Document 1).

JP-A-2010-057073

However, in the technique described in Patent Document 1, for example, when the image being selected or being selected is an image subject to the above restrictions, the user can select an image that is not subject to the above restrictions from among a large number of images. It is very troublesome to find and reselect.

An object of the present invention is to present image data of selection candidates based on the information of the selected image data.

The information processing device of the present invention is an information processing device having a storage function for storing a plurality of image data, and includes a subject detecting means for detecting the subject of the image data, a selection receiving means for accepting selection of the image data, and the like. Replacement display control means for controlling to replace and display candidate image data, which is candidate image data for which selection is accepted by the selection reception means, according to information of a subject in the image data for which selection is accepted by the selection reception means. It is characterized by having.

According to the present invention, image data of selection candidates can be presented based on the information of the selected image data.

It is a figure which shows an example of the system structure in embodiment of this invention. It is a figure which shows an example of the hardware composition of various devices in embodiment of this invention. It is a figure which shows an example of the functional structure of various devices in embodiment of this invention. It is a flowchart which shows the flow of the registration process of a personal image in embodiment of this invention. It is a flowchart which shows the flow of the registration process of a photograph in embodiment of this invention. It is a flowchart which shows the flow of the subject identification processing in embodiment of this invention. It is a flowchart which shows the flow of layout change processing in embodiment of this invention. It is a flowchart which shows the flow of the equality calculation process of a section unit in embodiment of this invention. It is a flowchart which shows the flow of the photograph exchange process of an inconspicuous subject in embodiment of this invention. It is a flowchart which shows the flow of the photograph exchange process of the conspicuous subject in embodiment of this invention. It is a figure which shows an example of the data structure in embodiment of this invention. It is a figure which shows an example of the data structure in embodiment of this invention. It is a figure which shows an example of the data structure in embodiment of this invention. It is a figure which shows an example of the structure of the personal information registration screen in embodiment of this invention. It is a figure which shows an example of the structure of the page edit screen in embodiment of this invention. It is a figure which shows an example of the structure of the page edit screen in embodiment of this invention. It is a figure which shows an example of the calculation formula of the equality calculation in embodiment of this invention. It is a figure which shows an example of the structure of the subject identification screen in embodiment of this invention. It is a figure which shows an example of the structure of the subject identification screen in embodiment of this invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. An example of the system configuration in the embodiment of the present invention will be described with reference to FIG. FIG. 1 is a diagram showing an example of a system configuration according to an embodiment of the present invention.

The client PC 100 displays image data on the display screen and receives an album editing instruction from the user. The server 200 stores image data for creating an album in a storage area such as an external memory (storage function), and receives an instruction from the client PC 100 to create and edit the album.

The image data for creating the album is transmitted to the server 200 via the network 1 in response to the operation instruction received from the user by the client PC 100, so that the image data is stored in a storage area such as an external memory of the server 200. It shall be remembered.

Further, the server 200 extracts candidate image data, which is a candidate image for the user to select, from the image data stored in the external memory, based on the information of the image data selected by the user on the client PC 100. It is transmitted to the client PC 100 via the network 1 so as to be displayed on the display screen of the client PC 100. The above is the description of an example of the system configuration in the embodiment of the present invention shown in FIG.

Next, with reference to FIG. 2, an example of the hardware configuration of various devices according to the embodiment of the present invention will be described. FIG. 2 is a diagram showing an example of hardware configurations of various devices according to the embodiment of the present invention.

The CPU 201 comprehensively controls each device and controller connected to the system bus 204 by using the programs and data stored in the RAM 202 and the ROM 203.

The RAM 202 has an area for temporarily storing programs and data loaded from the external memory 211, and also has a work area used by the CPU 201 for performing various processes.

The ROM 203 stores various programs such as a computer boot program and a BIOS. The external memory 211 stores programs, data, and the like for causing the CPU 201 to execute the processing described later performed by the OS (operating system) and the information processing device, and these are read into the RAM 202 under the control of the CPU 201 as necessary. It will be issued and executed.

Further, in the ROM 203 or the external memory 211, a BIOS (Basic Input / Output System) which is a control program of the CPU 201, an operating system program (hereinafter referred to as an OS), and various programs necessary for realizing a function executed by each PC are used. Etc. are remembered. The RAM 202 functions as a main memory, a work area, and the like of the CPU 201.

The CPU 201 realizes various operations by loading a program or the like necessary for executing a process into the RAM 202 and executing the program.
Further, the input controller (input C) 205 controls the input from a pointing device such as a keyboard 209 or a mouse.

The video controller (VC) 206 controls the display on a display such as the display 210. The display may be a liquid crystal display (LCD) or a CRT display (that is, a display control means for displaying a display screen including a plurality of objects on the display unit).

The memory controller (MC) 207 is a hard disk (HD) or floppy (registered trademark) disk (registered trademark FD) that stores boot programs, browser software, various applications, font data, user files, editing files, various data, and the like. Alternatively, it controls access to an external memory 211 such as a CompactFlash® memory connected to the PCMCIA card slot via an adapter.

The communication I / F controller (communication I / FC) 208 connects and communicates with an external device via a network, and executes communication control processing on the network.

The CPU 201 can display the outline font on the display 210 by executing the outline font expansion (rasterization) process on the display information area in the RAM 202, for example. Further, the CPU 201 enables a user instruction with a mouse cursor or the like (not shown) on the display 210.

The program for realizing the present invention is stored in the external memory 211, and is executed by the CPU 201 by being loaded into the RAM 202 as needed. The above is the description of an example of the hardware configuration of various devices according to the embodiment of the present invention shown in FIG.

Next, with reference to FIG. 3, an example of the functional configuration of various devices according to the embodiment of the present invention will be described. FIG. 3 is a diagram showing an example of the functional configuration of various devices according to the embodiment of the present invention.

The client PC 100 includes an image transmission unit 301, an image selection reception unit 302, a layout change instruction reception unit 303, an album creation screen display unit 304, a trimming range designation reception unit 305, and the like.

The image transmission unit 301 is a transmission unit that transmits image data in order to store the image data in the server 200. For example, the image data is transmitted so that the image data that is a candidate for the image (album image) to be posted on the album is stored in the external memory of the server 200. Further, for example, information on a subject such as a face image is transmitted so as to be stored in the server 200.

The image selection reception unit 302 accepts the selection of the image data to be posted on the album and the image data to be replaced (deleted from the album) with another image data on the album creation screen displayed on the album creation screen display unit 304. It is a selection reception department.

The layout change instruction receiving unit 303 is an instruction receiving unit that receives an instruction to change the layout of an album (album image) on the album creation screen displayed on the album creation screen display unit 304. The album creation screen display unit 304 is a display unit that accepts selection of image data to be posted on an album and displays a screen for creating an album.

The trimming range designation reception unit 305 is an instruction reception unit that receives an instruction for designating the trimming range when a part of the image data of the replacement candidate is trimmed and selected as an image to be posted on the album.

The server 200 includes an image storage unit 311, a personal identification information storage unit 312, a correspondence information storage unit 313, an equality calculation unit 314, a screen information generation unit 315, an image editing unit 316, a highlighting control unit 317, and the like. Has been done.

The image storage unit 311 is a storage unit that stores image data for creating an album. The client PC 100 acquires the image data stored in the image storage unit, displays the image data on the display screen, and allows the user to select the album image to specify the album image.

In addition, the image storage unit 311 stores image data for identifying the subject of each image data stored in the image storage unit 311 and image data for posting on an album. For example, the photograph 1300 of FIG. 13 and the face 1310 and the like.

Here, an example of a data structure in the embodiment of the present invention will be described with reference to FIG. FIG. 13 is a diagram showing an example of a data structure in the embodiment of the present invention.

The information shown in FIG. 13 is generated by, for example, in the processes of FIGS. 4 and 5 described later, when the client PC 100 transmits image data to the server 200 and gives a storage instruction, and is generated by an external memory of the server 200. Is remembered in.

As shown in FIG. 13, Photo 1300 is composed of Project ID 1301, Photo ID 1302, Photo 1303, Shooting Date and Time 1304, Size 1305, Type 1306, Event 1307, Grade ID 1308, and the like.

The project ID 1301 is the ID of the project in which the photograph 1300 is registered. A project is an order unit for album creation managed by an album creator. The album (album ID) is linked under the project.

When the client PC 100 receives a storage instruction to the server 200, both the image data and the list information are stored in the external memory of the server 200 in association with the project that has received the designation of the project. The data structure for album management of the present invention including the project will be described later in the description of FIG.

Here, an example of a data structure in the embodiment of the present invention will be described with reference to FIG. FIG. 11 is a diagram showing an example of a data structure according to the embodiment of the present invention.

It is assumed that each data shown in FIG. 11 is stored and registered in the external memory of the server 200 in advance before the image data and the list information are registered in the server 200. Note that the photo ID 1135 and trimming coordinates 1137 of the photo arrangement information 1130, which will be described later, are not limited to this.

The project 1100 is composed of a project ID 1101, a project name 1102, a project manager ID 1103, a contractor information 1104, and the like. The project ID 1101 is identification information for uniquely identifying the project.

The project name 1102 is the name of the project. The project manager ID 1103 is identification information of a user (album editor) who is a manager of each project and has editing authority. That is, a user ID for identifying the user is set.

The user ID information is separately stored in the external memory of the server 200 in a data table (not shown) in association with the password. The album editor logs in to the server 200 using the user ID and password, and creates / edits an album via the client PC 100. The contractor information 1104 indicates the information of the contractor who is the contractor of the project shown in the project.

The album 1110 is composed of a project ID 1111, an album ID 1112, an album name 1113, an update date and time 1114, and the like. The project ID 1111 is the ID of the project corresponding to each album stored in the album 1110.

The album ID 1112 is identification information for uniquely identifying the album. The album name 1113 is the name of the album. The update date and time 1114 is the last update date and time of the album.

Page 1120 is composed of project ID 1121, album ID 1122, page ID 1123, INDEX 1124, editor ID 1125, and the like. The project ID 1111 is the ID of the project to which the album corresponding to the page 1120 belongs.

The album ID 1122 is the ID of the album (to which the page belongs) corresponding to each page. The page ID 1123 is identification information for uniquely identifying the page. INDEX1124 is information for indexing each page. The editor ID 1125 is the ID of a user (editor) who has the editing authority for the page.

The photo arrangement information 1130 is composed of a project ID 1131, an album ID 1132, a page ID 1133, an arrangement ID 1134, a photo ID 1135, an arrangement coordinate 1136, a trimming coordinate 1137, and the like. The project ID 1131 is the ID of the project corresponding to the photo arrangement information.

The album ID 1132 is the ID of the album. Page ID 1133 is the ID of the page corresponding to the photo arrangement information. The arrangement ID 1134 is identification information for uniquely identifying the photo arrangement information.

Photo ID 1135 is an ID of a photographic image (image data) arranged at arrangement coordinates 1136. The trimming coordinates 1137 are the trimming coordinates in the image data when a part of the image data shown in the photograph ID 1135 is trimmed and abolished. The above is the description of an example of the data structure in the embodiment of the present invention shown in FIG.

Returning to the description of FIG. The photo ID 1302 of the photo 1300 is identification information for uniquely identifying a photographic image (image data). Photo 1303 is an entity of image data. The shooting date and time 1304 is the shooting date and time of each image data.

The size 1305 is the size of the image data. Type 1306 is a type of image data (for example, a snapshot, an individual photograph, or a group photograph). The personal photograph referred to here is an image registered in the external memory of the server 200 for identifying the subject (individual).

Event 1307 is the attribute information of the image data, and indicates which event (scene) of the image data is the image. In the event 1307 of FIG. 13, the event name is described. For example, a data table in which the event name and the event ID for identifying the event are associated with each other is separately stored in the external memory of the server 200. The event ID may be inserted into the event 1307.

The grade ID 1308 indicates which grade (affiliation / organization) the image data is. The grade ID 1212 in the grade 1210 of FIG. 12, which will be described later, is inserted into the grade ID 1308.

Which class of the grade indicated by the grade ID 1308 the image data belongs to can be identified according to the ratio of the student class indicated by the face image shown in the image data. For example, when more than half of the face images reflected in the image data are face images of students belonging to the A class, the image data is assumed to be the image data of the A class.

The face 1310 is composed of a project ID 1311, a face ID 1312, a face photograph 1313, a photograph ID 1314, a face coordinate 1315, a face orientation 1316, a smile degree 1317, an identification personal ID 1318, and the like. The project ID 1311 is the identification information of the project. The face ID 1312 is identification information of each data stored in the face 1310.

The face photograph 1313 is an entity of the face image of the subject registered according to the operation instruction of the user. The face image is, for example, image data obtained by trimming a face-recognized area in a personal photograph.

Further, for example, in a group photo or a snapshot, face identification processing (face identification processing / a face image in a photographic image is associated with a student's identification information as an image of a certain student's face according to a user's operation instruction. This is image data generated by trimming the image of the area in the group photograph or the snapshot photograph designated for the face identification when the process of registering the image is performed.

The photo ID 1314 is identification information of the generation source of the face photo 1313 (image data obtained by trimming the face photo 1313). The face coordinates 1315 are the coordinates of the face photograph 1313 in the photographic image indicated by the photograph ID 1314.

The face orientation 1316 is the orientation of the face in the face image of the face photograph 1313. The smile degree 1317 is the smile degree of the face image of the face photograph 1313. The identification personal ID 1318 is a student ID of the student shown in the face photograph 1313, which is inserted when the above-mentioned face identification process is performed in response to a user's operation instruction.
The data in which the identification personal ID 1318 is not inserted is the image data in which the face identification process has not been performed.

The identification candidate 1320 is composed of the project ID 1321, the face ID 1322, the student ID 1323, the similarity 1324, and the like. The identification candidate 1320 indicates the face image recognized in all the photographic images (image data) and which student's face is similar to the face image.
The project ID 1321 is the identification information of the project. The face ID 1322 is the identification information of the face data in the face 1310.

The student ID 1323 is identification information for identifying the student. The similarity 1324 is, for example, the feature amount of the face image (student face image indicated by the student ID 1323) stored and registered in the external memory of the server 200 in the registration process of the student's personal photo of FIG. 4, which will be described later, and the face ID 1322. It is a value of the degree of similarity with the feature amount of the face image (face image extracted from the image data for publication in the album stored in the server 200) indicated by.

It is assumed that the data indicating the feature amount of the image data of the student's face image is stored in the external memory 211 of the server 200 in association with the student ID. In addition, the data indicating the feature amount of each face image extracted from the image data for publication in the album is the photo ID of the photographic image including the face image and the coordinates of the face image in the photographic image indicated by the photo ID. It is assumed that it is stored in the external memory 211 of the server 200 in association with the information. The above is the description of an example of the data structure in the embodiment of the present invention shown in FIG.

The personal identification information storage unit 312 is a storage unit that stores information for identifying a subject, such as the name of a subject (person, student, animal, etc.). For example, information such as the student list 1200 (list information) and grade 1210 in FIG. 12 is stored.

Here, an example of a data structure in the embodiment of the present invention will be described with reference to FIG. FIG. 12 is a diagram showing an example of a data structure according to the embodiment of the present invention. The information shown in FIG. 12 is generated by, for example, in the process of FIG. 4 described later, the client PC 100 transmits information on the list of subjects (students) to the server 200 and gives a storage instruction, and is generated outside the server 200. It is stored in the memory.

As shown in FIG. 12, the student list 1200 is composed of a project ID 1201, a student ID 1202, a student name 1203, a grade ID 1204, a class ID 1205, and the like.

The project ID 1201 is the identification information of the project in which the student list 1200 is registered. Student ID 1202 is student identification information. Student name 1203 is information on the student's name. The grade ID 1204 is identification information of the grade to which the student belongs. Class ID 1205 is identification information of the class to which the student belongs.

The grade 1210 is composed of a project ID 1211, a grade ID 1212, an affiliation name 1213, a parent affiliation 1214, and the like. The project ID 1211 is the identification information of the project in which the grade 1210 is registered.

The grade ID 1212 is the identification information of the grade. The affiliation name 1213 is, for example, the name of the grade. The parent affiliation 1214 is identification information of a higher-ranking organization (for example, a school) corresponding to the grade.

Class 1220 is composed of project ID 1221, class ID 1222, affiliation name 1223, parent affiliation 1224, and the like. The project ID 1221 is the identification information of the project in which the class 1220 is registered.

The class ID 1222 is class identification information. The affiliation name 1223 is, for example, the name of the class. The parent affiliation 1214 is identification information of a higher level organization (for example, a grade) corresponding to the class. The above is the description of an example of the data structure in the embodiment of the present invention shown in FIG.

Returning to the description of FIG. The correspondence information storage unit 313 is a storage unit that stores the correspondence information of the correspondence between the image data and the personal identification information and the correspondence between each image data and the album (the photograph arrangement information 1130 in FIG. 11 described above, and the correspondence information storage unit 313). Included in the description of Photo 1300 of FIG. 13).

The equality calculation unit 314 determines the frequency of appearance of the same subject, the size of each subject, and whether or not the subject is focused in the image data (for example, a plurality of image data) selected as the image data to be posted on the album. It is a calculation unit that calculates whether each subject is photographed equally (whether it is equally reflected in the image data published in the album) based on the information such as.

The screen information generation unit 315 is a generation unit that generates screen data to be displayed on the client PC 100, such as an album edit screen. When an instruction to change the layout of an album is accepted, or when an instruction to replace candidate image data, which is candidate image data for accepting selection as an image to be posted on an album, is accepted, screen information is regenerated and the client PC100 is used. Send to.

The image editing unit 316 is an editing unit that edits (trimming, scaling, etc.) image data when receiving an instruction to edit image data from the client PC 100. The highlighting control unit 317 is a control unit that controls the image data to be highlighted so as to display young birds when the screen information generation unit 315 generates screen information. The above is the description of an example of the functional configuration of various devices according to the embodiment of the present invention shown in FIG.

Next, with reference to FIG. 4, the flow of the personal image registration process in the embodiment of the present invention will be described. FIG. 4 is a flowchart showing the flow of personal image registration processing according to the embodiment of the present invention.

The CPU 201 of the client PC 100 accepts the user's login and requests the server 200 for screen information for registering information for personal identification (screen information such as the personal information registration screen 1400 in FIG. 14) (step). S401).

Here, an example of the configuration of the personal information registration screen in the embodiment of the present invention will be described with reference to FIG. FIG. 14 is a diagram showing an example of the configuration of the personal information registration screen in the embodiment of the present invention.

As shown in FIG. 14, the personal information registration screen 1400 has a "grade selection" button 1401, a "class selection" button 1402, an uploaded image display unit 1403, a selected image display unit 1404, a face recognition frame 1405, a correction frame 1406, and a personal photo. It is composed of a registration unit 1407, an attendance number display unit 1408, a name display unit 1409, and the like.

The "grade selection" button 1401 is a selection reception unit that allows the user to select the grade of the user who registers / edits personal information. The "class selection" button 1402 is a selection reception unit that allows the user to select the class of the user who registers / edits personal information.

For example, the CPU 201 of the client PC 100 displays grade candidates in a pull-down format by accepting the press of the "grade selection" button 1401, and class candidates in the pull-down format by accepting the press of the "class selection" button 1402. Is displayed to let the user select the grade / class for which personal information is to be registered.

The uploaded image display unit 1403 is a display unit that displays thumbnails of images (personal images / identification images) uploaded (stored) in the server 200. The selected image display unit 1404 is a display unit that enlarges and displays the image data of the personal image that has received the selection instruction from the user.

The face recognition frame 1405 is a frame that surrounds the subject face image, which is recognized in the image displayed on the selected image display unit 1404. The face recognition frame (face recognition area) is changed by accepting drag-and-drop and move operations on the edge of the correction frame 1406.

The personal photo registration unit 1407 is a display unit for registering and displaying image data corresponding to the attendance number displayed on the attendance number display unit 1408 and the name displayed on the name display unit 1409.

For example, by moving the image data to the personal photo registration unit 1407 corresponding to the name display unit 1409 displayed as Mr. A by dragging and dropping, the image data is associated and registered as the image data of Mr. A.

The association information is stored in the external memory of the server 200. The above is the description of an example of the configuration of the personal information registration screen in the embodiment of the present invention shown in FIG.

Returning to the description of FIG. The server 200 receives the screen information request from the client PC 100 (step S402) and returns the screen information (step S403).
The CPU 201 of the client PC 100 receives the screen information (information on the personal information registration screen) and displays it on the display screen (step S404).

The CPU 201 of the client PC 100 receives an instruction to transmit the list information (step S405). For example, when a CSV format file in which the information of the list is described is dragged and dropped to either the attendance number display unit 1408 or the name display unit 1409 of FIG. 14, the instruction to transmit the list information is accepted. The list information is transmitted to the server 200.
The list information referred to here is, for example, a file in which information such as a student's name, grade, class, attendance number, etc. is described.

Further, the CPU 201 of the client PC 100 transmits image data (face image data) for subject identification, that is, for associating with the data of each student included in the list information transmitted in step 405, to the server 200 (step S406). ).

For example, when the image data is dragged and dropped on the uploaded image display unit 1403 of FIG. 14, it is assumed that the transmission instruction of the image data is received, and the image data is transmitted to the server 200.

The CPU 201 of the server 200 receives the list information and image data (student list 1200 / grade 1210 / photo 1300) from the client PC 100 and stores them in the external memory (step S407 / step S408).

Then, the attendance number and name information in the stored list information are displayed on the attendance number display unit 1408 and the name display unit 1409 of the personal information registration screen 1400, and the stored image data is displayed on the upload image display unit 1403. In order to generate the information on the personal information registration screen.

The information on the screen is transmitted to the client PC, and the CPU 201 of the client PC 100 displays this on the display screen. That is, the image data and the list information uploaded to the server 200 are reflected on the personal information registration screen 1400 displayed on the display screen of the client PC 100 in real time.

The CPU 201 of the client PC 100 receives an instruction (drag instruction) to select the image data corresponding to the name displayed on the name display unit 1409 from the upload image display unit, and the selected image data is the personal photo registration unit 1407. In order to associate the student information (1 data in the list information) of the name and attendance number corresponding to the personal photo registration unit 1407 with the image data when placed in (drop instruction), the image data The correspondence instruction including the identification information, the name of the correspondence target, and the information of the school registration number is transmitted to the server 200 (step S409).

The CPU 201 of the server 200 receives the corresponding instruction and stores the image data and the student information (personal information) included in the list information in association with each other (step S410).

The CPU 201 of the server 200 acquires the feature amount of the associated image data (selected by the user) and executes the face identification (face detection / face recognition / subject detection) process (step S411). The face identification information acquired in the face identification process is stored in an external memory in association with the image data.

The CPU 201 of the server 200 surrounds the outer ring of the face identified by the identification process with a rectangle (face recognition frame 1405 in FIG. 14), and displays an image in which the outer ring of the face is surrounded by the rectangle in order to notify the user as the face identification result. , The screen information displayed on the selected image display unit 1404 is transmitted to the client PC 100 (step S412).

The CPU 201 of the client PC 100 receives the information on the screen and displays it on the display screen (step S413). That is, the personal information registration screen 1400 in which the face identification result is displayed on the selected image display unit 1404 is displayed on the display screen. In addition, the correction frame 1406 is also displayed on the display screen.

The CPU 201 of the client PC 100 determines whether or not the instruction for moving and scaling the correction frame 1406 (that is, the instruction for correcting the face identification area) has been accepted (step S414), and if it is determined that the instruction for correction has been accepted (step S414). YES in S414), the corrected information, that is, the information of the image surrounded by the correction frame 1406 is transmitted to the server 200 as the correction information (step S415).
The CPU 201 of the server 200 receives the correction information, reflects it in the face identification information (face coordinate information) (step S416), and returns the process to step S412.

When the CPU 201 of the client PC 100 determines in step S414 that the correction instruction is not accepted (NO in step S414), it determines whether or not the personal information registration completion instruction has been accepted (step S417), and registers the personal information. If it is determined that the completion instruction is not accepted (NO in step S417), the process returns to step S409.
For example, when the pressing of the registration completion button (not shown) on the personal information registration screen Article 1400 is accepted, it is determined that the instruction to complete the registration has been accepted.

When the CPU 201 of the client PC 100 determines that the personal information registration completion instruction has been accepted (YES in step S417), the CPU 201 transmits the registration completion instruction to the server 200 (step S418).

The CPU 201 of the server 200 receives the registration completion instruction, deletes the unregistered personal image (uploaded image data that has not been associated) from the external memory (step S420), and ends the process. The above is the description of the flow of the personal image registration process in the embodiment of the present invention shown in FIG.

Next, with reference to FIG. 5, the flow of the photo registration process in the embodiment of the present invention will be described. FIG. 5 is a flowchart showing the flow of the photo registration process in the embodiment of the present invention.

The CPU 201 of the client PC 100 requests the server 200 for screen information of an image registration screen (not shown) (step S501). The image registration screen referred to here is a photograph for posting in an album, which is different from the personal image registered and stored in the server by the process shown in FIG. For example, it is image data in which a plurality of students are shown.

The CPU 201 of the server 200 receives the request for the image registration screen (step S502) and transmits the information on the screen to the client PC 100 (step S503).

The CPU 201 of the client PC 100 receives the information on the image registration screen and displays it on the display screen (step S504). Further, when the image registration instruction (transmission instruction / upload instruction to the server 200) is received on the image registration screen, the image data to be registered is transmitted to the server 200 in accordance with the instruction (step S505). The method of registering and selecting the image data to be registered follows, for example, the method of personal image registration in S406 of FIG.

When the CPU 201 of the server 200 receives the image data (step S506), the CPU 201 stores the image data in the external memory (storage area) of the server 200 (step S507). Then, the subject in the image data is identified by using, for example, a face recognition technique (step S508), and the registration process of the identification result information and the image data transmitted from the client PC 100 including the image data is completed. Information indicating that this has been done is transmitted to the client PC 100 (step S509).

The CPU 201 of the client PC 100 receives the information and displays it on the display screen (step S510). After that, it was determined whether or not the pressing of the "registration completed" button on the image registration screen (not shown) was accepted (whether or not the registration completion instruction was accepted) (step S511), and it was determined that the registration completion instruction was not accepted. If (NO in step S511), the process returns to step S505.

When it is determined that the registration completion instruction has been accepted (YES in step S511), the process ends. The above is the description of the flow of the photo registration process in the embodiment of the present invention shown in FIG.

Next, with reference to FIG. 6, the flow of the subject identification process in the embodiment of the present invention will be described. FIG. 6 is a flowchart showing the flow of subject identification processing in the embodiment of the present invention.

In the process of FIG. 6, a process for confirming and correcting the presence or absence of a subject not identified (detected / recognized) in step S508 of FIG. 5 in the image registered in the server 200 in the process of FIG. Is.

The CPU 201 of the client PC 100 requests the server 200 for screen information of the student identification screen (for example, the subject identification screen 1800 in FIG. 18) in response to an operation instruction from the user (step S601). The server 200 receives the request (step S602) and transmits screen information to the client PC 100 (step S603).

The CPU 201 of the client PC 100 receives the screen information and displays the subject identification screen on the display screen (step S604). The subject identification screen is a user's manual operation of the face image in the image data in order to accurately count the number of face images in the image data stored in the external memory of the server 200 for posting on the album. This is a screen for associating with the identification information of the subject. For example, it is a screen such as the subject identification screen 1800 of FIG.

An example of the configuration of the subject identification screen in the embodiment of the present invention will be described with reference to FIG. FIG. 18 is a diagram showing an example of the configuration of a subject identification screen according to the embodiment of the present invention.

The grade selection unit 1801, the class selection unit 1802, and the student selection unit 1803 are selection reception units for accepting student selection. The personal photograph 1804 is a display unit that displays a face image of the student when the user receives a selection instruction from the user in the grade selection unit 1801, the class selection unit 1802, and the student selection unit 1803. The face image displayed here is image data of a photographic image stored in the external memory of the server 200 in association with the student identification information in step S410 of FIG.

The face image display unit 1805 is a face image cut out from a snapshot image stored in the external memory of the server 200, and is similar to the face image of the student whose selection is accepted by the student selection unit 1803. This is a display unit that displays a list of face image data to which the face identification process (process with correspondence between a student and a face image) is not applied in 1800.

That is, the face of the identification candidate data in which the student ID is not inserted in the identification personal ID 1318 of the face 1310 in FIG. 13 and the student ID of the student selected by the user operation is held in the student ID 1323 of the identification candidate 1320. The face photograph 1313 having the ID matching the ID 1322 in the face ID 1312 is displayed.

The event selection reception unit 1809 determines which of the photographic images of the students selected by the student selection unit 1803, which has the attribute of the event as the property, to display the face image included in the photographic image on the face image display unit. It is a selection reception section to select.

For example, when "athletic association" is selected in the event selection reception unit 1809, it will be the face of the student selected by the student selection unit 1803 included in the image data having the value of "athletic association" in event 1307. The face image is displayed on the face image display unit.

Here, in the face image displayed on the face image display unit 1805, the student ID 1323 of the identification candidate 1320 in FIG. 13 is the ID of the student selected by the student selection unit 1803, and the similarity 1324 is equal to or higher than a predetermined value. It is a face image (a face image similar to a certain level or more). It is assumed that the predetermined value is stored in the external memory of the server 200 in advance (predetermined value = similarity: 0.7 or more, etc.).

Further, for example, the face image displayed on the face image display unit 1805 is sorted and displayed from the upper left portion to the lower right portion of the face image display unit 1805 in descending order of the value of the similarity 1324 in the above-mentioned identification degree candidate 1320. It shall be.

By accepting the selection of the face image displayed on the face image display unit, the snapshot photo display unit 1807 displays the snapshot (candidate image data) of the face image for which the selection has been accepted.

The face recognition frame 1808 is a face recognition frame which is a frame in which the image recognized as a face is surrounded by a rectangle when the face recognition process is performed by the CPU 201 of the server 200 in the snapshot. The face recognition frame surrounding the face image in the snapshot corresponding to the face image selected by the user on the face image display unit 1805 shall be highlighted to distinguish it from other face recognition frames.

Further, the face recognition frame 1808 is deformed by receiving an enlargement instruction, a reduction instruction, and a transformation instruction of the face recognition frame 1808, and by receiving a pressing instruction of the "face image addition" button 1810, the face recognition frame 1808 is deformed. The image of the area surrounded by the face recognition frame after the transformation is stored in association with the face image of the student.

The "Add face image" button 1810 is a button for newly associating an image (face image) of the area surrounded by the face recognition frame 1808 that has received the selection as a face image of the student that has received the selection in the student selection unit 1803. Is. By accepting the press of the "add face image" button 1810, the student ID of the student whose selection has been accepted by the student selection unit 1803 is inserted into the identification personal ID 1318 of the face data having the ID of the face image in FIG. Will be done.

Further, by performing the corresponding processing, the face image is controlled not to be displayed on the face image display unit 1805 (deleted from the face image display unit 1805), and is additionally displayed on the identification result display unit 1812. ..

The "face image deletion" button 1811 deletes the face image selected in the face image display unit 1805 from the face image display unit 1805, and ends the display of the photographic image in which the face image is shown in the snap photo display unit 1807. It is a button to do.

The identification result display unit 1812 is a display unit that displays a face image newly associated as a face image of a student whose selection has been accepted by the student selection unit 1803. The "identification result deletion" button 1813 is a button for canceling the correspondence of the face image for which the selection has been accepted among the face images displayed on the identification result display unit 1812. The "photo confirmation" button 1806 is a button for transitioning the display screen to the subject identification screen 1900, which is also the subject identification screen. The above is the description of an example of the configuration of the subject identification screen according to the embodiment of the present invention shown in FIG.

Here, an example of the configuration of the subject identification screen in the embodiment of the present invention will be described with reference to FIG. FIG. 19 is a diagram showing an example of the configuration of a subject identification screen according to the embodiment of the present invention.

The grade selection unit 1901, the class selection unit 1902, the student selection unit 1903, the personal photograph 1904, the face recognition frame 1908, the identification result display unit 1912, and the "identification result deletion" button 1913 are the grade selection unit 1801 and the class in FIG. Since they are the same as the selection unit 1802, the student selection unit 1803, the personal photograph 1804, the face recognition frame 1808, the identification result display unit 1812, and the "identification result deletion" button 1813, the description thereof is omitted here.

The snapshot list display unit 1905 is a display unit that displays a list of snapshots including a face image similar to a predetermined value or more on the student's face selected by the student selection unit 1903. The enlarged display unit 1907 is an enlarged display unit that enlarges and displays a photographic image selected by the snapshot list display unit 1905. The "face image confirmation" button 1906 is a button for transitioning the display screen to the subject identification screen 1800, which is also the subject identification screen.

The "add face image" button 1910 is a button for associating an image (face image) of the area surrounded by the face recognition frame 1908 that has received the selection as a face image of the student that has received the selection in the student selection unit 1903. .. By accepting the press of the "add face image" button 1910, the student ID of the student whose selection has been accepted by the student selection unit 1803 is inserted into the identification personal ID 1318 of the face data having the ID of the face image in FIG. Will be done.

Further, by performing the corresponding processing, the snapshot including the face image is controlled not to be displayed on the snapshot list display unit 1905, and the face image is additionally displayed on the identification result display unit 1912. ..

The "Delete face image" button 1911 deletes a snapshot (an image surrounded by a face recognition frame 1908) including a face image selected in the enlarged display unit 1907 from the snapshot list display unit 1905, and enlarges the display unit 1907. Is a button for ending the display of the enlarged photographic image. The above is the description of an example of the configuration of the subject identification screen according to the embodiment of the present invention shown in FIG.

Returning to the description of FIG. When the CPU 201 of the client PC 100 receives the student selection instruction, it transmits the information (identification information / for example, student ID) of the selected student to the server 200 (step S605). The server 200 receives this (step S606).

The CPU 201 of the server 200 acquires the face image data of the student (character face photograph 1313 in which the value is not inserted in the identification personal ID 1318 of the face 1310) to which the identification process of step S615 described later is not applied from the external memory. It is transmitted to the client PC 100 (step S607).

The CPU 201 of the client PC 100 receives the face image data and displays a list of the face image data on the subject identification screen 1800 on the display screen (step S608). When the CPU 201 of the client PC 100 receives an instruction to select face image data from the list of displayed face images, the CPU 201 displays the image data (snapshot) of the selected face image data on the display screen (FIG. 13). Snap photo display unit 1807).

The CPU 201 of the client PC 100 determines whether or not the correction instruction of the face image data recognition frame (face recognition frame) in the snapshot has been accepted (step S609). That is, it is determined whether or not the deformation instruction of the face recognition frame 1808 has been accepted.

When it is determined that the deformation instruction of the face recognition frame 1808 has been accepted (YES in step S609), the coordinate information (correction information) of the face recognition frame after the deformation is transmitted to the server 200, and the CPU 201 of the server 200 sends the correction information. Is received, the display on the display screen of the modified face recognition frame is deformed (step S611), and the process returns to step S607.

That is, in step S607, the screen information is transmitted to the client PC 100 in order to display the deformed face recognition frame on the subject identification screen 1800.

When it is determined that the deformation instruction of the face recognition frame 1808 is not accepted (NO in step S609), it is determined whether or not the face identification instruction is accepted (step S612). That is, it is determined whether or not the pressing instruction of the "face image addition" button 1810 in FIG. 18 has been accepted.

When it is determined that the face identification instruction has been accepted (YES in step S612), the face ID 1312 of the face image selected by the face image display unit 1805 (the face image selected by the snap photo display unit 1807) and the student selection unit 1803 The student ID of the selected student and the corresponding instruction of the face image and the student are transmitted to the server 200 (step S613).

The CPU 201 of the server 200 receives the information transmitted in step S613 (step S614), and inserts the student ID also received in step S614 into the identification personal ID 1318 of the face data held in the face ID 1312 received in step S614. (Step S615 / subject detection). Then, the process returns to step S607.

When it is determined that the face identification instruction is not accepted (NO in step S612), it is determined whether or not the photo confirmation instruction is accepted (step S616). That is, it is determined whether or not the instruction to press the "photo confirmation" button 1806 has been accepted.

When it is determined that the photo confirmation instruction has been accepted (YES in step S616), the screen information of the subject identification screen 1900 and the candidate including the face image similar to the student transmitted in step S605 to be displayed on the subject identification screen 1900. The image data is requested from the server 200 (step S617).

The CPU 201 of the server 200 receives the request (step S618), and transmits the screen information of the subject identification screen 1900 and the image data to be displayed on the subject identification screen 1900 to the client PC 100 (step S619).

The CPU 201 of the client PC 100 receives the screen information of the subject identification screen 1900 and the image data to be displayed on the subject identification screen 1900 and displays them on the display screen (step S620).

The CPU 201 of the client PC 100 determines whether or not the face registration instruction has been accepted (step S621). That is, it is determined whether or not the pressing instruction of the "add face image" button 1910 has been accepted.

When it is determined that the face registration instruction has been accepted (YES in step S621), the face ID of the face image selected by the enlarged display unit 1907 and the student ID of the student selected by the student selection unit 1903 are transmitted to the server 200. (Step S622).

The CPU 201 of the server 200 receives this (step S623) and stores it in association with the external memory (step S624 / subject detection). That is, the received student ID is inserted into the identification personal ID 1318 of the face data of the face 1310 having the face ID received in step S622. Then, the process returns to step S619.

When the CPU 201 of the client PC 100 determines in step S616 that the photo confirmation instruction is not received (NO in step S616), the process proceeds to step S625. If it is determined in step S621 that the face registration instruction is not accepted (NO in step S621), the process proceeds to step S625 in the same manner.

The CPU 201 of the client PC 100 determines whether or not the completion instruction of the subject identification process has been received (step S625). That is, for example, it is determined whether or not the press of the close button (not shown) is accepted on the subject identification screen 1900.

When the CPU 201 of the client PC 100 determines that the completion instruction of the subject identification process has been accepted (YES in step S625), the CPU 201 of the client PC 100 ends the process and determines that the completion instruction of the subject identification process has not been accepted (NO in step S625). , The process shifts to before step S605. The above is the description of the flow of the subject identification process in the embodiment of the present invention shown in FIG.

Here, the flow of the layout change processing in the embodiment of the present invention will be described with reference to FIG. 7. FIG. 7 is a flowchart showing a flow of layout change processing according to the embodiment of the present invention.

The CPU 201 of the client PC 100 requests the server 200 for information on the layout screen (page edit screen) in response to an instruction from the user (step S701).

The CPU 201 of the server 200 receives the request for the layout screen information (step S702) and transmits the layout screen information to the client PC 100 (step S703).

The CPU 201 of the client PC 100 receives the layout screen information and displays it on the display screen (step S704). If it is determined whether or not the instruction for image arrangement (arrangement of image data on the album page) has been accepted (step S705), and if it is determined that the instruction for image arrangement has been accepted (YES in step S705), the image data is arranged. The layout change instruction of the page is transmitted to the server 200 (step S706).

The layout change is a page layout change such as a change in the position of image data in the album page. The CPU 201 of the server 200 receives the layout change instruction, changes the layout of the album page according to the received instruction (step S707), and returns the process to step S703. That is, the screen information after the layout change is transmitted to the client PC 100 so as to be displayed on the display screen of the client PC 100.

If the CPU 201 of the client PC 100 determines in step S705 that it has not received the image arrangement instruction (NO in step S705), the process proceeds to step S708, and whether or not the instruction for equality determination for each section has been accepted. Is determined (step S708).

A section here is a unit that divides (classifies) the inside of an album. For example, when creating an album that summarizes photographic images for one year, when collecting each page / image data in the album into event units (section units) such as athletic meet, school festival, school trip, etc. Used. It is assumed that which page and which image data belongs to which section is stored in the external memory of the server 200 in the section management table (photo ID-section ID association table) (not shown). When the equality calculation process described later is executed for each section, the image data to be judged for equality is specified based on the information in the section management table (not shown), and the equality calculation process is executed. ..

Further, the equality calculation process (equality determination process) is a process of determining how equal each subject is in the image data in the same section or the same album.

In addition, as a result of the determination, the subject judged to be unequal, the image data of the subject is replaced, the image data including the subject is added to the album, or the display control of the image data for deleting from the album ( It is a process of displaying, highlighting, etc. of image data of replacement candidates.

The CPU 201 of the client PC 100 determines whether or not the instruction for calculating equality for each section has been received (step S708). When it is determined that the instruction for calculating equality for each section has been accepted (YES in step S708), the process proceeds to step S709, and the equality verification process for each section is executed. The details of the equality calculation process for each section will be described later in the description of FIG.

Here, with reference to FIG. 8, the flow of the section-based equality calculation process in the embodiment of the present invention will be described. It is a flowchart which shows the flow of the equality calculation process of a section unit in embodiment of this invention.

The CPU 201 of the client PC 100 requests the server 200 for screen information of the page edit screen (screen such as the page edit screen 1500) (step S801). The CPU 201 of the server 200 receives the request (step S802) and returns the screen information to the client PC 100 (step S803).

The CPU 201 of the client PC 100 displays a page edit screen (a screen such as the page edit screen 1500) on the display screen using the screen information received from the server 200 (step S804 / candidate image display control means).

Here, an example of the configuration of the page editing screen in the embodiment of the present invention will be described with reference to FIG. FIG. 15 is a diagram showing an example of the configuration of the page editing screen in the embodiment of the present invention.

As shown in FIG. 15, the page edit screen 1500 has an edit mode 1501, a scope 1502, a replacement mode 1503, a page structure 1504, a section property 1505, a photo property 1506, a snap photo display unit 1507, a page layout display unit 1508, and a selected snap. Photo display 1509, selected layout photo display 1510, "replace" button 1511, "automatic replacement" button 1512, "grade selection" button 1513, "class selection" button 1514, personal photo display 1515, number of appearances 1516, It is composed of a conspicuity degree 1517, a trimming frame 1518, a section 1519, and the like.

The edit mode 1501 indicates the mode of the page. That is, it indicates that the page edit screen 1500 is a screen for page edit and equality judgment. The scope 1502 indicates whether the page edit screen 1500 is a screen for determining equality for each section or a screen for determining equality for the entire album.

The replacement mode 1503 is a selection reception unit for accepting selection from which type of photographic image the candidate photographic image to be replaced with the laid-out photo on each page of the album is selected.

The page structure 1504 indicates which page in the album the page to be edited on the page edit screen 1500 is. Further, by accepting the selection of the page displayed in the page structure 1504, the page to be edited can be changed on the page edit screen 1500.

The section property 1505 is a property of the section including the page to be edited on the page edit screen 1500. The section property 1505 defines, for example, the minimum number of times and the maximum number of times each student should appear in a photographic image in the same section, and the content of the property is displayed on the page edit screen 1500.

The photo property 1506 is a photo image that has been laid out on the album page, and displays the replacement constraint information preset for the selected image. The replacement constraint information includes, for example, mandatory, scene constraint, event constraint, class constraint, and the like. Image data for which replacement constraint information is set is restricted so that it can only be replaced with image data having the same attribute information (for example, scene, event, and class information preset for each image data). Information. In addition, image data for which "required" is set as attribute information cannot be replaced.

The snapshot image display unit 1507 displays candidate image data which is a candidate image to be replaced with the layout image. It is also a selection reception unit that accepts selection of image data. The candidate image data is acquired from a storage area such as an external memory of the server 200.

The image data displayed on the snapshot display unit 1507 shall be displayed in ascending order based on the name of the image data when the page edit screen is opened for the first time. When the page edit screen is opened for the second time or later, the information of the page opened last time and the information of the image data shall be received from the server 200 and displayed.

For example, when the personal image displayed on the personal photo display unit 1515 or the selection of the column in which the personal image is displayed is accepted, the image of the individual (student) indicated by the personal image and the current page. Images (snapshots / candidate image data) that can be posted on are displayed in descending order of the student's face image.

The page layout display unit 1508 is a layout display unit that displays which image is laid out (posted) in which place on the page to be edited. It is also a selection reception unit that accepts selection of image data. The selected snap photo display unit 1509 is a display unit that enlarges and displays an image for which a selection instruction has been received among the photographic images displayed on the snap photo display unit 1507.

The selected layout photo display unit 1510 is a display unit that enlarges and displays an image that has received a selection instruction among the photographic images displayed on the page layout display unit 1508. The "replace" button 1511 is a button for replacing the image data selected by the snapshot display unit 1507 and the page layout display unit 1508.

The "automatic replacement" button 1512 uses, for example, the value of each student's appearance count 1516, which will be described later, so that the appearance count 1516 of each student in the section is equal (approximate value). This is a button for replacing the image data displayed / laid out on the display unit 1508 with the candidate image data.

The "grade selection" button 1513 accepts a selection of which grade of student information is to be displayed on the list display unit including the personal photo display unit 1515. The "class selection" button 1514 accepts a selection of which class of student information is to be displayed on the list display unit including the personal photo display unit 1515.

The personal photo display unit 1515 displays the number of appearances 1516 in the same section of the students of the class whose selection is accepted by the "grade selection" button 1513 and the "class selection" button 1514. The number of appearances 1516 indicates the number of times the student is listed in the same section.

The conspicuity degree 1517 indicates which student's image is conspicuous in the same section. The conspicuity 1517 is, for example, the sum of the conspicuity of the student (= the conspicuity of the student's face alone (= the square root of the size of the face on the page ÷ the square root of the size of the photo posted on the page) × constant. ).

The trimming frame 1518 indicates which region of the snap image (candidate image) is used to replace the laid-out image, and which region is selected in the selected image data.

Section 1519 represents one section (in the notation of FIG. 15, the section referred to as the "12th Year Cultural Festival"). The above is the description of an example of the configuration of the page editing screen in the embodiment of the present invention shown in FIG.

Returning to the description of FIG. The CPU 201 of the client PC 100 determines whether or not the instruction to change the automatic layout has been received (step S805). That is, it is determined whether or not the "automatic replacement" button 1512 is pressed.

When it is determined that the instruction for automatic layout change has been accepted (YES in step S805), the CPU 201 of the client PC 100 has image data arranged (laid out) on each page of the section including the page displayed on the page layout display unit 1508. The identification information of the above is transmitted to the server 200 (step S806), and the CPU 201 of the server 200 receives this (step S807).

The CPU 201 of the server 200 determines whether or not the image data is not arranged and there is an area in which the image data can be arranged on the page belonging to the section (step S808).

When it is determined that there is an area that can be arranged (YES in step S808), the image data is arranged in the area that can be arranged (step S809). The image data to be arranged here is randomly selected and arranged from the above-mentioned candidate image data (image data stored in the external memory of the server 200 and not arranged on the page of the album).

Hereinafter, the processes of steps S810 to S814 are repeated until the application of the processes of steps S811 to S814 is completed for all the layout image data (images / posted images already arranged on the page). After that, the process proceeds to step S803.
The CPU 201 of the server 200 acquires the student information included in the layout image data of the corresponding section and performs the equality calculation process (step S810).

For example, in the section, the number of times which student appears in the photographic image is totaled, and the image data is displayed on the student (for example, the personal photo display unit 1515 in FIG. 15) that should appear in the section. The score when the students) appear the same number of times is set to 100, and when there is a difference in the number of appearances, the score for equality is calculated by subtracting the score according to the value of the difference.

The CPU 201 of the client PC 100 has "essential" restrictions indicating that the layout image data (images / posted images already placed on the page) can be replaced, that is, the attribute information and properties cannot be replaced. One image data that has not been obtained is acquired (step S811), one image data that can be replaced with the acquired image data is acquired from the candidate image data, and the image data acquired in step S810 and the candidate are obtained. The score of equality when the image data is replaced is calculated (step S812).

The CPU 201 of the client PC 100 includes the layout image data acquired in step S811 and all candidate image data (image data other than the personal photographic image stored in the external memory of the server 200 and stored in step S408 of FIG. 4). The process of step S812 is applied to the combination of, and the score with the highest equality (the score closest to 0.0) among the calculated equality scores, the score calculated in step S810, and the score calculated in step S810. Is compared to determine whether or not the score is improved (step S813). That is, it is determined whether the score of equality after the image replacement is closer to 100 (score value) than the score calculated in step S810.

Here, the equality calculation process is executed for the combination of the layout image data acquired in step S811 and all the candidate image data. For example, the layout image data acquired in step S811 and the layout thereof. For combination with candidate image data belonging to the same section as the image data (image data indicated by a photo ID having the same section ID by referring to a section management table (correspondence table of photo ID and section ID) (not shown)). On the other hand, the equality calculation process may be executed.

When it is determined that the score is improved (YES in step S813), the image data corresponding to the score with the highest equality (the score closest to 0.0) calculated in step S812 and the layout image data Perform the replacement process (step S814 / replacement control). If it is determined that the score has not improved (NO in step S813), the images are not replaced and the process proceeds.

The CPU 201 of the client PC 100 applies the processes of steps S811 to S814 to all the layout image data, and then shifts the processes to step S803.

If the CPU 201 of the client PC 100 determines in step S805 that it has not received the instruction to change the automatic layout (NO in step S805), it shifts the process to step S815 and accepts the selection instruction of the less conspicuous student. (Step S815).

For example, in the case where the selection of the personal image displayed on the personal photo display unit 1515 is accepted on the page edit screen 1500 of FIG. 15, the value of the student conspicuity 1517 indicated by the selected personal image corresponds to the corresponding case. If it is less than the average value of all the prominence 1517 values corresponding to the students included in the section, it is determined that the selection instruction of the less conspicuous student has been accepted.

When it is determined that the CPU 201 of the client PC 100 has received the selection instruction of the student with low conspicuity (YES in step S815), the photo arrangement process of the student with low conspicuity is executed (step S816). The details of the photo arrangement processing of the less conspicuous student will be described later in the explanation of FIG.

Here, with reference to FIG. 9, a flow of a photo replacement process for an inconspicuous subject in the embodiment of the present invention will be described. FIG. 9 is a flowchart showing a flow of photo replacement processing of an inconspicuous subject in the embodiment of the present invention.

The CPU 201 of the client PC 100 transmits the student information selected according to the user's instruction, that is, the identification information for identifying the student (for example, attendance number / student information) to the server 200 (step S901 / subject selection reception). .. The CPU 201 of the server 200 receives the student information (step S902), and transmits the information of the image data of the student to the client PC 100 (step S903).

The CPU 201 of the client PC 100 receives the information of the image data, and among the image data, the candidate image data which is the image data not published in the album is displayed on the snapshot photo display unit 1507 of the page edit screen 1500 (step). S904).

Here, the CPU 201 of the server 200 transmits information on the image data of the student selected by the user to the client PC 100, and the CPU 201 of the client PC 100 displays the image data including the face of the student on the display screen. (Step S903), for example, in addition to the processing, an area including the student's face image in the image data including the student's face may be cut out and displayed on the display screen. ..

For example, when student A and student B are shown in the layout image data selected by the user, the CPU 201 of the server 200 extracts the image data N including the student A and student B from the external memory and sends it to the client PC 100. Send.

The CPU 201 of the client PC 100 displays the image data N on the display screen in a state where the area in which the student A and the student B are shown in the image data N is surrounded by a trimming frame. For example, the image data is displayed as shown in the page edit screen 1600 shown in FIG.

Here, an example of the configuration of the page editing screen in the embodiment of the present invention will be described with reference to FIG. FIG. 16 is a diagram showing an example of the configuration of the page editing screen in the embodiment of the present invention.

Since the configuration of the page editing screen 1600 is almost the same as the configuration of the page editing screen 1500 of FIG. 15, detailed description thereof will be omitted. The trimming frame 1601 of the page edit screen 1600 is an area for cutting out the subject to be reflected in the layout image data 1602 selected according to the operation instruction of the user from the candidate image data received from the server 200.

When the "replace" button is pressed, the CPU 201 of the client PC 100 selects the coordinate information of the trimming frame 1601, the identification information of the image to be trimmed, and the identification information of the selected layout image data 1602. The data is transmitted to the server 200, and the CPU 201 of the server 200 executes a process of exchanging the layout image data 1602 and the image (cut out image) of the area surrounded by the trimming frame 1601.

By doing so, for example, it is possible to prevent the equality score from being lowered due to an increase or decrease in the number of subjects due to image replacement. The above is the description of an example of the configuration of the page editing screen in the embodiment of the present invention shown in FIG.

Returning to the description of FIG. The CPU 201 of the client PC 100 also emphasizes the layout image data which is the image data published in the album among the received image data and is displayed on the page layout display unit 1508 of the page edit screen 1500. indicate. (Step S904).

The highlighting here means, for example, a process of changing the color of a rectangular line surrounding an image. By displaying the rectangular line thickly, it may be highlighted.

The CPU 201 of the client PC 100 determines whether or not the selection of the candidate image data has been accepted (step S905), and if it determines that the selection has been accepted (YES in step S905), the identification information of the candidate image data that has accepted the selection is used. It is transmitted to the server 200 (step S906).

The CPU 201 of the server 200 receives the information (step S907), and the section when the images are exchanged for each combination of the image data for which the selection is accepted and the image data displayed on the page layout display unit 1508. Equality score in (step S908).

The CPU 201 of the server 200 transmits information for display control according to the calculation result of the equality score in each combination to the client PC 100 (step S909).

When the CPU 201 of the client PC 100 receives the information and replaces the image data displayed on the page layout display unit 1508, the CPU 201 has the highest equality score (step S910 / replacement display). control).

Also, highlight the image data for which a score higher than the pre-replacement equality score (current equality score in the section) has been calculated (step S910). Further, the layout image data having the same attributes as the image data selected in step S905 is highlighted. Here, it is assumed that each image data is surrounded by different colors according to the type of highlighting to control the display of the highlighting.

The CPU 201 of the client PC 100 determines whether or not the layout image data selection instruction has been accepted (step S911), and if it determines that the selection has not been accepted (NO in step S911), returns the process to before step S905. ..

If it is determined that the selection has been accepted (YES in step S911), the identification information of the selected layout image data is transmitted to the server 200 (step S912). The CPU 201 of the server 200 receives this (step S913) and replaces the candidate image data selected in step S905 with the layout image data selected in step S911, which is a score in step S908. The calculated equality score information is transmitted to the client PC 100.

The CPU 201 of the client PC 100 receives the score information and displays it on the display screen (step S915). For example, it is displayed together with the equality information before replacement in a pop-up window (not shown).

The CPU 201 of the client PC 100 determines whether or not the replacement confirmation instruction has been accepted (step S916). For example, it is determined whether or not the pressing of the “replacement confirmation” button (for example, displayed in the pop-up window (not shown) described above) is accepted.

If it is determined that the replacement confirmation instruction has not been accepted (NO in step S916), the process returns to step S911. When it is determined that the replacement confirmation instruction has been accepted (YES in step S916), the process proceeds to step S917, and the replacement confirmation instruction is transmitted to the server 200 (step S918).

The CPU 201 of the server 200 receives the instruction for confirming the replacement (step S918), executes the image replacement process (step S919) in response to the instruction, and returns the process to step S903.

When the CPU 201 of the client PC 100 determines in step S905 that the selection of the candidate image data is not accepted (NO in step S905), the process proceeds to step S920 and determines whether or not the page editing end instruction has been received. (Step S920).

When it is determined that the page edit end instruction has been received (YES in step S920), the process ends. If it is determined that the page edit end instruction has not been received (NO in step S920), the process returns to step S901. The above is the description of the flow of the photo-replacement process of an inconspicuous subject in the embodiment of the present invention shown in FIG.

Returning to the description of FIG. When the CPU 201 of the server 200 determines in step S816 that the selection instruction of the student with low conspicuity is not accepted (NO in step S815), the process shifts to step S817 and the selection instruction of the student with high conspicuity is accepted. It is determined whether or not it is (step S817).

For example, in the case where the selection of the personal image displayed on the personal photo display unit 1515 is accepted on the page edit screen 1500 of FIG. 15, the value of the student conspicuity 1517 indicated by the selected personal image corresponds to the corresponding case. If it exceeds the average value of all the prominence 1517 values corresponding to the students included in the section, it is determined that the selection instruction of the highly conspicuous student has been accepted.

When it is determined that the selection instruction of the highly conspicuous student has been accepted (YES in step S817), the process proceeds to step S818, and the photo arrangement process of the highly conspicuous student is executed (step S818). The details of the photo arrangement process of the highly conspicuous student will be described later in the explanation of FIG.

When it is determined that the selection instruction of the highly conspicuous student is not accepted (NO in step S817), the process ends. The above is the description of the flow of the equality calculation process for each section in the embodiment of the present invention shown in FIG.

Next, with reference to FIG. 10, a flow of a conspicuous subject photo replacement process in the embodiment of the present invention will be described. FIG. 10 is a flowchart showing a flow of a conspicuous subject photo replacement process in the embodiment of the present invention.

Since the processes of steps S1001 to S1004, steps S1015 to S1019, and steps S1020 correspond to the processes of S901 to S904, steps S915 to S919, and step S920 of FIG. 9, the description thereof is omitted here.

The CPU 201 of the client PC 100 determines whether or not the selection of the layout image data (posted image) displayed on the page layout display unit 1508 has been accepted (step S1005). When it is determined that the selection of the layout image data is not accepted (NO in step S1005), the process proceeds to step S1020.

When it is determined that the selection of the layout image data has been accepted (YES in step S1005), the identification information of the layout image data is transmitted to the server 200 (step S1006).

The CPU 201 of the server 200 receives the information (step S1007), and the image data for which the selection is accepted and the image data in which the same student as the student appearing in the image data appears, which can be replaced with the image data. For each combination, the score of equality in the section when the images are swapped is calculated (step S1008).

The CPU 201 of the server 200 transmits the image data that can be replaced with the selected layout image data and the information for display control according to the calculation result of the equality score in each combination to the client PC 100 (step S1009). ..

The CPU 201 of the client PC 100 receives the information and displays the information on the snapshot display unit 1507 in descending order of score (score = close to 0.0) (step S1010 / replacement display control). The CPU 201 of the client PC 100 determines whether or not the selection instruction of the candidate image data has been accepted (step S1011), and if it determines that the selection has not been accepted (NO in step S1011), returns the process to before step S9105. ..

If it is determined that the selection has been accepted (YES in step S1011), the identification information of the selected candidate image data is transmitted to the server 200 (step S1012). The CPU 201 of the server 200 receives this (step S1013) and replaces the layout image data selected in step S1005 with the candidate image data selected in step S1011, which is a score in step S1008. The calculated equality score information is transmitted to the client PC 100. The above is the description of the flow of the photo-replacement process of a conspicuous subject in the embodiment of the present invention in FIG.

Returning to the description of FIG. When the CPU 201 of the client PC 100 determines that the instruction for determining equality for each section is not received (NO in step S708), the process proceeds to step S710. The CPU 201 of the client PC 100 determines whether or not the instruction for determining the equality of the student's photographs in the entire album has been accepted (step S710).

When it is determined that the instruction for determining the equality of the student's photographs in the entire album has been accepted (YES in step S708), the process proceeds to step S711, and the album equality verification process is executed (step S711).

Since the details and flow of the album equality verification process are almost the same as the flow of the section-based equality calculation process of FIG. 8, the detailed explanation is omitted here. Hereinafter, the difference between the album equality verification process and the section-based equality calculation process (step S709) will be described.
For example, in the album equality verification process, "album" is selected and displayed on the scope 1502 of the page edit screen 1500 of FIG.

If it is determined in step S805 of FIG. 8 that the instruction to change the automatic layout has been accepted (YES in step S805), the CPU 201 of the client PC 100 is arranged on each page of the album including the page displayed on the page layout display unit 1508. The identification information of the (laid-out) image data is transmitted to the server 200 (step S806).

Further, in step S808, the CPU 201 of the server 200 determines whether or not the image data is not arranged and there is an area in which the image data can be arranged in each page belonging to the album selected in the scope 1502 (step S808). ..

Further, in step S810, the CPU 201 of the server 200 acquires the student information included in the layout image data of the corresponding album and performs the equality calculation process (step S810).

For example, the conspicuity of a single face in each image (= square root of the size of the face on the page ÷ the square root of the size of the photo posted on the page) and the conspicuity of the student (= the sum of the conspicuity of the face of the student) Value x constant) and equality score (= (1-Gini coefficient calculated from student conspicuity) x 100), and equality score (the larger 0.0 to 1.0 is, the more unequal) And. The following is an example of a calculation formula for calculating the equality score. Further, the same formula as the calculation formula is shown in FIG.

Further, when the CPU 201 of the client PC 100 determines in step S805 that the instruction for automatic layout change is not accepted (NO in step S805), the process shifts to step S815 and the selection instruction of the less conspicuous student is accepted. It is determined whether or not the client has it (step S815).

For example, in the case where the selection of the personal image displayed on the personal photo display unit 1515 is accepted on the page edit screen 1500 of FIG. 15, the value of the student conspicuity 1517 indicated by the selected personal image corresponds to the corresponding case. If it is less than the average value of all the prominence 1517 values corresponding to the students included in the album, it is determined that the selection instruction of the less conspicuous student has been accepted.

If the CPU 201 of the server 200 determines in step S816 that it has not received a selection instruction for a student with a low degree of conspicuity (NO in step S815), the process shifts to step S817, and the selection instruction for a student with a high degree of conspicuity is given. Is determined (step S817).

For example, in the case where the selection of the personal image displayed on the personal photo display unit 1515 is accepted on the page edit screen 1500 of FIG. 15, the value of the student conspicuity 1517 indicated by the selected personal image corresponds to the corresponding case. If it exceeds the average value of all the prominence 1517 values corresponding to the students included in the album, it is determined that the selection instruction of the highly conspicuous student has been accepted. The above is an explanation of the difference between the album equality verification process and the section-based equality calculation process (step S709).

When the CPU 201 of the client PC 100 determines that it has not received the instruction for determining the equality of the student's photographs in the entire album (NO in step S710), the process ends. The above is the description of the flow of the layout change processing in the embodiment of the present invention shown in FIG. 7.

In the above description of the embodiment of the present invention, the method of performing the equality calculation process (equality determination process) for each section has been described. However, for example, the equality calculation process is performed for each project. You may.

For example, when "album ID = 0001" is selected in the scope 1502 of FIG. 15, and the equality calculation process is executed, the candidate image data (external memory of the server 200) to be the target of the calculation process is executed. It is the image data stored in the image data to be replaced with the image data being published in the album, and the photograph 1300) having the same project ID 1301 was acquired and associated with "album ID = 0001". Image data is acquired (see Photo Arrangement Information 1130), and equality calculation processing is executed when the respective images are exchanged.

Further, in the above description of the embodiment of the present invention, the student's selection is accepted, the photographic image (image data) in which the student's face is detected is extracted and displayed on the display screen, and the photographic image is selected. By accepting, the equality in the album or the equality in the section when the image data is replaced with another image data is calculated. For example, the face reflected in the image data for which the selection is accepted is detected. Then, the equality score when the image data and each image data of the replacement candidate are replaced may be calculated, and the image data of the replacement candidate may be displayed on the display screen in descending order of the equality score. ..

Specifically, the CPU 201 of the client PC 100 accepts the selection of the image data displayed on the display screen (for example, the image data displayed on the page layout display unit 1508 of FIG. 15) and transmits the information of the image data to the server 200. Send to.

The CPU 201 of the server 200 detects the face image included in the image data that has received (received) the selection, and replaces the image data with each image data of the replacement candidates in the external memory, in the album. -The equality in the section is calculated, the image data of the candidate is sorted in the order in which the equality is judged to be high, and the image data is transmitted to the client PC 100.

The CPU 201 of the client PC 100 receives the image data and displays the image data on the display screen (for example, the snapshot display unit 1507 of FIG. 15) in the sorted order. The CPU 201 of the client PC 100 is instructed to replace the image data displayed on the display screen (for example, the image data displayed on the page layout display unit 1508 of FIG. 15 and the image data displayed on the snap photo display unit 1507 of FIG. 15). By accepting the replacement instruction), the information of the image data to be replaced may be transmitted to the server 200, and the CPU 201 of the server 200 may execute the replacement process.

Further, for example, by accepting the selection of the image data displayed on the display screen (snap photo display unit 1507 of FIG. 15), detecting the face image included in the image data for which the selection is accepted, and replacing the image data with the image data. , The image data in which the score of equality in the section or in the album is improved (for example, the most improved) is extracted from the image data posted on each page of the album and displayed on the page layout display unit 1508 of FIG. You may try to do it.

By doing so, the user can ensure the equality of the image data in the album without being aware of which subject is included in the image data to be posted in the album.

As described above, according to the present invention, the image data of the selection candidate can be presented based on the information of the selected image data.

It should be noted that the structure and contents of the various data described above are not limited to this, and it goes without saying that the structure and contents are various depending on the intended use and purpose.

For example, it can be implemented as a system, an apparatus, a method, a program, a recording medium, or the like, and specifically, it may be applied to a system composed of a plurality of devices, or one. It may be applied to a device consisting of devices.

A recording medium on which a program that realizes the functions of the above-described embodiment is recorded is supplied to the system or device, and the computer (or CPU or MPU) of the system or device reads and executes the program stored in the recording medium. Needless to say, the object of the present invention is achieved.

In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium storing the program constitutes the present invention.

Recording media for supplying programs include, for example, flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, and silicon. Disks, solid state drives, etc. can be used.

Further, by executing the program read by the computer, not only the function of the above-described embodiment is realized, but also the OS (operating system) or the like running on the computer is actually operated based on the instruction of the program. Needless to say, there are cases where a part or all of the processing is performed and the processing realizes the functions of the above-described embodiment.

Further, after the program read from the recording medium is written in the memory provided in the function expansion board inserted in the computer or the function expansion unit connected to the computer, the function expansion board is based on the instruction of the program code. It goes without saying that there are cases where the CPU or the like provided in the function expansion unit performs a part or all of the actual processing, and the processing realizes the functions of the above-described embodiment.

Further, by downloading and reading a program for achieving the present invention from a server, database, or the like on the network by a communication program, the system or device can enjoy the effect of the present invention. It should be noted that the present invention also includes all configurations in which each of the above-described embodiments and modifications thereof are combined.

1 Network 100 Client PC
200 servers

Claims (11)

An information processing device that stores multiple selected images,
An image identification means for identifying a candidate image for newly accepting selection based on the values of a plurality of subjects appearing in the plurality of images.
A program for functioning as a recognition display control means for recognizablely displaying a difference in the value of the subject when a candidate image specified by the image specifying means is selected and when the image is not selected. An information processing device that stores multiple selected images,
An image that identifies a candidate image to be replaced with the selected image based on a comparison result of subject values in the selected image and a reference value for identifying an image to be replaced with any of the selected images. A program to function as a specific means. The program according to claim 1 or 2, wherein the value of the subject is a score calculated based on the number of appearances of the subject in a plurality of selected images. Of the information processing apparatus, which further stores which album the selected plurality of images are included in.
A claim for making the image specifying means function as a means for identifying a candidate image for which a new selection is accepted based on the values of a plurality of subjects appearing in the plurality of images included in the album for each album. The program according to any one of 1 to 3. Of the information processing apparatus, which further stores which section of which album contains the selected plurality of images.
To make the image identifying means function as a means for identifying a candidate image for which a new selection is accepted based on the values of a plurality of subjects appearing in the plurality of images included in the section for each section in the album. The program according to any one of claims 1 to 4. A control method for an information processing device that stores a plurality of selected images.
An image identification step in which the image identification means identifies an image that is a candidate for accepting a new selection based on the values of a plurality of subjects appearing in the plurality of images.
A recognition display control step that recognizablely displays the difference in the value of the subject between the case where the candidate image specified by the image identification step is selected and the case where the image is not selected.
A control method for an information processing device, which comprises. An information processing device that stores a plurality of selected images.
An image identification means for identifying a candidate image for newly accepting selection based on the values of a plurality of subjects appearing in the plurality of images.
A recognition display control means for recognizablely displaying the difference in the value of the subject between the case where the candidate image specified by the image identification means is selected and the case where the image is not selected.
An information processing device characterized by being equipped with. An information processing system including a client device and an information processing device that stores a plurality of selected images.
The client device
Display hand stage received from the information processing apparatus, based on the information for identifying the image, displays the image
With
The information processing device
An image identification means for identifying a candidate image for newly accepting selection based on the values of a plurality of subjects appearing in the plurality of images.
A recognition display control means that controls the client device to recognize the difference in the value of the subject when the candidate image specified by the image specifying means is selected and when the image is not selected.
A transmission means for transmitting information for identifying an image identified by the image identification means to the client device, and a transmission means.
An information processing system characterized by being equipped with. A control method for an information processing device that stores a plurality of selected images.
An image that identifies a candidate image to be replaced with the selected image based on a comparison result of subject values in the selected image and a reference value for identifying an image to be replaced with any of the selected images. Specific process
A control method for an information processing device, which comprises. An information processing device that stores a plurality of selected images.
An image that identifies a candidate image to be replaced with the selected image based on a comparison result of subject values in the selected image and a reference value for identifying an image to be replaced with any of the selected images. An information processing device characterized by being provided with specific means. An information processing system including a client device and an information processing device that stores a plurality of selected images.
The client device
A display means for displaying an image based on information for identifying an image received from the information processing device.
With
The information processing device
An image that identifies a candidate image to be replaced with the selected image based on a comparison result of subject values in the selected image and a reference value for identifying an image to be replaced with any of the selected images. specific hand stage
An information processing system characterized by being equipped with.

Images