JP7574880B1 - Recording device and program - Google Patents

Abstract

[Problem] To provide a recording device etc. that leads to the promotion of the use of photos etc. [Solution] A user terminal 2 has a display means 210 that displays photos on a display, an acquisition means 220 that acquires passive reaction data of a user based on the emotions felt when viewing a photo displayed on the display, and a recording means 230 that records the result of the user's reaction when viewing a photo based on the reaction data, linking it to the photo. The reaction data is, for example, biometric information of the user, and the acquisition means 220 receives the reaction data from a wearable device 3 worn by the user. [Selected Figure] Figure 4

Description

The present invention relates to a recording device that promotes the effective use of images such as photographs.

As photography becomes digitalized and the number of photos taken increases dramatically, the task of printing good photos and compiling them into a photo book becomes more involved, and often the photos you've taken end up going unnoticed.

In response to this, Patent Document 1 discloses a system that makes it easy to search for "good photos" using objective criteria such as the sharpness of the focus, the quality of the composition, and the degree to which the subject smiles, helping to make effective use of the photos that have been taken.

JP 2007-097090 A

However, the things that make us feel excited, thrilled, or evoke memories when we see them in real life do not necessarily match what we consider to be "good photos" analyzed using the objective criteria mentioned above.

For example, we may sense a sense of dynamism in an out-of-focus, blurry photograph, or feel attached to a child's face that is visible in the image. A photograph of a crying child may also evoke feelings of love for the child's parents that are as strong as a smiling face, making it difficult to objectively measure the importance of such photographs using criteria such as focus, composition, or the degree of the smile.

The value of a photograph lies in the fact that it allows us to remember the excitement, thrills, and memories we feel when we see it, and evokes them every time we look at it. However, conventional technology has not been able to effectively search for or extract photos that have this subjective importance, and this has created challenges in terms of promoting the use of photos.

The present invention was made in consideration of the above problems, and aims to provide a recording device etc. that will lead to the promotion of the use of photographs etc.

The first invention for solving the above-mentioned problems is a recording device that records a user's reaction results when viewing an image by linking the result to the image, characterized in that the recording device has: a display means that displays an existing image stored in a memory unit of the recording device or a computer connected to the recording device via a network on a display in response to user operation; an acquisition means that acquires passive reaction data of the user based on emotions when viewing the image displayed on the display; and a recording means that records the user's reaction results when viewing the image based on the reaction data by linking the result to the image.
The recording device repeats the process of displaying the images on the display, acquiring the reaction data, and recording the reaction results in association with the images, for example, until all of the images to be displayed have been displayed.

In this invention, passive reaction data based on the emotions of people who view images such as photographs is acquired, and the reaction results of the people when they viewed the image are linked to the image and recorded. The same image will have different reactions depending on the person viewing it, and these reactions often differ from the evaluation of the image based on objective measures. In this invention, by assigning the user's emotional (subjective) evaluation to the image, such as "I was excited when I saw the image," "It made my heart flutter," or "It evoked memories from the past," it becomes possible to search for or extract images later based on these evaluations, contributing to the effective use of images. In addition, the operation is simple, as users only need to look at the photo and no active actions such as entering their impressions in text or pressing the "Like" button are required.

It is desirable that the reaction data is biometric information of a user, and that the acquisition means receives the reaction data from a wearable device worn by the user.
The reaction data may be biological information such as the user's body temperature or heart rate, and may also include reactions that the user is not particularly aware of.

It is desirable that the reaction data is a captured image of the user showing a facial expression of the user, and that the acquisition means acquires the captured image by photographing the user.
The reaction data may be a captured image showing the user's facial expression. In this case, known image recognition technology or the like may be used to determine the type of emotion the user is feeling from the captured image and record it as the reaction result.

It is also preferable that the recording device is a mobile terminal equipped with the display.
The recording device may be, for example, a mobile terminal equipped with a display, such as a smartphone, and may record the user's reaction to an image displayed on the smartphone or the like.

A second invention is a program for causing a computer to function as a recording device that records the results of a user's reaction when viewing an image by linking the image to the recording device, the recording device having: a display means that displays on a display an existing image stored in the memory unit of the recording device or a computer connected to the recording device via a network in response to user operation; an acquisition means that acquires passive reaction data of the user based on the emotions felt when viewing the image displayed on the display; and a recording means that records the results of the user's reaction when viewing the image based on the reaction data by linking the image to the image.

This invention makes it possible to provide a recording device etc. that will lead to increased use of photography etc.

FIG. 1 shows a recording system 1. FIG. 2 shows a configuration of a user terminal 2. FIG. 2 is a diagram showing the configuration of a wearable device 3. FIG. 2 is a diagram showing the functions of the recording system 1. 4 is a flowchart showing a process in the recording system 1. 1 is an example of a system including a server 4. 13 is a flowchart for a case where a photographed image of a user is used as reaction data.

The following describes in detail a preferred embodiment of the present invention with reference to the drawings.

(1. Recording System 1)
1 is a diagram showing a recording system 1 including a user terminal 2 according to an embodiment of the recording apparatus of the present invention. The recording system 1 records a user's reaction when viewing an image displayed on the display of the user terminal 2 in association with the image, and has a configuration in which the user terminal 2 and a wearable device 3 are communicatively connected as shown in FIG.

The user terminal 2 is a terminal owned by the user, and is a computer (computer terminal) that performs various processes related to recording the above-mentioned reaction results. In this embodiment, a mobile terminal such as a smartphone or a tablet terminal is used as the user terminal 2.

Figure 2 is a diagram showing the configuration of the user terminal 2. As shown in Figure 2, the user terminal 2 is configured by connecting a control unit 21, a storage unit 22, a display 23, an input unit 24, a communication unit 25, a camera 26, an audio input/output unit 27, etc., via a bus or the like. However, the user terminal 2 is not limited to this and can have various configurations as appropriate.

The control unit 21 is composed of a CPU, ROM, RAM, etc. The CPU loads programs related to the processing of the user terminal 2 stored in storage media such as the memory unit 22 and ROM into a work area on the RAM, executes them, and controls each part of the user terminal 2 to realize the processing described below. The ROM is a non-volatile memory, and permanently stores programs such as the boot program and BIOS, as well as data. The RAM is a volatile memory, and temporarily stores programs and data loaded from the memory unit 22, ROM, etc., and has a work area that the control unit 21 uses to perform various processes.

The storage unit 22 is a flash memory or the like, and stores programs executed by the user terminal 2 during processing described below, data required for program execution, an OS (operating system), etc. These programs and data are read and executed by the control unit 21 as necessary.

The display 23 is, for example, a liquid crystal panel, and is realized as a touch panel display that also serves as the input unit 24. The display 23 displays various screens related to the operation and selection of the user terminal 2, and accepts user input of instructions, etc., via touch panel operations on the screen.

The communication unit 25 is a communication interface that mediates communication via wireless communication, etc., and communicates with the wearable device 3, etc.

The camera 26 is an area camera having an imaging element such as a CCD (Charged Coupled Device), and the audio input/output unit 27 has a microphone for audio input and a speaker for audio output.

Returning to the explanation of FIG. 1, the wearable device 3 is a terminal worn by the user, for example a wristband-type or watch-type terminal. However, the wearable device 3 is not limited to this, and earphone-type, earring-type, bracelet-type, headgear-type, or eyeglass-type terminals can also be used.

Figure 3 is a diagram showing the configuration of the wearable device 3. As shown in Figure 3, the wearable device 3 has a main controller 31, a display 32, a body temperature sensor 33, a heart rate sensor 34, a wireless module 35, etc.

The main controller 31 is a control unit that controls the entire wearable device 3, and has a CPU, ROM, RAM, memory, etc. The CPU, ROM, and RAM are substantially the same as those described above. The memory stores various setting information for the wearable device 3, measurement data measured by the wearable device 3, etc.

Display 32 is a simple display such as a small LCD panel or electronic paper.

The body temperature sensor 33 and the heart rate sensor 34 measure the body temperature and heart rate of the user, respectively, as biometric information of the user wearing the wearable device 3. Known sensors can be used for the body temperature sensor 33 and the heart rate sensor 34.

The wireless module 35 is a communication unit that performs wireless communication with the user terminal 2 and transmits measurement data from the above-mentioned sensors 33 and 34 to the user terminal 2, and is equipped with an antenna, etc.

(2. Functions of Recording System 1)
Fig. 4 is a block diagram showing the functions of the recording system 1. As shown in Fig. 4, in the recording system 1, the user terminal 2 (recording device) has a display unit 210, an acquisition unit 220, a recording unit 230, and the like.

The display means 210 causes the control unit 21 of the user terminal 2 to display the photographs (still images) stored in the memory unit 22 on the display 23.

The acquisition means 220 is for the control unit 21 of the user terminal 2 to acquire passive reaction data of the user based on the emotions felt when viewing a photo displayed on the display 23. Passive reaction data is data indicating reactions that do not involve active actions such as inputting impressions in text or pressing a "Like" button. This reaction data is, for example, the biometric information of the user described above, but is not particularly limited as long as it is based on the user's emotions.

The recording means 230 is a means by which the control unit 21 of the user terminal 2 records the user's reaction results when viewing a photo based on the above-mentioned reaction data, linking the reaction results to the photo.

(3. Processing of Recording System 1)
Next, with reference to Fig. 5 etc., a description will be given of the processing executed in the recording system 1. Fig. 5 is a flowchart showing the processing in the recording system 1, and each processing in Fig. 5 is executed by the control unit 21 of the user terminal 2 by controlling each unit of the user terminal 2.

In this embodiment, the user terminal 2 launches an application in response to a user operation and displays the photos stored in the memory unit 22 on the display 23 (S1).

The user looks at the image displayed on the display 23. At this time, the wearable device 3 continuously measures the user's biological information and transmits the measured biological information to the user terminal 2. The user terminal 2 receives and acquires the biological information as reaction data (S2). As described above, the biological information is body temperature measured by the body temperature sensor 33 and heart rate measured by the heart rate sensor 34, but is not limited to these. For example, it may be brain waves, etc.

The user terminal 2 analyzes the reaction data (S3), and if the reaction data indicates the user's emotions such as joy, surprise, or calm (S4; YES), the user terminal 2 records information indicating that emotion occurred when the user viewed the photograph as a reaction result, linking it to the photograph (S5). If the biometric information does not indicate any particular emotion of the user (S4; NO), the process proceeds directly to S6.

The method of analyzing and judging the reaction data is not particularly limited, but for example, in S3, the value of the reaction data or its change over time is compared with a predetermined threshold, and in S4, if the value of the reaction data or its change over time is equal to or greater than the threshold, it can be judged that an emotion corresponding to the threshold (e.g., joy, etc.) is expressed in the reaction data. Alternatively, if the value of the reaction data is less than the threshold, it can be judged that an emotion corresponding to the threshold (e.g., calmness, etc.) is expressed in the reaction data. In addition, two or more types of bioinformation may be combined and used for analysis and judgment, or one type of bioinformation may be used for analysis and judgment.

The user terminal 2 repeats the processes from S1 to S5 until all photos to be displayed have been displayed (S6; NO), and ends the process when all photos have been displayed (S6; YES).

The reaction results recorded in this way can be used to search and extract photos. For example, photos that evoked emotions such as joy in the viewer can be easily searched and extracted using the reaction results as a key, and compiled into a photo book or the like. Furthermore, by recording the viewer's emotions as added value to the photo, it is possible to learn the viewer's feelings about the photos left behind after their death. Furthermore, by having celebrities or athletes as viewers, it is possible to sell photos with added value in the form of the emotions of the celebrity or athlete, etc.

As explained above, in this embodiment, passive reaction data based on the emotions of the person who viewed the photo is acquired, and the reaction of the person when viewing the photo is linked to the photo and recorded. The same photo will have different reactions depending on the person viewing it, and these reactions often differ from the evaluation of the photo based on objective measures. In this embodiment, by assigning the user's emotional (subjective) evaluation to the photo, such as the photo making the user excited, making the heart flutter, or evoking memories from the past, it becomes possible to later search or extract photos based on these evaluations, which contributes to the effective use of photos. In addition, the user only needs to look through many photos at once, and there is no need for active actions such as entering impressions in text or pressing the "Like" button, so operation is simple.

In addition, in this embodiment, the above reaction data is treated as biometric information such as the user's body temperature and heart rate, making it possible to record reactions that the user is not particularly aware of.

However, the present invention is not limited to the above embodiment. For example, in this embodiment, photos stored in the user terminal 2 are displayed, but as shown in FIG. 6, photos may be stored in a server 4 (computer) connected to the user terminal 2 via a network in association with a user account, etc., and the photos may be sent to the user terminal 2 and displayed on the display 23.

In this case, the user terminal 2 receives the reaction data from the wearable device 3 and requests the server 4 to record the reaction results, or the server 4 receives the reaction data (measured by the wearable device 3) from the user terminal 2 and records the reaction results. In the former case, the user terminal 2 functions as the recording device of the present invention, and in the latter case, the server 4 functions as the recording device of the present invention.

In addition, in this embodiment, biometric information measured by the wearable device 3 is used as passive reaction data of the user, but currently, research is progressing on image recognition technology using AI (artificial intelligence) and the like that analyzes emotions from images of people's facial expressions. Therefore, instead of biometric information, it is also possible to use images of the user's facial expressions captured by the camera 26 of the user terminal 2.

The flow in this case is shown in Figure 7, which differs from the flow in Figure 5 described above in that instead of receiving reaction data from the wearable device 3 (see S2 in Figure 5), the user terminal 2 activates the camera 26 to capture the user's facial expression and obtains the captured image as reaction data (S2' in Figure 7). In S3, the above-mentioned image recognition technology is used to analyze the user's facial expression in the captured image, and in S4 it is determined whether or not the facial expression shows any emotion. If the user's facial expression shows any emotion, the type of emotion (happiness, surprise, calm, etc.) can be determined and the result recorded as a reaction result.

In this embodiment, the user's reaction results can be recorded for a photo displayed on a user terminal 2 such as a smartphone, but it is also possible to send the photo from the user terminal 2 to a wearable device 3 such as a head-mounted display for display, and it is also possible to photograph the user's facial expression with the wearable device 3 and transmit the captured image to the user terminal 2 as reaction data. It is also possible to obtain biological information such as pupil size from the captured image of the user as reaction data.

In addition, in this embodiment, a photo (still image) is displayed and the reaction results to the photo are recorded in association with the photo, but a video (moving image) may be displayed instead of a photo, and the reaction results to the video may be recorded in association with the video.

The above describes preferred embodiments of the present invention with reference to the attached drawings, but the present invention is not limited to these examples. It is clear that a person skilled in the art can come up with various modified or revised examples within the scope of the technical ideas disclosed in this application, and it is understood that these also naturally fall within the technical scope of the present invention.

1: Recording system 2: User terminal 3: Wearable device 4: Server 23, 32: Display 210: Display means 220: Acquisition means 230: Recording means

Claims (5)

A recording device that records a user's reaction to an image in association with the image,
a display unit that displays, in response to a user's operation, the existing images stored in a storage unit of the recording device or in a computer connected to the recording device via a network on a display;
an acquisition means for acquiring passive reaction data of a user based on emotions felt when viewing the image displayed on the display ;
a recording means for recording a reaction result of the user when viewing the image based on the reaction data in association with the image;
A recording apparatus comprising: The reaction data is biometric information of a user,
2. The recording apparatus according to claim 1, wherein the acquisition means receives the reaction data from a wearable device worn by a user. the reaction data is a captured image of the user showing a facial expression of the user;
2. The recording apparatus according to claim 1, wherein the acquisition means captures an image of the user by photographing the user. The recording device according to claim 1, characterized in that the recording device repeats the process of displaying the images on the display, acquiring the reaction data, and recording the reaction results in association with the images until all of the images to be displayed are displayed . Computer,
A recording device that records a user's reaction to an image in association with the image,
a display unit that displays, in response to a user's operation, the existing images stored in a storage unit of the recording device or in a computer connected to the recording device via a network on a display;
an acquisition means for acquiring passive reaction data of a user based on emotions felt when viewing the image displayed on the display ;
a recording means for recording a reaction result of the user when viewing the image based on the reaction data in association with the image;
A program for causing the device to function as a recording device having the above configuration.

Images