JP5662549B1 - Memory playback device - Google Patents

Abstract

[PROBLEMS] To identify locations corresponding to four types of events such as “meal”, “conversation”, “drug intake”, and “laughter” from video and audio records captured by the user's visual and auditory senses. Provided is a storage / reproduction device capable of efficiently reading and reproducing. In a recording / reproducing apparatus, a processing unit is built in an arm of a frame. The processing unit 16 generates events such as “conversation”, “laughter”, “meal”, and “drug intake” based on information captured by the camera 3, the microphone 4, the bone conduction sensor 6, and the pressure sensor 7. The event determination means 16a for determining the timing, the data search means 16b for extracting necessary information from the storage section 15, and the control means 16c for controlling the operation of the storage section 15 and the like. [Selection] Figure 2

Description

  The present invention relates to a storage / playback apparatus configured to record and reproduce video and audio captured by the user's visual and auditory senses, and particularly to a memory that can efficiently reproduce desired video and audio. The present invention relates to a playback device.

Human memory includes short-term memory that lasts for several tens of seconds and long-term memory that is permanently retained, and the decline in memory with age is mainly due to the decline in short-term memory. . As a result, the elderly often can remember the contents of the last action or conversation. At this time, if there are people in the vicinity, they can be taught about the last action, etc., so the forgotten content can be remembered relatively easily. Even if there are no people around you, if you are at home, you can obtain the same effect by shooting your actions with a fixed camera and viewing the video. However, if you are in a place where a fixed camera cannot be installed and there are no people around you, you cannot take these measures.
Therefore, in order to deal with such problems, in recent years, a technology relating to a device that is worn on the body and records the surrounding situation and the wearer's own behavior has been developed, and some inventions and devices have already been disclosed in this regard. Has been.

For example, Japanese Patent Laid-Open No. 2004-228561 is an apparatus that includes a magnetic disk with the name “storage system”, records various information on behalf of a person, and reproduces the recording to assist the recall of storage. An invention related to this is disclosed.
The invention disclosed in Patent Document 1 includes a camera, a microphone, a GPS, a temperature sensor, a vibration sensor, and the like, extracts a preset event pattern from information captured by these sensors, and assigns an identifier. It has a configuration.
According to such a structure, information related to the surrounding situation is taken in without being conscious of the user and is stored in the magnetic disk after being recognized and processed, so that necessary information can be retrieved and reproduced as needed. Can do.

Patent Document 2 discloses an invention relating to an apparatus that detects the occurrence of an event “to eat” with the name “event sensor”.
The invention disclosed in Patent Document 2 is based on the pressure detection signal, voice signal, and temperature detection signal acquired by the pressure sensor, the bone conduction microphone, and the radiation thermometer, to determine whether or not the patient (wearer) has eaten. It determines and transmits the event occurrence information.
According to such a configuration, the patient is strongly aware that he / she has eaten, so that he / she can prevent forgetting to take the medicine after the meal.

JP 2005-94170 A JP 2009-22504 A

  However, in the invention disclosed in Patent Document 1 which is the above-described prior art, among the actions of the user, “meal”, “conversation”, “drug intake”, and “laughter” are particularly important. Since it was not the structure which can recognize that four kinds of events occurred, there was a subject that it could not be used as an aid to remember the memory that the elderly forgot.

  Further, in the invention disclosed in Patent Document 2, among the four types of events described above, “meal” can be determined, but “drug intake”, “conversation”, and “laughter” cannot be determined. was there.

  The present invention has been made in response to such a conventional situation, and "meal", "conversation", "medicine" are recorded from video and audio records captured by the user's visual and auditory senses. It is an object of the present invention to provide a storage / reproduction device capable of efficiently reading and reproducing portions corresponding to four types of events of “ingestion” and “laughter”.

In order to achieve the above object, a storage / reproducing apparatus according to the first aspect of the present invention includes a sensor unit that detects video, audio, and the like captured by the user's vision and hearing, and converts the image and sound into an electrical signal, and the sensor unit. A storage unit that stores the electrical signal detected by the detection unit in a state associated with the detection time, an output unit that converts the electrical signal into video and audio, and reproduces it, and processes the electrical signal received from the sensor unit to the storage unit A processing unit that writes and reads a corresponding electric signal from the storage unit in a state associated with the detection time in accordance with a user instruction input from the operation unit, and sends the output image and sound to the output unit. parts are a camera, a microphone, a bone conduction sensor, detectably variations in epidermis of "temple" of a user, or a pressure sensor which is installed so as to abut on the "temple" Becomes, the processing section of the two received from the pressure sensor and the bone conduction sensor electrical signals of a user by combining the result of comparing respectively the first and second thresholds "options" and "drug An event determination unit that determines whether or not an event “ ingestion ” has occurred, identifies the time point when these events occur, a data search unit that reads an electrical signal corresponding to the event from the storage unit, and a time point when the event occurs The playback point time is set on the basis of the above, and the playback point is stored in the storage unit in time series for each event, and the electrical signal read from the storage unit is converted into video or audio at the output unit and detected time It is characterized by comprising control means for reproducing from the reproduction point according to the above.

When the user performs “mastication”, the mandibular movement moves and the temporal muscles contract, and the epidermis of “Komekami” rises. However, in the memory reproduction device of the present invention, the fluctuation of the “Komekami” epidermis is detected by the pressure sensor. Therefore, when the electrical signal detected by the pressure sensor exceeds the first threshold, the event determination means determines that the user is performing “mastication” and is detected by the bone conduction sensor. When the electrical signal exceeds the second threshold, it is determined that the user is “swallowing”. And when the event determination means determines that the user is performing “chewing” and “swallowing”, the event determination means determines that the user is “mealing” and performs only “swallowing”. If it is determined, the user is determined to be “taking medicine”. Then, the processing unit stores the playback point in the storage unit for each event in chronological order, for example, not at the time when those events occur, but at a time several seconds before the playback point.

In the storage / playback device configured as described above , the playback point is searched for a scene that includes “meal” or “drug intake” from the video or audio captured by the camera or microphone and stored in the storage unit. Therefore, the data retrieval unit can accurately read out desired data related to video and audio from the storage unit.
By the event determination means, the time when the user performs “meal” and “drug intake” is determined, and from the playback point set at that time, the video and audio stored in the storage unit are reproduced, It has an effect that the scene where the user performs “meal” or “drug intake” is reproduced efficiently.

According to a second aspect of the present invention, in the storage / reproducing apparatus according to the first aspect, the event determining means outputs the two types of electric signals received from the microphone and the bone conduction sensor to the third threshold and the fourth It is characterized by determining whether or not the user's “ laughter ” and “ conversation ” events have occurred by combining the results compared with the threshold values, and identifying when these events have occurred. .

When the user is “conversing” with the surrounding people, the voice data detected by the microphone is a mixture of the user's voice and the surrounding people's voice. It is difficult to determine whether or not the voice is included. On the other hand, if a bone conduction sensor is used, a user's voice can be detected in a state where it can be distinguished from the voices of surrounding people.
Therefore, in the storage / playback device of the present invention, when the electrical signal exceeding the third threshold is detected by the bone conduction sensor, the event determination means generates a voice in response to the voice detected by the microphone. It is determined that the user has started “conversation” at that time. Further, when the electrical signal detected by the bone conduction sensor has a peak value exceeding the fourth threshold, the event determination means determines that the user is “laughing” and when the peak value occurs. It is determined that the user has started “laughing”.

According to such a configuration , the playback point is a clue when searching for a scene containing “laughter” or “conversation” from the video or audio captured by the camera or microphone and stored in the storage unit. Therefore, the data search means has an effect that desired data related to video and audio is accurately read from the storage unit.

As described above, according to the storage / reproduction device of the first aspect of the present invention, the electrical signal read from the storage unit by the data search means is converted into video or audio by the output unit, and is shown as a reproduction point. It is possible to efficiently reproduce the scene where the user performs “ meal ” or “ intake of medicine ” by playing the video from the time when the video is played. In addition, for an event called "meal" and "intake of medicine" is keep the playback point several seconds before the time from the point in time that occurred, so that these events are reproduced from the scene immediately before the occurrence, "meal" And the situation where “ drug intake ” occurred.

According to the storage / reproducing apparatus of the second aspect of the present invention, in addition to the effects of the first aspect of the invention, the electrical signal read from the storage unit by the data search means By converting to sound and playing it from the time indicated as the playback point, it is possible to efficiently reproduce the scene where the user has “ laughed ” or “ conversed ”.

It is a perspective view which shows the external appearance of Example 1 of the storage / reproducing apparatus which concerns on embodiment of this invention. 1 is a block diagram illustrating a configuration of a storage / reproduction device according to Embodiment 1. FIG. (A) thru | or (e) are the graphs which show an example of the time waveform of the electric signal detected by the microphone and the bone conduction sensor in the memory | storage / reproducing apparatus of Example 1. FIG. 6 is a flowchart illustrating a determination procedure performed by an event determination unit in the storage / reproduction device according to the first embodiment. (A) is the figure which showed typically the state in which a reproduction | regeneration point is stored in a time table, (b) is a figure which supplements the flowchart shown in FIG. (A) And (b) is the image figure which showed the menu screen and scene selection screen which are each displayed on a display in the storage / reproducing apparatus of Example 1. FIG. (A) And (b) is the image figure which showed the conversation screen and time setting screen which are each displayed on a display in the storage / reproducing apparatus of Example 1. FIG. FIG. 8 is an image diagram illustrating a state in which a video selected on the screen of FIG. 7 is displayed on a display. (A) And (b) is a perspective view which shows the external appearance of the main-body part and the display which respectively comprise Example 2 of the storage / reproducing apparatus based on embodiment of this invention.

The structure of the recording / reproducing apparatus of the present invention will be specifically described with reference to FIGS.
The storage / playback apparatus of the present invention has almost the same shape as glasses, and can record video and audio captured by the user of the present invention (hereinafter simply referred to as the user) by visual and auditory senses. A camera and a microphone are installed on the bridge and arm of the frame, respectively. The video and audio captured by the camera and microphone are converted into electrical signals and stored in a storage unit built in the arm of the frame. The electrical signal read from the storage unit is converted into video or audio by the output unit and displayed on the display.
In the first embodiment, a mechanism for projecting an image read from the storage unit to a lens that also serves as a display will be described. In the second embodiment, the above image is displayed on a monitor display provided separately from the main body. The mechanism to be performed will be described.

FIG. 1 is a perspective view showing the appearance of the storage / reproduction device 1a of this embodiment, and FIG. 2 is a block diagram showing the configuration of the storage / reproduction device 1a.
As shown in FIG. 1, the recording / reproducing apparatus 1a is installed in the bridge 2a of the frame 2 with the camera 3 facing the lens 3a forward. In the arm 2b of the frame 2, the microphone 4 is embedded in the vicinity of the hinge 2c so as to open outward, and the speaker 5 is embedded in the vicinity of the ear hook 2d so as to open inward. Further, a bone conduction sensor 6 is installed at the inner tip of the ear hook 2d, and a support 2e is erected on the upper surface of the arm 2b. A pressure sensor 7 is attached to the tip of the support 2e. It is attached so that the fluctuation of the epidermis can be detected.

  Two lenses 8 and 8 are fitted in the frame 2, and a mirror (not shown) and a half mirror (not shown) are provided inside one lens 8, and an arm on the side close to this lens. A liquid crystal display (not shown) is installed inside 2b so that an image can be projected onto the mirror. Then, a mirror is disposed so as to reflect the image projected from the liquid crystal display toward the half mirror, and the image projected so as to pass through the inside of the lens 8 from the mirror is reflected toward the user. A half mirror is arranged as shown. That is, the mirror and the half mirror constitute the video display means 9 (see FIG. 2) together with the liquid crystal display.

The arm 2b includes a processing unit 16 (see FIG. 2) for processing various data detected by the camera 3, the microphone 4, the bone conduction sensor 6, and the pressure sensor 7, a battery (not shown), and a storage unit 15 (see FIG. 2) is built-in. One ear hook 2d is provided with a screen switching button 10, a selection button 11, and a determination button 12 for operating a menu on the screen displayed on the lens 8 by the video display means 9, and the other. The ear hook 2d is provided with a power switch 13 for switching the ON / OFF state of the processing unit 16 and a connection port 14 for connecting an external terminal such as a USB connector.
Note that the battery is charged by an external power source connected to the connection port 14 via a power cable, but such a structure is not necessarily required. For example, the battery can be structured so as to be detachable from the arm 2b. In this case, since the battery can be charged in a state where it is taken out from the arm 2b, workability is good. Furthermore, a commercially available battery can also be used as this battery.

As shown in FIG. 2, the processing unit 16 includes a storage unit 15, an operation unit 17 including a power switch 13, a screen switching button 10, a selection button 11, and a determination button 12, a camera 3, a microphone 4, and bone conduction. A sensor unit 18 composed of the sensor 6 and the pressure sensor 7 and an output unit 19 composed of the speaker 5 and the image display means 9 are connected.
Based on the information captured by the sensor unit 18, the processing unit 16 determines an event determination unit 16 a that determines the timing at which each of four types of events “conversation”, “laughter”, “meal”, and “drug intake” has occurred. And data search means 16b for reading out information corresponding to a predetermined event from the storage unit 15 based on an instruction input from the operation unit 17, and operations of the storage unit 15, the operation unit 17, the sensor unit 18, and the output unit 19. It is comprised by the control means 16c to control.
The information acquired by the sensor unit 18 is measured by the sensor unit 18 in addition to the information acquired from the sensor unit 18 in real time, temporarily stored in the storage unit 15 described later, and then acquired again by the processing unit 16. It is a concept that includes information.

  In the storage unit 15, video data 15 a captured by the camera 3, audio data 15 b and 15 c captured by the microphone 4 and the bone conduction sensor 6, and the time when the detection value 15 d of the pressure sensor 7 is detected, respectively. Stored in an associated state. In addition, the storage unit 15 stores the time 5 seconds before the occurrence of each event of “conversation”, “laughter”, “meal”, and “drug intake” as a playback point in time series 15e. Stored.

FIGS. 3A and 3B show electric signals E ₁ and E ₂ detected by the microphone 4 and the bone conduction sensor 6 when the user is “conversing” with the surrounding people, respectively. cage, FIG. 3 (c) shows an electrical signal E ₂ detected by the bone conduction sensor 6 when the user is "smiling".
3D and 3E show the electric signal E ₂ detected by the bone conduction sensor 6 when the user is “mealing” and “taking medicine”, respectively. Show. That is, in these drawings, the horizontal axis represents elapsed time, and the vertical axis represents voltage magnitude.
In addition, although the time waveform is not illustrated about the electric signal (detection value 15d) detected by the pressure sensor 7, it makes it easy to distinguish from the electric signals E ₁ and E ₂ detected by the microphone 4 and the bone conduction sensor 6. Therefore, in the present specification, the electrical signal detected by the pressure sensor 7, for convenience, is intended to refer to an electrical signal E _3.

When the user is “conversing” with the surrounding people, the user's voice is detected by the microphone 4 in a mixed state with the surrounding person's voice and the like, so the electric signal E ₁ is shown in FIG. As shown in a), it is not easy to determine whether or not the user's voice is included. On the other hand, the bone conduction sensor 6 detects the user's voice in a state where the voice of the user can be distinguished from the voices of surrounding people. Therefore, the event determination unit 16a, as shown in FIG. 3 (b), and determines the user when the electric signal E ₂ exceeds is a "conversation" a preset threshold B.
The threshold value A shown in FIGS. 3 (a) is used in determining the presence or absence of audio data by the event determination unit 16a in the electric signal E ₁ as described below.

When the user is “laughing”, a peak value appears continuously in a short time in the electric signal E ₂ detected by the bone conduction sensor 6. Therefore, the event determination unit 16a, as shown in FIG. 3 (c), the user when the peak value exceeds the threshold value C set in advance appeared in quick succession in the electric signal E ₂ is "laughing Is determined.
Also, if the user has a "diet", as shown in FIG. 3 (d), substantially uniform peak value showing the operation of "swallowing" the electrical signal E ₂ detected by bone conduction sensor 6 It appears periodically with a large size. Furthermore, as shown in FIG. 3 (e), the user peak value showing the operation of "swallowing" described above is likewise appears in the electric signal E ₂ even if you are "ingested drugs." However, as is clear when comparing FIG. 3 (d) and FIG. 3 (e), when “meal” is used, “chewing” is different between “swallowing” and “chewing”. A waveform indicating that “
The number of “chewing” during meals varies from person to person, but for the same person, the number of “chewing” until “swallowing” is almost constant, and the peak value indicating the action of “swallowing” Occurs periodically. On the other hand, the peak values appearing during “laughing” are not uniform, and the intervals are often non-uniform and shorter than the peak value intervals indicating the “swallowing” action. Therefore, as is apparent from a comparison between FIG. 3C and FIG. 3E, regarding the electric signal E ₂ detected by the bone conduction sensor 6, if attention is paid to the magnitude and period of the peak value, It is easy to discriminate which operation indicates “laughing” or “swallowing”.

When the user performs “mastication”, the lower jaw moves and the temporal muscles contract, and the epidermis of “Komekami” rises. However, as described above, the epidermis of “Komekami” in the memory reproduction device 1a of the present invention. the fluctuation of is detected by the pressure sensor 7, if the electric signal E ₃ exceeds a predetermined threshold value (not shown), the user can determine that perform "chewing". Therefore, the event determination unit 16a, as shown in FIG. 3 (d) and FIG. 3 (e), the ones the user performs an "swallowing" when a preset threshold value D are electric signals E ₂ exceeds In addition, when it is determined that the user has “masticated” based on the electric signal E ₃ , it is determined that the user is “mealing”. On the other hand, the event determination means 16a determines that the user is “taking medicine” when it is determined that the user does not “chew” but only “swallow”.

Next, based on the electric signals E _{1 to} E ₃ detected by the microphone 4, the bone conduction sensor 6, and the pressure sensor 7, four types of events “conversation”, “laughter”, “meal”, and “drug intake” 4 will be described with reference to FIG. 3 as needed, with reference to FIG.
FIG. 4 is a flowchart showing a determination procedure performed by the event determination means 16a. FIG. 5 (a) is a diagram schematically showing a state where playback points are stored in the time table, and FIG. It is a figure which supplements the flowchart shown in FIG.
As shown in FIG. 5 (a), the time table 15e can be stored in the form of time-series data with the time 5 seconds before the occurrence of the above-described four types of events as a playback point. That is, as will be described below, “meal” is i = 1 and “drug intake” with respect to the time table 15e expressed as Ts (i, j) (i = 1 to 4, j = 1 to N). I = 2, “laughter” is i = 3, and “conversation” is i = 4, and up to N playback points are stored.

As shown in FIG. 4, first, in step S1, Ts (i, j) is set to 0 for i from 1 to 4 and j from 1 to N, i and j are set to 1, and the process proceeds to step S2. In step S2, the electric signals E _{1 to} E ₃ detected by the microphone 4, the bone conduction sensor 6, and the pressure sensor 7 are taken into the processing unit 16 together with the detection time t.
Next, in step S3, the event determination unit 16a by the user based on the electric signal E ₂ it is determined to have been "swallowed", the flow proceeds to step S4. In step S4, the event determination unit 16a the user based on the electric signal E ₃ is determined to have done a "chewing", the user is determined to be a "meal" in step S5, an electric signal the value of Ts (1, j) to 5 seconds before the time when it is judged to "chewing" of the user is started in E ₃ as a reproduction point, the flow returns to step S2.
On the other hand, in step S4, if the event determination unit 16a is not determined that the user based on the electric signal E ₃ is performing "chewing", and the user at step S6 is "ingesting drugs" together, the value of Ts (2, j) 5 seconds before the time when it is judged to "swallow" the user is started in the electric signal E ₂ as a reproduction point, the flow returns to step S2.

In step S3, if the event determination unit 16a is not determined that the user based on the electric signal E ₂ is performing the "swallowing", the flow proceeds to step S7. In step S7, if the event determination means 16a it is determined as a peak exceeding the threshold C in a short time in an electric signal E ₂ are irregularly generated, with the user determines that "smiling" in Step S8 Then, the time 5 seconds before the time when the user determines that the user has started “laughing” in the electric signal E ₂ is set as the value of Ts (3, j) as the playback point, and the process returns to step S 2.
On the other hand, in step S7, the event determination unit 16a, when it is not determined that the peaks in excess in a short time in an electric signal E ₂ the threshold C are irregularly generated, the process proceeds to step S9. In step S9, the event determination unit 16a, when the electric signal E ₂ does not exceed the threshold value B, and the user silent, determined not to be the people around "conversation", the flow returns to step S2.

In step S9, if the electric signal _{E 2} exceeds the threshold value B, the process proceeds to step S10. In step S10, when the electric signal E ₁ does not exceed the threshold value A, the event determination unit 16a, since the sound around the person is not detected, if the user has not the surrounding people "Conversation" Determine and return to step S2.
On the other hand, in step S10, when the electric signal E ₁ exceeds the threshold value A, in step S11, with the event determination unit 16a it is determined that the user is a person around "conversation", an electric signal E ₁ , the time 5 seconds before it is determined that the “conversation” has started is set as the value of Ts (4, j), and the process returns to step S2.

In Steps S5, S6, S8, and S11, if a time indicating a playback point is already stored for Ts (i, j), Step S4 and Step S5 or The processing as shown in FIG. 5B is performed during step S6, between step S7 and step S8, and between step S10 and step S11.
As shown in FIG. 5B, when each determination is made by the event determination unit 16a in steps S4, S7, and S10, the process proceeds to step S20, in which case the value of TS (i, j) is greater than 0, that is, If the time indicating the playback point has already been stored for Ts (i, j), j is incremented by 1 in step S21, and the process returns to step S20. On the other hand, if the value of TS (i, j) is not greater than 0 in step S20, that is, if the time indicating the playback point is not yet stored for Ts (i, j), the process proceeds to step S22. move on. When j reaches N, the event determination unit 16a ends the event determination process. On the other hand, if j has not reached N, the process proceeds to one of step S5, step S6, step S8, and step S11.
In the above description, the time 5 seconds before the event occurrence time is used as the playback point, and the value of Ts (i, j) is used. However, this time of 5 seconds includes 0 seconds (simultaneously). It can be appropriately changed to a desired time.

In the present embodiment, the time table 15e is assumed as Ts (i, j) (i = 1 to 4, j = 1 to N), and up to N Ts ( i, j) is stored, but the method of acquiring Ts (i, j) is not limited to this method. For example, without executing step S1 of FIG. 4, information stored in the storage unit 15 in real time by the sensor unit 18, that is, the electric signals E ₁ , E ₂ , E ₃ , together with the detection time t by the processing unit 16. The event determination means 16a reads and determines which event “meal”, “drug intake”, “laughter”, or “conversation” has occurred based on the respective electric signals E ₁ , E ₂ , E ₃ . , And referring to the detection time t, the start time of the event is determined, and 5 seconds before is set as the value of Ts (i, j).
In this case, Ts (1,1), Ts (2,1), Ts (2,2), Ts (3,1), Ts (4,1) are not time-sequentially but in time series. , Ts (3, 2), etc. Ts are listed in time series, not by event. I representing the type of event and j representing the order of occurrence in the event are listed one-dimensionally. As a specific data format, for example, the entire data is represented by an 8-digit number, the first 6 digits represent time, the seventh digit represents an event, and the eighth digit is generated in each event. This is a case where the data represented by the 8-digit number is arranged in a form such as j representing the rank.
However, even in this case, j needs to be incremented by one. In order to obtain such a sequence, for example, in step S1, i = 0 and j = 0 of Ts (i, j) are set, and after Y and N in step S4, after Y in step S7, step S10 By inserting a step of j = j + 1 after steps S5, S6, S8, and S11 determined as “meal”, “drug intake”, “laughter”, and “conversation”. Is possible.
In this way, when j is determined as each event, it is incremented by one.
As described above, according to the present invention, the time of the playback point Ts can be set on the basis of the time when the event occurs, and the playback point can be stored in the storage unit 15 in time series so that the playback point can be identified for each event. The playback point may be determined in any way.

An operation method of the storage / reproduction device 1a will be described with reference to FIGS.
6 (a) and 6 (b) are image diagrams showing a menu screen and a scene selection screen displayed on the display in the storage / playback apparatus 1a, and FIGS. 7 (a) and 7 (b) are similarly displayed. It is the image figure which showed the conversation screen and time setting screen which are displayed on. FIG. 8 is an image diagram showing a state in which the video selected on the screen of FIG. 7 is displayed on the display. In addition, about the component shown in FIGS. 1-5, the same code | symbol is attached | subjected and the description is abbreviate | omitted.
When the power of the storage / reproduction device 1a is turned on by operating the power switch 13, a main menu as shown in FIG. 6A is displayed on the lens 8 by the video display means 9. In the main menu, the selection button 11 can be operated to select one of “short-term memory reproduction”, “scene selection”, and “time designation”. “Short-term memory playback” is a mode in which video and audio stored in the storage unit 15 are sequentially played back from a plurality of preset times such as 30 seconds ago and 1 minute ago.

When “select scene” is selected with the selection button 11 and the enter button 12 is pressed, a scene selection screen as shown in FIG. 6B is displayed on the lens 8. On this screen, one of “conversation”, “meal”, “drug intake”, and “laughter” can be selected. When the selection button 11 and the decision button 12 are operated to select, for example, “conversation”, a conversation screen as shown in FIG. 7A is displayed. When one of the playback points displayed on this screen is selected by operating the selection button 11 and the decision button 12, the scene where the user has “conversed” is selected on the conversation screen as shown in FIG. Play from the point.
On the other hand, when the “time designation” is selected by operating the selection button 11 and the decision button 12 in the main menu shown in FIG. 6A, a time designation scene as shown in FIG. 7B is obtained. On this screen, it is possible to confirm the time by operating the selection button 11 to change the time (set time) at which video or audio playback is started and pressing the enter button 12.
In these operations, when the screen switching button 10 is pressed, the main menu can be returned from any screen.

  As described above, according to the storage / reproduction device 1a of the present invention, the video data 15a and the audio data 15b read from the storage unit 15 by the data search unit 16b are converted into video and audio by the video display unit 9 and the speaker 5. By converting and reproducing from the time indicated as the reproduction point, it is possible to efficiently reproduce the scene where the user performed “laughter”, “conversation”, “meal”, and “drug intake”. At this time, since the playback point is set to the time 5 seconds before the time when those events occur, the playback starts from the scene immediately before each event occurs. Therefore, it is possible to accurately grasp the situation where each event has occurred.

The recording / reproducing apparatus 1b of Example 2 will be described with reference to FIG.
9 (a) and 9 (b) are perspective views showing the external appearance of the main body 20a and the monitor 20b constituting the storage / reproduction device 1b. In addition, about the component shown in FIGS. 1-4, the same code | symbol is attached | subjected and the description is abbreviate | omitted.
As shown in FIGS. 9A and 9B, the storage / reproduction device 1b of the present embodiment is provided separately from the main body 20a in place of the lens 8 in the storage / reproduction device 1a of the first embodiment. An image is displayed on the display 21 of the monitor 20b. That is, the speaker 5, the video display means 9, the screen switching button 10, the selection button 11, the determination button 12, the power switch 13, and the connection port 14 in the storage / playback apparatus 1a are separated from the frame 2 and collected on the monitor 20b. The display 21 and the power lamp 22 are newly provided.
In the storage / reproduction device 1b having such a structure, the video and audio stored in the storage unit 15 of the main body 20a can be easily displayed on the display 21 of the monitor 20b without the user wearing the main body 20a on the head. Can be confirmed.
The structure of the present invention is not limited to that shown in the above-described embodiment,
For example, an organic EL technology can be applied to the lens portion to display an image. Further, the storage / reproduction device of the present invention is not necessarily limited to the glasses type. For example, a structure in which a camera, a microphone, or the like is attached to a hat or a band wound around the head can be used.

  The invention described in claims 1 and 2 can be used as a memorandum as well as in the case of looking back on his / her actions for the purpose of recalling memories forgotten by the elderly.

  DESCRIPTION OF SYMBOLS 1a, 1b ... Memory | storage reproduction | regeneration apparatus 2 ... Frame 2a ... Bridge 2b ... Arm 2c ... Hinge 2d ... Ear hook part 2e ... Supporting part 3 ... Camera 3a ... Lens 4 ... Microphone 5 ... Speaker 6 ... Bone conduction sensor 7 ... Pressure sensor 8 DESCRIPTION OF SYMBOLS ... Lens 9 ... Image | video display means 10 ... Screen switching button 11 ... Selection button 12 ... Decision button 13 ... Power switch 14 ... Connection port 15 ... Memory | storage part 15a ... Video | video data 15b, 15c ... Audio | voice data 15d ... Detection value 15e ... Time table DESCRIPTION OF SYMBOLS 16 ... Processing part 16a ... Event determination means 16b ... Data search means 16c ... Control means 17 ... Operation part 18 ... Sensor part 19 ... Output part 20a ... Main-body part 20b ... Monitor 21 ... Display 22 ... Power lamp

Claims (2)

A sensor unit that detects video or audio captured by the user's visual or auditory sense and converts it into an electrical signal;
A storage unit for storing the electrical signal detected by the sensor unit in a state associated with a detection time;
An output unit for converting the electrical signal into the video or the audio and reproducing the same, and
A state in which the electrical signal received from the sensor unit is processed and written to the storage unit, and the corresponding electrical signal from the storage unit is associated with the detection time according to the user's instruction input from the operation unit And a processing unit that reads out and sends the video and the audio to the output unit,
The sensor unit is composed of a camera, a microphone, a bone conduction sensor, and a pressure sensor installed so as to be in contact with the “temple” so as to detect a change in the epidermis of the user's “temple” .
The processor is
The user's “ meal ” and “ drug intake ” are combined by combining the results of comparing the two types of the electrical signals received from the bone conduction sensor and the pressure sensor with the first threshold value and the second threshold value, respectively. An event determination means for determining whether or not an event has occurred, and identifying the time point when these events have occurred;
Data search means for reading out the electrical signal corresponding to the event from the storage unit;
The time of the playback point is set on the basis of the time when the event occurs, the playback point is stored in the storage unit in time series for each event, and the electrical signal read from the storage unit is stored in the output unit. A storage / reproducing apparatus comprising control means for converting to video or audio and reproducing from a reproduction point according to a detection time. The event determination means combines the results of comparing the two types of the electrical signals received from the microphone and the bone conduction sensor with a third threshold value and a fourth threshold value, respectively , thereby making the user “ laugh ” and 2. The storage / reproducing apparatus according to claim 1, wherein it is determined whether or not an event of “ conversation ” has occurred, and a point in time at which these events have occurred is specified.

Images