JP3505040B2 - Portable information processing device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a portable information processing device such as a portable computer or an electronic notebook.

[0002]

2. Description of the Related Art A portable computer having various functions has been developed. For example, as shown in Japanese Patent Laid-Open No. 3-55655, an image input device is provided in addition to a key input device to provide a document. A device for inputting information such as a person's face and the situation of a site as image data, and transmitting the input image data using a telephone line, and JP-A-6-301446. It has the function of receiving GPS signals,
A device has been developed that identifies the position of a person who is carrying it by using position data such as latitude and longitude based on a signal.

[0003]

As described above, the conventional portable computer can input the image of the face of the other party, the situation of the scene, etc., and the position of the person carrying it. It was not possible to input and identify the behavior of the person carrying the device.

The present invention has been made in order to improve such a point, and provides a portable information processing apparatus capable of inputting and specifying the action itself of a person carrying it to obtain various detailed information. The purpose is to obtain.

[0005]

A portable information processing apparatus according to the present invention has an exercise state detecting section, a pattern storing section, an action recognizing section, a recording section and an output section, and the exercise state detecting section is portable. Detects changes in vertical movement, horizontal movement, and rotational movement of the person who performs the action, and the pattern storage unit stores behavior model patterns in accordance with various changes in the vertical movement, horizontal movement, and rotational movement of the person who carries it, and The recognition unit changes the movement state detected by the movement state detection unit and the behavior model pattern stored in the pattern storage unit.
The action pattern of the person who carries it is specified from the screen, the recording unit sequentially stores the action pattern specified by the action recognition unit, and the output unit stores it in the recording unit when the output of the action result is designated. It is characterized by outputting the action identification result.

A second portable information processing apparatus according to the present invention has a motion state detecting section, a position detecting section, a position change calculating section, a pattern storing section, an action recognizing section, a recording section and an output section.
The movement state detection unit detects changes in vertical movement, horizontal movement, and rotation movement of the carrying person, the position detection unit detects the position of the carrying person, and the position change calculation unit uses the position signal from the position detection unit to carry the movement. The pattern storage unit stores behavior model patterns according to various changes of the person's vertical movements, horizontal movements, and rotational movements, and the action recognition unit detects movement states. Change of the movement state detected by the section and the presence or absence of the position change of the carrying person detected by the position change calculating section, and the behavior pattern of the carrying person is specified from the behavior model pattern stored in the pattern storage section, The recording unit sequentially stores the action patterns specified by the action recognition unit, and the output unit outputs the action identification result stored in the recording unit when the output of the action result is designated.

The motion state detecting section may include an acceleration sensor for detecting accelerations in three orthogonal axes and a gyro for detecting an angular velocity around the axes.

Further, it is desirable that the action pattern identification of the action recognition unit is executed by the selection of the person carrying the action pattern.
Whether or not the action pattern identification of the action recognition unit is performed may be selected by the voice input means.

Further, it has an image input means for inputting an image of the surroundings of the person carrying it, calculates the moving direction of the person carrying from the change of the inputted image, and classifies and stores it in the pattern storage section. It is desirable to select the corresponding behavior model pattern group from the behavior model patterns.

Further, there is a distance detecting means for detecting the distance to the object inputted by the image inputting means, and the behavior model pattern stored in the pattern storing section according to the detected moving distance.
It is good to select a group of groups.

[0011]

BEST MODE FOR CARRYING OUT THE INVENTION In the present invention, a portable information processing apparatus has a motion state detecting section, a pattern storing section, an action recognizing section, a recording section, an output section and a display section. The motion state detection unit detects changes in vertical movements when a person carrying the device stands or sits, horizontal movements when walking, and changes in rotational motions.
An acceleration sensor that detects the acceleration of the axis and a gyro that detects the angular velocity around each axis orthogonal to each other. The pattern storage unit stores in advance a behavior model pattern according to various changes in the behavior of the person carrying it.

Then, when the person carrying the cell phone starts to act, the acceleration sensor of the motion state detecting section and the gyro sequentially input a signal indicating the action of the person carrying the cell to the activity recognizing section. The action recognition unit compares the combination of input signals with various action model patterns stored in the pattern storage unit, identifies the action pattern of the person who carries it, and records the identified action pattern. Store in the department. The input of the action state of the person carrying the cellphone and the identification of the action pattern are repeated until the person carrying the cellphone completes a predetermined action, and the output of the action content is instructed in order to confirm the action up to that point. Then, the output unit displays the action identification result stored in the recording unit on the display unit and also outputs the result to the host device. In this way, since the contents of various actions of the person carrying the mobile phone are specified and stored at predetermined time intervals, the person carrying the mobile phone can confirm the state of the activity accurately later.

Further, by specifying the action pattern by the voice instruction of the person carrying the action pattern, the required action information can be accurately obtained.

Further, the position sensor detects the position of the person carrying the cellphone, and if it is determined whether the person carrying the cellphone is walking or stopped, various behavior model patterns stored in the pattern storage unit are stored. It is possible to specify the action pattern without searching all of the above, and it is possible to reduce the search processing amount of the pattern storage unit when the action pattern is specified.

Further, the image around the person carrying the image is inputted by the image inputting means, the moving direction of the person carrying the image is calculated from the change of the inputted image, and the distance to the object inputted by the image inputting means is calculated. The behavior model pattern detected by the distance detecting means, classified from the movement direction and movement distance, and the output signal from the movement state detecting section at that time into the pattern storing section and corresponding to the behavior model pattern stored in the pattern storing section. By selecting a group of actions, it is possible to obtain an action model pattern that matches the individuality of the person who actually carries the portable information processing device, and it is possible to more accurately recognize the action.

[0016]

1 is a block diagram showing the configuration of an embodiment of the present invention. As shown in the figure, the portable information processing device 1
Is a motion state detection unit 2, a pattern storage unit 3, and an action recognition unit 4
It has a recording unit 5, an output unit 6 and a display unit 7. The motion state detection unit 2 detects changes in vertical motion when the person carrying the device stands or sits, horizontal motion when walking, and changes in rotary motion. An acceleration sensor 21 for detecting acceleration and a gyro 22 for detecting angular velocity around each axis orthogonal to each other are included. The pattern storage unit 3 stores in advance action model patterns according to various changes in the actions of the person carrying the device.
This behavior model pattern is, for example, as shown in the pattern diagrams of FIGS. 2, 3 and 4, the X-axis obtained from the acceleration sensor 21 for each of various actions of a person who carries it, such as standing, sitting or walking. Symbols indicating the action of various combinations of waveform signals of acceleration in the direction of the Y-axis and Z-axis and waveform signals corresponding to the waveform signals of the angular velocity around the X-axis, around the Y-axis, and around the Z-axis obtained from the gyro 22. Action A, Action B,
It is formed with action C and the like. For example, the behavior model patterns 31a and 31 of the behaviors A and B shown in FIGS.
b shows a state in which the person carrying the mobile phone is walking, and behavior model pattern 31c of behavior C shown in FIG. 4 shows a state in which the person carrying the mobile phone stands and sits down. Show. The action recognition unit 4 compares the combination of the waveform signals output from the acceleration sensor 21 and the gyro 22 of the motion state detection unit 2 with various action model patterns 31 stored in the pattern storage unit 3 and compares them with those of a person who carries them. Identify behavior patterns. The recording unit 5 sequentially stores the action patterns specified by the action recognition unit 4. The output unit 6 displays the action identification result stored in the recording unit 5 on the display unit 7 when the output of the action content is designated, and outputs it to the external host device.

The operation of inputting and processing the behavior of the person who is carrying the portable information processing apparatus 1 having the above-described configuration will be described with reference to the flowchart of FIG. When the person carrying the cell phone starts to take action, the acceleration sensor 21 and the gyro 22 of the motion state detection unit 2 indicate the behavior of the person carrying the cell phone, such as vertical vibration generated when walking, acceleration in the forward direction, and the like. The state is detected, and the waveform signals of the acceleration in the X-axis direction, the Y-axis direction, and the Z-axis direction and the waveform signals of the angular velocity around the X-axis, the Y-axis, and the Z-axis are sequentially input to the action recognition unit 4 (step S1). ). The action recognition unit 4 is a waveform signal of acceleration in the X-axis direction, the Y-axis direction, and the Z-axis direction, which is input from the motion state detection unit 2 at a predetermined constant time after the person who carries the device starts the action. And a combination of waveform signals of angular velocities around the X-axis, the Y-axis, and the Z-axis, and various behavior model patterns 31 stored in the pattern storage unit 3 are compared, and the waveform output from the motion state detection unit 2 The behavior pattern pattern 31 that matches the signal combination is searched, and the behavior pattern such as walking, sitting, having a meeting, eating, etc. of the person carrying is specified, and the specified behavior pattern is set every hour. It is stored in the recording unit 5. The input of the behavior state of the person carrying the cellphone and the identification of the behavior pattern are repeated at regular intervals until the person carrying the cellphone completes a predetermined behavior (step S
4). When the action of the person who is carrying the device is completed and the output of the action content is instructed to confirm the action up to that point (steps S4 and S5), the output unit 6 displays the action identification result stored in the recording unit 5. It is displayed on the section 7 (step S6). Further, when the action specifying result is output, when the output unit 6 is connected to the external host device, the output unit 6 also outputs the action specifying result stored in the recording unit 5 to the host device.

As described above, since the contents of various actions of the person who carries the mobile phone are specified and stored at predetermined time intervals, the person who carries the mobile phone can accurately confirm the state of the activity later. In addition, by outputting the specified contents of various behaviors to the host device, it is possible to easily manage and use the situation of the carried behavior.

In the above embodiment, the case where the person carrying the mobile phone outputs the action specifying result stored in the recording unit 5 when the predetermined action is completed has been described. At this time, the action identification result may be output to the display unit 7 or the host device.

In the above embodiment, the action pattern is specified every predetermined time, but the time when the action pattern is specified is selected according to the instruction of the person carrying the device. May be.

Further, in the above embodiment, when the action pattern is specified by the action recognition unit 4, the waveform signals of the acceleration in the X-axis direction, the Y-axis direction and the Z-axis direction which are input from the motion state detection unit 2. And the case where the action pattern is specified by comparing the combination of the waveform signals of the angular velocities around the X-axis, the Y-axis, and the Z-axis with all of the various action model patterns 31 stored in the pattern storage unit 3. As described above, if it is determined whether the person who is carrying the robot has changed its position and whether the person is walking or stopped, the behavior model patterns 31 stored in the pattern storage unit 3 are stored. The action pattern can be specified without searching all.

In FIG. 6, the time when the action pattern is specified is selected according to the instruction of the person carrying it, and the pattern storage unit 3 is used.
FIG. 9 is a block diagram showing the configuration of another embodiment of the present invention in which an action pattern is specified without searching all of the various action model patterns 31 stored in FIG. As shown in FIG. 6, the portable information processing device 1 includes a motion state detection unit 2 and a pattern.
In addition to the storage unit 3, the action recognition unit 4, the recording unit 5, the output unit 6, and the display unit 7, a position detection sensor 8, a position change calculation unit 9, a voice input unit 10, a switch 11, a voice recognition unit 12, and a CCD sensor. 13, a moving direction calculating unit 14, a distance sensor 15, a moving distance calculating unit 16, and a pattern selecting unit 17.

The position detecting sensor 8 is composed of a GPS receiver, and specifies the position of a person who is carrying it by using position data such as latitude and longitude based on GPS signals. The position change calculation unit 9 detects whether or not the position of the person who is carrying the device changes based on the position signal from the position detection sensor 8. The voice input unit 10 is composed of a small microphone and inputs the voice of a person who carries the voice. The person who carries the voice recognition unit 12 has the switch 1
When 1 is turned on, the voice input from the voice input unit 10 is recognized and a voice recognition signal is output. CCD sensor 1
3 inputs the image around the person who carries it. The moving direction calculation unit 14 detects the moving direction of the person who is carrying the device based on the change in the image input from the CCD sensor 13. The distance sensor 15 is composed of, for example, an ultrasonic sensor, and detects the distance to the object displayed in the image input by the CCD sensor 13. The moving distance calculation unit 16 calculates the moving distance of the person who carries the mobile phone from the change in the distance input from the distance sensor 15. The pattern selection unit 17 roughly classifies and stores the actions in the pattern storage unit 3 based on the moving direction and the moving distance of the person who carries the mobile device, which are output from the moving direction calculating unit 14 and the moving distance calculating unit 16. From the model patterns 31, a corresponding behavior model pattern group is selected.

The operation of the portable information processing apparatus 1 configured as described above will be described with reference to the flowchart of FIG.
When the person carrying the mobile phone starts to take action, the motion state detection unit 2
The acceleration sensor 21 and the gyro 22 are waveform signals of acceleration in the X-axis direction, Y-axis direction, and Z-axis direction, and waveform signals of angular velocity around the X-axis, around the Y-axis, and around the Z-axis, which indicate the behavior of the person carrying the cell Are sequentially input to the action recognition unit 4 (step S1
1). Further, the position detection sensor 8 detects the position of the person who is carrying it, and the presence / absence of a position change is input from the position change calculation unit 9 to the action recognition unit 4 (step S12). At the same time, CC
The D sensor 13 inputs an image of the surroundings of the person who carries it,
The distance sensor 15 detects the distance to the object displayed in the image input by the CCD sensor 13. The movement direction calculation unit 14 detects the movement direction of the person who carries it by the change in the image input from the CCD sensor 13 and sends it to the pattern selection unit 17, and the movement distance is detected from the change in the distance input from the distance sensor 15. The calculation unit 16 calculates the movement distance of the person who carries it and sends it to the pattern correction unit 17 (step S1).
3). When the voice recognition signal is not output from the voice recognition unit 12, the pattern selection unit 17 determines the pattern storage unit based on the moving direction and the moving distance of the person who is carrying the input, such as up, down, front and back, and left and right. A corresponding action model pattern group is selected from the action model patterns 31 roughly classified into 3 and stored (steps S14 and S15). Since the behavior model pattern group is selected according to the actual behavior of the person who carries the portable information processing apparatus, the behavior model pattern that matches the exercise state of the person who actually carries the portable information processing device 1 is surely obtained. It can be obtained, and the behavior can be recognized more accurately.

When it becomes necessary for the person carrying the device in this state to specify the action pattern, the switch 11 is turned on and the voice "recognition" is input from the voice input unit 10 (step S14). The voice input by the voice input unit 10 is recognized by the voice recognition unit 12, the voice recognition unit 12 holds the recognized voice recognition signal, and continuously recognizes the voice recognition signal while the switch 11 is on. Send to Part 4. When the voice recognition signal is sent, the action recognition unit 4 starts the action recognition process of the person who carries it (step S16). When the action recognition process is started, the action recognition unit 4 carries the signal from the position change calculation unit 9 based on the signal indicating the presence / absence of the position change of the carrying person, as shown in the flowchart of FIG. Recognizes whether the walking person is walking or stopped (step S2
1) When the person carrying the device is walking, waveform signals of acceleration in the X-axis direction, the Y-axis direction, and the Z-axis direction, which are input from the motion state detection unit 2, and the X-axis rotation, the Y-axis rotation, and Z-axis. The combination of the waveform signals of the angular velocity around the axis and the behavior model pattern 31 during walking stored in the pattern storage unit 3, for example, as shown in FIGS. Behavior pattern pattern 3 that matches the combination of the generated waveform signals
1 is specified to specify the action pattern of the person who carries it (step S22). Further, when the person carrying the mobile phone is stopped, waveform signals of acceleration in the X-axis direction, the Y-axis direction, and the Z-axis direction input from the motion state detection unit 2 and the X-axis rotation, the Y-axis rotation, and the Z-axis rotation are input. The motion state detection unit 2 compares the combination of the waveform signals of the angular velocities with the pattern stored in the pattern storage unit 3 with the behavior model pattern 31 in a stopped state as shown in FIG.
The behavior pattern pattern 31 that matches the combination of the waveform signals output from is searched to specify the behavior pattern of the person who carries it (step S23). And the specified action pattern
Stored in the recording unit 5. In this way, it is determined whether the person who carries the device is walking or stopped, and the action model pattern 31 stored in the pattern storage unit 3 is searched based on the determination result, so that the action pattern is specified. It is possible to reduce the amount of search processing of the pattern storage unit 3 when performing.

The identification of the action pattern of the carrying person is repeated until the carrying person completes a predetermined action (step S17). Then, when the predetermined action of the person who carries the mobile phone is finished and the output of the action content is instructed to confirm the action up to that point (steps S17 and S18), the output unit 6 specifies the action stored in the recording unit 5. The result is displayed on the display unit 7, and when it is connected to the external host device 18, both the host device 18 and the action identification result are output to the host device. In this way, it is possible to accurately confirm the action content required by the person who carries it.

In the above embodiment, the case where the specified action pattern is stored in the recording unit 5 has been described, but the position detected by the position detection sensor 8 and the moving direction detected by the moving direction calculation unit 14 are described. Alternatively, the movement distance calculated by the movement distance calculation unit 16 may be recorded in the recording unit 5 together with the action pattern. By storing the action pattern and the position at that time together with the action pattern, more detailed action information can be obtained.

Further, in the above embodiment, the corresponding behavior model pattern group is selected from the behavior model patterns 31 which are roughly classified and stored in the pattern storage unit 3 when no voice signal is input. As described above, when a new person starts using the portable information processing device 1, a behavior model pattern group 31 is selected from the behavior model patterns 31 in the initial processing according to the motor movement of the person who has started using it. It may be done.

[0029]

As described above, according to the present invention, when the person carrying the mobile phone starts the action, a signal indicating the action of the mobile person is sequentially input from the motion state detecting section to the action recognizing section to perform the action. The recognition unit compares the combination of input signals with various behavior model patterns stored in advance, identifies the behavior pattern of the person who carries it, and sequentially stores the identified behavior pattern in the recording unit, When the output of the action content is instructed in order to confirm the action up to that point, the action identification result stored in the recording unit is output to the display unit or the host device, so that the person who carries it can change the action state later. It is possible to check accurately and easily manage and use the status of various actions.

In addition, the behavior of the person carrying the cell is orthogonal to each other 3
Since the gyro that detects the angular velocity around the three axes orthogonal to the acceleration sensor that detects the acceleration of the axis is used, it is possible to accurately detect changes in the vertical movement, horizontal movement, and rotational movement of the person carrying it. The behavior model pattern can also be accurately formed.

Further, by specifying the action pattern by the voice instruction of the person carrying the action pattern, it is possible to accurately obtain information on the required action.

Furthermore, the position of the person carrying the device is detected,
If it is determined whether the person carrying the device is walking or stopped, the action pattern can be specified without searching all of the various action model patterns stored in the memory.
It is possible to reduce the pattern search processing amount when specifying a pattern.

Further, by inputting an image of the surroundings of the person who carries the mobile phone, the moving direction of the mobile phone is calculated from the change of the input image, or the distance to the object input as an image is detected to detect the moving direction. When the corresponding behavior model pattern group is selected from the behavior model patterns stored in the pattern storage unit according to the output signals from the movement distance and the motion state detection unit at that time, the actual mobile type pattern group is selected. A behavior model pattern that matches the exercise state of a person carrying an information processing device
It is possible to obtain behavior and to recognize behavior more accurately.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.

FIG. 2 is a pattern diagram showing a configuration of a behavior model pattern.

FIG. 3 is a pattern showing a configuration of a second behavior model pattern.
Fig.

FIG. 4 is a pattern showing a configuration of a third action model pattern.
Fig.

FIG. 5 is a flowchart showing the operation of the above embodiment.

FIG. 6 is a block diagram showing a configuration of a second exemplary embodiment.

FIG. 7 is a flowchart showing the operation of the second embodiment.

FIG. 8 is a flowchart showing the action pattern specifying operation of the second embodiment.

[Explanation of symbols]

1 Portable information processing device 2 Motion status detector 3 pattern storage 4 Action recognition section 5 recording section 6 Output section 7 Display 8 Position detection sensor 9 Position change calculator 10 Voice input section 11 switch 12 Speech recognition unit 13 CCD sensor 14 Moving direction calculator 15 distance sensor 16 Moving distance calculator 17 Pattern selection section 18 Host device 21 Accelerometer 22 Gyro

─────────────────────────────────────────────────── ─── Continuation of the front page (58) Fields surveyed (Int.Cl. ⁷ , DB name) A61B 5/11 G01B 21/00 G01C 19/00

Claims (7)

(57) [Claims] 1. A motion state detection unit, a pattern storage unit, an action recognition unit, a recording unit, and an output unit, and the motion state detection unit detects changes in vertical movement, horizontal movement, and rotational movement of a person who carries it. However, the pattern storage unit stores behavior model patterns corresponding to various changes in vertical movements, horizontal movements, and rotational movements of a person who carries it, and the action recognition unit recognizes the movement state changes detected by the movement state detection unit. The action pattern of the person who carries it is specified from the action model pattern stored in the pattern storage unit, the recording unit sequentially stores the action pattern specified by the action recognition unit, and the output unit outputs the action content. A portable information processing device, which outputs the action identification result stored in the recording unit when is designated. 2. A motion state detecting unit, a position detecting unit, a position change calculating unit, a pattern storing unit, an action recognizing unit, a recording unit and an output unit, wherein the motion state detecting unit is a vertical movement of a person who carries it. The position detection unit detects the position of the person who carries it, and the position change calculation unit detects the presence or absence of the position change of the person who carries it from the position signal from the position detection unit. The storage unit stores behavior model patterns corresponding to various changes in the person's vertical movement, horizontal movement, and rotational movement, and the action recognition unit calculates changes in movement state and position changes detected by the movement state detection unit. The presence / absence of the position change of the carrying person detected by the department and the behavior pattern of the carrying person are specified from the behavior model patterns stored in the pattern storage unit, and the recording unit identifies the behavior pattern specified by the behavior recognition unit. Memory is stored sequentially, and the output section indicates the output of the action content. Portable information processing apparatus and outputting the behavior identification result stored in the recording unit when it is. 3. The portable information processing device according to claim 1, wherein the motion state detection unit includes an acceleration sensor that detects accelerations of three orthogonal axes and a gyro that detects an angular velocity around the axes. 4. The portable information processing apparatus according to claim 3, wherein the action pattern identification of the action recognition unit is executed by selection of a person carrying the action pattern. 5. The portable information processing apparatus according to claim 4, wherein the voice input means selects whether or not to execute the action pattern identification of the action recognition unit. 6. An image input means for inputting an image of the surroundings of the person carrying the portable person, wherein the moving direction of the person carrying the portable person is calculated from the change of the input image, and is classified and stored in the pattern storage unit. 4. The portable information processing apparatus according to claim 3, wherein a corresponding behavior model pattern group is selected from the behavior model patterns. 7. An action model pattern group stored in a pattern storage unit is selected according to the detected moving distance, the distance detecting means detecting the distance to the object input by the image input means. The portable information processing device described.