KR101998662B1 - Apparatus and storage medium for mutually complementary motion recognition - Google Patents

Abstract

In the present invention, in recognizing whether a predetermined operation has occurred based on a sensing value of one or more sensors, it is possible to share motion information through interaction with other apparatuses and reduce misrecognition rate using the shared motion information A method for recognizing a complementary motion and an apparatus therefor, the method comprising: collecting sensed sensed values from at least one sensor that senses a physical change at regular intervals; calculating motion information from the sensed sensed values; The method of claim 1, further comprising the steps of: recognizing an operation in which one or more unit operations are consecutively performed by comparing information with a predetermined operation recognition condition, The motion information of the time zone corresponding to the supplementary interval By requesting the device to using the motion information of the other apparatus that by determining whether the normal recognition of the complementary region, more improving the reliability of the motion recognition.

Description

[0001] Apparatus and recording medium for motion recognition [0002]

The present invention relates to an apparatus and method for recognizing an operation of a user, and more particularly, to an apparatus and method for recognizing an operation of a user, To an apparatus and a recording medium for motion recognition.

In recent years, with the introduction of an open OS, a smart phone combining a high function of a PC with a mobile phone has become popular, and various attempts have been made to utilize a high-function and high-performance smart phone.

In particular, with the development of micro-fabrication technology, advanced sensors become smaller and cheaper, and more sensors can be mounted on smart phones, and intelligent applications utilizing such sensors, such as augmented reality and 3D games, are being developed .

In addition, sensors mounted on smart phones will be able to play a key role in communicating emotions with human beings by evolving into an intelligent sensor that takes into consideration user's physical changes and emotional states in a device that senses the surrounding environment. As a result, intelligent applications utilizing sensors are expected to grow even more.

There are camera (image) sensor, acoustic sensor, proximity sensor, illuminance sensor, gravity sensor, GPS sensor, acceleration sensor, gyro sensor, and geomagnetic sensor.

Among them, a camera (image) sensor is a sensor that detects light and converts the intensity of the light into digital image data, which can be used for face recognition and the like. The proximity sensor is not a detection system based on mechanical contact but a proximity sensor that detects the presence or absence of an object to be detected when the detection object comes close to the proximity sensor Detection sensor that can be used to automatically turn off the screen when the smartphone is brought close to the face or placed in a pocket for a normal call.

The illuminance sensor is a sensor for sensing the ambient brightness. Normally, the illuminance sensor is used to reduce the power consumption of the mobile terminal and to reduce the fatigue of the eyes. It is a sensor that detects the direction of gravity and detects movement of objects. It is used to automatically correct the direction of the screen by determining the display direction (horizontal and vertical) of the smartphone.

In addition, the GPS sensor is a sensor capable of collecting time and position information of an object through a satellite positioning system, and is utilized for various location-based services. The acceleration sensor detects changes in object speed per unit time, In recent years, three-axis acceleration sensors using MEMS technology have become popular, and even object movements such as tilt changes and shakes can be detected. A geomagnetic sensor is a sensor that detects the azimuth angle like a compass by grasping the flow of the earth's magnetic field, The gyro sensor detects the inertial force of an object as an electric signal and mainly detects the rotation angle. It can detect the height, the rotation, and the tilt directly, so that it can recognize more precise motion when connected with the 3-axis acceleration sensor. Do.

Among these various sensors, motion recognition using a proximity sensor or an illuminance sensor has a small recognition error due to a simple recognition method, but the types of recognizable operations are limited, and an operation using an acceleration sensor, a geomagnetic sensor, a gyro sensor Recognition can recognize more diverse actions than proximity sensors or illuminance sensors, but recognizes complex actions in which the recognition rate is lowered without explicitly specifying a range of values, and unit movements of different directions, speeds, and times are connected it's difficult.

However, as the application fields of the recent motion recognition technology are gradually expanded, it is required to expand the motion recognition range from simple motion detection to motion of a specific pattern shape.

Korean Patent Laid-Open No. 10-2010-0081552, published on July 19, 2010 (name: device and method for detecting motion of portable terminal)

In the present invention, in recognizing whether a predetermined operation has occurred based on a sensing value sensed by one or more sensors, motion information is shared through interaction with another device, and the erroneous recognition rate is reduced using shared motion information And to provide a device and a recording medium for mutually complementary motion recognition.

The present invention provides, as a solution to the above-mentioned problems, a sensing value collection module for collecting sensed sensed values from at least one sensor that senses a physical change; A calculation module for calculating motion information from the collected sensing values; An event transmission / reception module for transmitting / receiving data to / from another device; An interworking control module for transmitting the motion information to another device through the event transmission / reception module or for receiving motion information detected by the other device; And an operation of recognizing the motion by comparing motion information calculated by the calculation module with motion recognition conditions of a predetermined motion, and correcting the motion recognition result using motion information or motion recognition information detected by the other device And a recognition module for recognizing the motion of the robot.

In the apparatus for complementary motion recognition according to the present invention, the calculation module may store the motion information in a buffer for a predetermined time period in units of preset time slots.

In addition, in the apparatus for recognizing complementary motion according to the present invention, the motion recognition module may determine a possibility of misrecognition in units of time slots during motion recognition, detect a complementary interval in units of time slots, The interworking control module generates and transmits a sharing request message for requesting motion information for a time zone corresponding to the supplementary interval.

In addition, in the apparatus for recognizing complementary motion according to the present invention, the motion recognition module may determine whether or not the supplementary section is normally recognized by using motion information detected by the other apparatus at a time zone corresponding to the supplementary section It can be judged.

In addition, in the apparatus for mutually complementary motion recognition according to the present invention, the interworking control module, when receiving a sharing request message from the other device through the event sending / receiving module, The motion information of the requested time slot can be extracted and transmitted to the other device.

In addition, the apparatus for complementary motion recognition according to the present invention may further include an interlock device management module for managing status information of another device including at least one of a remaining battery level, a data transmission frequency, a movement pattern, A battery management module for checking the remaining battery power of the device; And a controller for managing system resources of the apparatus and extracting state information including at least one of a battery remaining amount, a data transmission frequency, a movement pattern, and a wear state and transmitting the extracted state information to the other apparatus, And a schedule management module for managing the schedule management module.

In addition, the present invention provides, as another solution to the above-mentioned problem, a method for detecting a physical change, comprising: collecting, at regular intervals, sensing values sensed by at least one sensor for sensing a physical change; Calculating motion information from the collected sensing values; Comparing the calculated motion information with a predetermined motion recognition condition, and recognizing an operation in which one or more unit operations are consecutively performed; Detecting a possibility of erroneous recognition by determining a possibility of erroneous recognition in units of a predetermined time slot while the operation recognition is performed; Requesting another device for motion information in a time zone corresponding to the supplementary interval and receiving the motion information; And determining whether the supplementary section is normally recognized by using motion information of the other apparatus.

The motion recognition apparatus according to the present invention is a device for recognizing a motion of a device mounted on a predetermined device and recognizing a motion of the device so as to share motion information sensed by another motion recognition device mounted on another device close to the device And more accurate motion recognition can be performed using the motion information.

In particular, the motion recognition apparatus according to the present invention collects sensing values at predetermined intervals, detects motion information, compares the detected motion information with predetermined motion recognition conditions, The method includes detecting a complementary interval in which a certain time slot can be erroneously recognized or uncertain in each time slot unit and a sensing value or motion information of the same time period as the detected complementary interval is requested and received from another device, It is possible to acquire necessary information while improving the accuracy of motion recognition.

FIG. 1 is a schematic diagram showing an example of application of a complementary motion recognition apparatus according to the present invention.
2 is a view for explaining a reference coordinate system applied for recognition of an operation in the first and second devices shown in FIG.
3 is a block diagram illustrating a configuration of a device having a complementary motion recognition apparatus according to the present invention.
4 is a block diagram illustrating the structure of a complementary motion recognition apparatus according to the present invention.
5 and 6 are flowcharts illustrating a complementary motion recognition method according to the present invention.
7 and 8 are views for explaining a sensing value sharing process according to the complementary motion recognition method according to the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description and the accompanying drawings, detailed description of well-known functions or constructions that may obscure the subject matter of the present invention will be omitted. It should be noted that the same constituent elements are denoted by the same reference numerals as possible throughout the drawings.

The terms and words used in the present specification and claims should not be construed to be limited to ordinary or dictionary meanings and the inventor is not limited to the concept of terminology for describing his or her invention in the best way. It should be interpreted as meaning and concept consistent with the technical idea of the present invention. Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention, and not all of the technical ideas of the present invention are described. Therefore, It is to be understood that equivalents and modifications are possible.

Also, terms including ordinal numbers such as first, second, etc. are used to describe various elements, and are used only for the purpose of distinguishing one element from another, Not used. For example, without departing from the scope of the present invention, the second component may be referred to as a first component, and similarly, the first component may also be referred to as a second component.

In addition, when referring to an element as being "connected" or "connected" to another element, it means that it can be connected or connected logically or physically. In other words, it is to be understood that although an element may be directly connected or connected to another element, there may be other elements in between, or indirectly connected or connected.

Also, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. It is also to be understood that the terms such as " comprising "or" having ", as used herein, are intended to specify the presence of stated features, integers, It should be understood that the foregoing does not preclude the presence or addition of other features, numbers, steps, operations, elements, parts, or combinations thereof.

The complementary motion recognition apparatus according to the present invention can be applied to two devices (10, 20) located within a certain distance and having one or more sensors as shown in FIG. The complementary motion recognizing apparatus according to the present invention is provided in each of the two devices 10 and 20 to perform motion recognition using sensing values of one or more sensors whose sensing values are changed according to motion of the device, The motion information of one device is shared with each other, and the motion recognition result is supplemented by using the shared motion information. In the following description, the main device for performing the motion recognition is referred to as a first device 10, and an auxiliary device for providing a sensing value to be used for supplementing the motion recognition result in the first device 10 is referred to as a second device 20 ), But the roles of the first and second devices can be changed.

For reference, it is preferable that the first device 10 and the second device 20 to be used in the present invention are held by the same user and are close to each other within a predetermined distance. The first and second devices 10 and 20 may be any one of a smart phone, a PDA, a wearable device in the form of a watch or a glasses, and a tablet PC.

For example, as shown in FIG. 1, the first device 100 may be a smartphone, the second device 200 may be a smart watch, and vice versa.

The first and second devices 10 and 20 are equipped with an operation recognizing function for recognizing a predefined specific movement (flipping, shaking in a specific direction, drawing a specific pattern, user access) The motion sensed by the sensors 10 and 20 can be represented by an azimuth, a pitch, and a roll defined on a three-dimensional rectangular coordinate system.

2, when the horizontal direction of the first and second devices 10 and 20 is the X-axis, the vertical direction is the Y-axis, and the width direction is the Z-axis, (Or -180 ° to 180 °) with respect to the Z-axis in accordance with the direction (east, west, south, north) of the first and second substrates 10 and 20, 90 degrees to 90 degrees according to the form in which the first and second devices 10 and 20 are erected and the roll is a rotation angle in the vertical axis direction and the first and second devices 10 and 20 are rotated in the range of -180 ° to 180 ° °. Of course, in the motion recognition, the reference coordinate system and the method of expressing the motion can be changed, and the above definition is only an example.

As described above, when the first and second devices 10 and 20 provided with the motion recognition device according to the present invention are worn or carried by the user, the first and second devices 10 and 20, May sense one or more sensed values corresponding to the motion of the user, and perform motion recognition based on the sensed values or sensed values. The present invention is based on the fact that one or more sensed values collected by the first device 10 and the second device 20 correspond to the same motion, 2 device 20 to determine whether the motion recognition result is uncertain or whether there is a possibility of erroneous recognition, thereby improving the reliability of the motion recognition result by reducing the error recognition rate. Here, the motion information shared between the first and second devices 10 and 20 may include at least one of a rotation angle, an angular velocity, and a movement pattern (+, -) detected by each device.

FIG. 3 is a block diagram illustrating the configuration of first and second devices 10 and 20 provided with a complementary motion recognition apparatus according to the present invention.

The first and second devices 10 and 20 are devices used by a user and may include various functions and configurations depending on the purpose. Basically, the first and second devices 10 and 20 include input units 11 and 21, Output units 12 and 22, communication units 13 and 23, storage units 14 and 24, sensors 15 and 25, and controllers 16 and 26. Of course, the input units 11 and 21 and the output units 12 and 22 may not be provided in some cases.

The input units 11 and 21 are means for inputting user commands and data to the first and second devices 10 and 20 and generate various input data including numbers, And transmits a command related to function setting and function control of the first and second devices 10 and 20 to the control units 16 and 26 according to a user operation. One or more buttons, a keypad, a touch pad, a mouse, and the like.

The output units 12 and 22 are means for outputting information generated according to the operations of the first and second devices 10 and 20 in a form recognizable by the user. The output units 12 and 22 may include at least one of a display device for outputting visual information, an audio device (speaker) for outputting auditory information, and a haptic device for outputting tactile information. In addition, the display device may be a touch screen. If a display device is formed in the form of a touch screen, the display device may perform some or all of the functions of the input devices 11 and 21 . An audio device may represent a speaker typically.

The communication units 13 and 23 are configured to transmit and receive data to and from other devices, and can transmit and receive data according to one or more communication methods among various communication methods as well as wire and wireless methods. To this end, the communication units 13 and 23 may include at least one network connection unit and a network access control module, respectively, which are configured to transmit and receive data according to different communication methods. Here, the network connection means a hardware configuration for network connection, and the network connection control module indicates a software configuration for performing data transmission / reception processing according to a set communication method. More preferably, the communication units 12 and 23 exchange data with nearby devices within a predetermined distance, and can transmit and receive data using a local communication method such as Wi-Fi, Bluetooth, NFC, or the like.

In particular, the first and second devices 10 and 20 having the complementary motion recognition apparatus according to the present invention recognize the partner device through the communication units 13 and 23, .

The storage units 14 and 24 are configured to store programs and data necessary for operation of the first and second devices 10 and 20. The storage units 14 and 24 basically store the OS (Operating System) program of the terminal device 10 and are also required to execute one or more applications operating based on the OS environment, The generated data can be stored. In particular, in the present invention, the storage units 14 and 24 may store motion information shared in a complementary motion recognition process.

The storage units 14 and 24 may be any of various types of storage media such as a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, a CD-ROM (Compact Disc Read Only Memory), an optical recording medium such as a DVD Media, a magneto-optical medium such as a floppy disk, and the like.

The sensor units 15 and 25 include at least one sensor for sensing a surrounding environment or a physical change of the first and second devices 10 and 20 and sense and output at least one sensed value for recognizing the operation . For example, the sensor units 15 and 25 include an acceleration sensor for detecting a change in an object speed per unit time, a change in a dynamic force such as an impact, a geomagnetic sensor for detecting an azimuth angle like a compass, And a gyro sensor that detects the rotation angle, and may include various kinds of sensors. Although an acceleration sensor, a geomagnetic sensor, and a gyro sensor are exemplified as representative sensors used for motion recognition, the present invention is not limited thereto. The sensor units 15 and 25 may further include a camera (image) sensor for detecting light and converting the intensity of the light into digital image data, an acoustic sensor for converting a physical sound into an electrical signal Sensor, a proximity sensor for determining the presence or absence of an object to be detected when the detection object comes close to the detection object, an illumination sensor for detecting the ambient brightness, detecting the direction in which gravity acts, A GPS sensor capable of collecting time and position information of an object through a satellite positioning system, and the sensing value sensed by the various sensors can be used for motion recognition.

The control units 16 and 26 are configured to control and process the overall operations of the first and second devices 10 and 20. The control units 16 and 26 include one or more processors and a RAM And a memory such as a memory. And performs predetermined functions and operations by executing one or more program modules loaded into the system memory with the processor.

The program modules executed by the control unit 16 and 26 according to the present invention may be prepared in any form of a programming language including a compiled or interpreted language or a proactive or procedural language, , Subroutines, or any other unit suitable for use in a computer environment. The program module does not necessarily correspond to a file in the file system, but may be stored in a single file provided to the requested program or in a file that stores multiple interactive files (e.g., one or more modules, (E.g., one or more scripts stored in a markup language document) within a file or other program or data file that holds other programs or data. A computer program may be deployed to run on multiple computers or on one computer, located on a single site or distributed across multiple sites and interconnected by a communications network. The processor and system memory of the controller 16, 26 may be supplemented or integrated into a special purpose logic circuit.

The complementary motion recognizing device according to the present invention is implemented in the form of the program module described above and is mounted on the first and second devices 10 and 20 and then executed by a processor of the control unit 16 or 26, Collects one or more sensing values from the sensor unit 15, calculates motion information from the collected one or more sensed values, compares the calculated motion information with preset motion definition information, And also acquires motion information sensed by the motion recognition device of another device at a time point corresponding to the supplementary interval through the communication units 13 and 23, It is possible to compensate the motion recognition result.

FIG. 4 is a block diagram illustrating a configuration of a complementary motion recognition apparatus according to the present invention, which may be provided in the first and second devices 10 and 20. Hereinafter, the operation recognition apparatus mounted on the first device 10 will be referred to as the first operation recognition apparatus 100, the operation recognition apparatus mounted on the second device 20 will be referred to as the second operation recognition apparatus 200 But this is merely for convenience of explanation and is not intended to limit the invention.

Referring to FIG. 4, the first and second motion recognition devices 100 and 200 include sensing value acquisition modules 110 and 210, operation modules 120 and 220, motion recognition modules 130 and 230, (140, 240), and an interlocking control module (150, 250).

In addition, the first motion recognition device 100 mounted on the first device 10 as a main device may further include an interlocking device management module 160, and may be mounted on the second device 20 as an assistant device The second motion recognition device 200 may further include a schedule management module 270 and a battery management module 280.

The sensing value acquisition modules 110 and 210 collect one or more sensed values from the sensor units 15 and 25. At this time, the sensed values to be collected may have different units and sizes depending on the sensor. For example, the sensing value of the illuminance sensor indicates the amount of illumination (Lux), the sensing value of the proximity sensor indicates the distance to and / or proximity to a nearby object, and the sensing value of the 3-axis acceleration sensor is 3 Axis, the sensing value of the three-axis gyro sensor indicates the angular velocity in the direction of the three axes (X, Y, Z), and the sensing value of the geomagnetism sensor indicates the acceleration value in the direction of the axis Direction. The sensing value collection module 100 collects sensing values from a plurality of predetermined sensors 10 or collects sensed values from at least one sensor selected from the plurality of sensors 10 under the control of the recognition module 300 . At this time, the sensing value collection modules 110 and 210 may collect the sensing value at a predetermined time period, that is, a predetermined sampling period.

The calculation modules 120 and 220 calculate motion information recognizable by the recognition module 300 from one or more sensing values collected by the sensing value acquisition modules 110 and 220. More specifically, the calculation modules 120 and 220 calculate motion information such as the azimuth angle, the pitch angle, the roll angle, and the like from the sensed values obtained through the rotation vector and rotation matrix calculations, The angular velocity, and the movement pattern. The calculation modules 120 and 220 may convert the sensed value of the illuminance sensor into the illuminance measurement value or the illuminance change amount or may calculate the proximity number and the proximity distance based on the sensing value of the proximity sensor. In addition, the operation modules 120 and 220 according to the present invention include buffers 121 and 221 of a predetermined size, and store motion information or sensing values calculated within a predetermined time range for a predetermined time. For example, the buffers 121 and 221 may store motion information within a predetermined time interval based on a current time point. In addition, when the motion information and the sensing value are stored in the buffer, the motion information and the sensing information and the corresponding time information may be stored together.

The motion recognition modules 130 and 230 are configured to recognize a predefined motion using the motion information calculated from the calculation modules 120 and 220. In particular, the motion recognition module 130, 230 compares motion information calculated from the calculation modules 120, 220 with a predetermined motion recognition condition, and when the motion information satisfies a specific motion recognition condition, It is determined that an operation matching the recognition condition is recognized.

Here, the operations recognized by the operation recognition modules 130 and 230 may be one or more unit operations. Here, the unit operation is an operation in which the previously defined operation is subdivided based on at least one of direction, time, and speed, and the operation recognition module 130, 230 sequentially recognizes the one or more unit operations, It is possible to judge that the corresponding operation is recognized when all of the unit operations included in the operation are sequentially recognized in the predetermined order.

In addition, the motion recognition condition may be set for each one or more unit operations, and may include at least one of a rotation angle, a rotation speed, and a movement time, for example.

In addition, the operation recognition modules 130 and 230 according to the present invention determine whether there is a possibility of misidentification during the operation recognition described above. The determination may be performed in units of predetermined time slots, for example, timeslots of a period longer than the sampling period of the sensing value collection module 110. [

That is, when the sensing value is collected from the sensing value collection module 110 at a predetermined sampling period, the calculation module 120 calculates motion information from the sensed values collected at each period according to the sampling period.

Then, the motion recognition modules 130 and 230 compare motion information continuously calculated at a constant sampling period with preset motion recognition conditions, and recognize motion corresponding to a continuous motion change.

The motion recognition modules 130 and 230 divide the consecutive processes in which consecutive motion recognition is performed by a predetermined unit of time t to extract a complementary interval with a high possibility of false recognition. More specifically, when the operation recognition module 130 or 230 recognizes only a part of operations performed by one or more unit operations, or when an operation in which some unit operations do not match occurs, the operation recognition module 130 and 230 can extract the corresponding section as a supplementary section.

The motion recognition modules 130 and 230 acquire motion information of other devices for the supplementary interval through the interlocking control modules 150 and 250.

Then, based on the acquired motion information of the other apparatus, the supplementary section is re-determined to extract whether it is normal recognition. More specifically, the motion recognition module 130 and 230 can determine whether the value or pattern change of the motion information of another device corresponding to the supplementary interval is made at the same time. Here, if the same motion change is detected in another apparatus, it can be determined as normal recognition, and if the same or similar motion change is not performed in another apparatus, it can be determined that the recognition is not normal.

For this purpose, time synchronization between the first and second motion recognition devices 100 and 200 may be performed.

The event transmission / reception modules 140 and 240 and the interlocking control module 150 are configured to perform interoperation between the first and second motion recognition devices 100 and 200. The event transmission / The first and second devices 10 and 20 transmit and receive data or messages related to the operation recognition to other devices through the communication units 13 and 23. The interlocking control modules 150 and 250 are interlocked with other devices . Specifically, the interworking control modules 150 and 250 generate a sharing request message for the supplementary interval according to a request from the operation recognition modules 130 and 230, and transmit the sharing request message to the event transmission / reception modules 140 and 240 And receives a sharing request message for a specific supplementary interval through the event transmission / reception module 140, 240, the motion information corresponding to the same time interval as the requested supplemental interval of the motion information stored in the buffer 121, And transmits the processed data through the event transmission / reception module 140 or 240. [0033] FIG.

In addition, the interworking control module 150 or 250 may associate sensing values collected by the sensing value collection modules 110 and 210 or motion recognition information recognized by the motion recognition modules 130 and 230 according to a preset schedule, And the like.

The data shared through the event transmission / reception module 140 and 240 includes a first field for recording a subject transmitting data, a second field for recording a data type, a third field for recording a sensing value, 5 fields, and detailed information for each field can be defined as shown in Table 1 below.

field Name Length Type Detail One Host 1 Byte Byte 0: Device
1 to 254: a wearable device
255: Error 2 Type 1 Byte Byte 0: Device information 1: Motion recognition information
2: Buffer data information
3: Accelerometer
4: Geomagnetic sensor
5: Gyro sensor
6 ~ 254: Other sensors
255: Error 3 Data1 4 Byte Float In case of device information, in case of remaining battery (1 ~ 100%) sensor, x axis measurement value
In the case of motion recognition information, the x-
For buffer data information, timestamp
Error code 4 Data2 4 Byte Float In case of device information, data transmission frequency (1 ~ 100 msec) In case of sensor, measurement value of y axis
In the case of motion recognition information, the y-
In the case of buffer data information, the sensor axis (x, y, z)
NULL if Error 5 Data3 4 Byte Float In the case of the device information, in case of the worn state sensor, the z axis measurement value
In the case of motion recognition information,
In the case of buffer data information, the sensor axis measurement value
NULL if Error

In Table 1, the motion recognition information indicates a motion pattern as a result of motion recognition recognized by the motion recognition modules 130 and 230, and the sensor values indicate motion patterns of the specific sensors collected by the sensing value collection modules 110 and 210 And is transmitted at the time of information sharing between the first and second devices 10 and 20 as a sensing value and the buffer data information is motion information calculated at a specific time zone by the calculation module 220 to perform complementary motion recognition The first and second devices 10 and 20 according to the present invention can share information for mutually complementary operation recognition through the data format as shown in Table 1 above.

In addition, the interlocking device management module 160 is a unit for managing the state of the device (the remaining battery level, the data transfer frequency, the movement pattern, the wearing state, etc.) provided with other interlocking devices, And stores it.

The interlocking device management module 160 is included in the first motion recognition device 100 mounted on the first device 10 as a main device but may be included in the second motion recognition device 200 as necessary.

The schedule management module 270 is a component included in the second motion recognition device 200 mounted on the second device 20 serving as the assistant device. The first motion recognition device 100 includes status information The remaining capacity of the battery, the frequency of data transmission, the movement pattern, the wearing state, etc.) to other devices, and manages system resources and schedules corresponding thereto.

The battery management module 280 is also included in the second motion recognition device 200 mounted on the second device 20 serving as the assistant device so that the second control device Checks the battery remaining amount of the device 20, and provides it to the schedule management module 270.

That is, the schedule management module 270 sets the data transmission frequency according to the battery remaining amount confirmed by the battery management module 280, and transmits the set data transmission frequency to the first device 10 as the main device To the first motion recognition device 100 mounted thereon.

Next, a mutually complementary operation recognition process performed through interlocking between the first motion recognition device 100 and the second motion recognition device 200 as described above will be described with reference to the flowcharts of FIGS. 5 and 6. FIG.

5 is a flowchart illustrating a complementary motion recognition method performed by the first motion recognition apparatus 100 according to the present invention.

For reference, the first device 10 having the first motion recognition device 100 performs a search or connection process with the second device 20 in the vicinity through the communication unit 13, (20). The above process can be performed, for example, according to the Paring procedure of the Bluetooth communication standard. Since this connection process is performed according to the procedure set in the local area communication method, a detailed description thereof will be omitted.

In this state, the first motion recognition apparatus 100 according to the present invention recognizes the second motion recognition apparatus 200 of the second device 20 connected through the above-described paring procedure, and recognizes the recognized second motion recognition apparatus 200 Status information for operation recognition interlocking is received from the device 200 (S110). The state information is information including at least one of a battery remaining amount, a data transmission frequency, a movement pattern, and a wear state, and is information necessary to determine movement pattern change based on data sharing and shared data for motion recognition.

In addition, the first motion recognition apparatus 100 may collect at least one sensed value from the sensor unit 15 provided in the first device 10 at regular intervals, calculate motion information from the sensed sensed value, The recognition module 130 compares the calculated motion information with the motion recognition condition for the predetermined motion, and performs motion recognition processing for recognizing the predetermined motion (S120, S130). The motion information generated at a predetermined period in the motion recognition process is stored in the buffer 121 together with time information (for example, a time stamp) corresponding to the corresponding value. The steps S130 and S130 may be performed simultaneously with the step S110, or may be performed after the step S110.

In addition, the first motion recognition apparatus 100 confirms whether there is a possibility of misidentification during the operation recognition through the above-described steps S120 to S130 (S140). This compares the motion information calculated at the predetermined period with a predetermined motion recognition condition and recognizes that there is a possibility of misrecognition if a part of the motion consisting of one or more consecutive unit motions does not match the motion recognition condition or if comparison is impossible . In addition, in step S140, the first motion recognition apparatus 100 detects a complementary interval in which there is a possibility of misrecognition or uncertainty of motion recognition among the motion information at the predetermined period (S150).

FIG. 7 and FIG. 8 are diagrams illustrating a process of detecting a complementary interval in the mutually complementary motion recognition method according to the present invention. Referring to FIG. 7, the first motion recognition apparatus 100 collects a sensing value obtained by sensing motion of the first device 10 and divides a time T for calculating motion information into a plurality of time slots (t0 to tn). The predetermined time unit may be equal to or larger than a sampling period at which the sensing value is collected, for example. Then, it is determined whether or not there is a possibility of erroneous recognition by analyzing the calculated motion information and motion recognition result for each time slot (t0 to tn). For example, in the embodiments of FIGS. 7 and 8, it is assumed that the motion information in the time slot t2 does not match the predetermined motion recognition condition, or it is impossible to judge it out of the comparable range. It is possible to extract the time slot t2 having the possibility of erroneous detection as a supplementary section.

Then, the first motion recognition apparatus 100 according to the present invention requests the recognized second motion recognition apparatus 200 for a sensing value for the time slot t2 determined as the complementary interval and receives the sensed value ). If the motion recognition module 130 determines that the request is a supplementary interval for the time slot t2, the interlocking control module 150 may include information (e.g., a time stamp) for the time slot t2 And the event sending and receiving module 140 transmits the generated sharing request message to the second motion recognition device 200 of the second device 20 through the communication unit 13 of the first device 10, Lt; / RTI >

Meanwhile, when the second motion recognition device 200 receives the motion information sensed at the time corresponding to the time slot t2, the first motion recognition device 100 recognizes the motion recognition module 130, In step S170, whether the recognition result of the supplementary section is normal recognition based on the received motion information. Specifically, the motion recognition module 130 determines the motion information value or the change pattern received from the second motion recognition device 200 based on the motion information detected in the same time slot in the first motion recognition device 100 If the difference does not deviate from the predetermined range, it is determined that the recognition is normal. If the difference is not within the predetermined range, it is determined that the recognition is abnormal.

For example, referring to the embodiment of FIG. 7, the first motion recognition apparatus 100 of the first device 10 may compare the motion information detected in the time slot t2 determined as the supplementary section, The second motion recognizing device 200 of the second camera 20 can compare the sensed value or the motion information detected in the same time slot to determine whether or not it is normal.

8, the first motion recognition apparatus 100 of the first device 10 may recognize that the time slot t2 is a complementary period, (Tk _+? ) (? Is a natural number equal to or greater than 0) in the predetermined range of the forward and backward directions based on the time slot ( _tk , for example, k = 2) The motion information may be received and the first device 10 may compare the sensed value or the motion information detected at the same time, respectively, to determine whether or not the motion information is normally recognized.

The first motion recognition device 100 outputs the motion recognition result in consideration of the determination result based on the motion information received from the second motion recognition device 200 of the second device 10 S180).

6 is a flowchart showing an operation process performed by the second motion recognition device 200 of the second device 20 linked to the operation of the first motion recognition device 100. As shown in FIG.

The second device 20 having the second operation recognition device 200 performs the search or connection process with the first device 10 which is located close to the second device 20 via the communication unit 23, The device 10 is recognized (S210).

In this state, the second motion recognition device 200 according to the present invention is configured such that the first device 10 can share motion information of the first device 10, Extracts state information including information including at least one of a battery remaining amount, a data transmission frequency, a movement pattern, and a wear state required for recognizing complementary motions by analyzing system resources and schedules of the mobile station 20, Information to the first motion recognition apparatus 100 of the first device 10 (S220).

The second motion recognition apparatus 200 may collect one or more sensing values from the sensor unit 25 provided in the second device 20 at regular intervals and calculate motion information from the collected sensing values (S230). In addition, the second motion recognition apparatus 200 compares the calculated motion information with the motion recognition condition for the predetermined motion through the motion recognition module 230, and performs motion recognition processing for recognizing the preset motion You may. The motion information generated in a predetermined period in the above process is stored in the buffer 221 together with time information (e.g., a time stamp) corresponding to the corresponding value.

In addition, the second motion recognition device 200 determines whether a sharing request message is received from the first motion recognition device 100 of the first device 10 (S240). At this time, the sharing request message includes information corresponding to a time slot detected as a complementary interval in the first motion recognition device 100. [

When the second motion recognition device 200 receives the sharing request message from the first motion recognition device 100, the second motion recognition device 200 acquires the motion information collected or calculated at the time corresponding to the requested time slot from the buffer 221 And transmits it to the first motion recognition device 100 (S250).

The complementary motion recognition method according to the present invention may be implemented in a form of software readable by various computer means and recorded in a computer-readable recording medium. Here, the recording medium may include program commands, data files, data structures, and the like, alone or in combination. Program instructions to be recorded on a recording medium may be those specially designed and constructed for the present invention or may be available to those skilled in the art of computer software. For example, the recording medium may be an optical recording medium such as a magnetic medium such as a hard disk, a floppy disk and a magnetic tape, a compact disk read only memory (CD-ROM), a digital video disk (DVD) Includes a hardware device that is specially configured to store and execute program instructions such as a magneto-optical medium such as a floppy disk and a ROM, a random access memory (RAM), a flash memory, do. Examples of program instructions may include machine language code such as those generated by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like. Such a hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

Further, the functional operations and subject matter implementations described herein may be implemented in other types of digital electronic circuitry, or may be implemented in computer software, firmware, or hardware, including the structures disclosed herein and their structural equivalents, ≪ / RTI > Implementations of the subject matter described herein may be embodied in one or more computer program products, i. E. One for computer program instructions encoded on a program storage medium of the type for < RTI ID = 0.0 & And can be implemented as a module as described above. The computer-readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter that affects the machine readable propagation type signal, or a combination of one or more of the foregoing.

In addition, while the specification contains a number of specific implementation details, it should be understood that they are not to be construed as limitations on the scope of any invention or claim, but rather on the features that may be specific to a particular embodiment of a particular invention Should be understood as an explanation. Certain features described herein in the context of separate embodiments may be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Further, although the features may operate in a particular combination and may be initially described as so claimed, one or more features from the claimed combination may in some cases be excluded from the combination, Or a variant of a subcombination.

Likewise, although the operations are depicted in the drawings in a particular order, it should be understood that such operations must be performed in that particular order or sequential order shown to achieve the desired result, or that all illustrated operations should be performed. In certain cases, multitasking and parallel processing may be advantageous. Also, the separation of the various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems will generally be integrated together into a single software product or packaged into multiple software products It should be understood.

Certain embodiments of the subject matter described herein have been described. Other embodiments are within the scope of the following claims. For example, the operations recited in the claims may be performed in a different order and still achieve desirable results. By way of example, the process illustrated in the accompanying drawings does not necessarily require that particular illustrated or sequential order to obtain the desired results. In certain implementations, multitasking and parallel processing may be advantageous.

The description sets forth the best mode of the invention, and is provided to illustrate the invention and to enable those skilled in the art to make and use the invention. The written description is not intended to limit the invention to the specific terminology presented. Thus, while the present invention has been described in detail with reference to the above examples, those skilled in the art will be able to make adaptations, modifications, and variations on these examples without departing from the scope of the present invention.

Therefore, the scope of the present invention should not be limited by the described embodiments but should be defined by the claims.

The motion recognition apparatus according to the present invention is a device for recognizing a motion of a device mounted on a predetermined device and recognizing a motion of the device by sharing motion information with an motion recognition device of another device close to the device, So that more accurate motion recognition can be performed.

In particular, the motion recognition apparatus according to the present invention collects sensing values at predetermined intervals, detects motion information, compares the detected motion information with predetermined motion recognition conditions, The method includes detecting a complementary interval in which a certain time slot can be erroneously recognized or uncertain in each time slot unit and a sensing value or motion information of the same time period as the detected complementary interval is requested and received from another device, It is possible to acquire necessary information while improving the accuracy of motion recognition.

10: first device 20: second device
100: first motion recognition device 200: second motion recognition device
110, 210: Sensing value collection module 120, 220: Operation module
121, 221: buffer 130, 230: motion recognition module
140, 240: an event transmission / reception module 150, 250: an interlocking control module
160: Interlocking device management module 270: Schedule management module
280: Battery management module

Claims (9)

A sensing value collection module for collecting sensed sensed values from at least one sensor that senses a physical change;
A calculation module for calculating motion information from the collected sensing values;
An event transmission / reception module for transmitting / receiving data to / from another device;
An interworking control module for transmitting the motion information or the sensing value to another device through the event transmission / reception module or for receiving motion information or sensing value detected by the other device; And
And a motion estimation unit for estimating motion information or sensing information detected by the other device in addition to the calculated motion information if it is determined that there is a possibility of erroneous recognition by recognizing a predefined motion based on the motion information calculated by the calculation module, And a motion recognition module for recognizing a predefined motion by using a motion vector and a motion vector together. 2. The apparatus of claim 1,
Storing the motion information in a buffer for a predetermined time slot in a predetermined time slot unit,
Wherein the motion recognition module detects a possibility of erroneous recognition in units of the time slots and detects a complementary interval in units of time slots and requests the other device for motion information or a sensing value corresponding to the supplementary interval, Lt; / RTI > The method according to claim 1,
The interworking control module determines whether motion information or a sensing value of the other device is necessary by comparing the motion information calculated in the calculation module with a predefined motion recognition condition and outputs the motion information or the sensing value Wherein the controller is configured to: The apparatus according to claim 2,
Determining whether the supplemental section is normally recognized by using the motion information detected by the other apparatus at a time zone corresponding to the supplemental section,
The motion information calculated by the calculation module is compared with a value or a conversion pattern of motion information of the other apparatus. If the difference does not deviate from a predetermined range, it is determined as normal recognition. If the difference is out of a predetermined range, Wherein the device is a device for recognizing motion. 3. The system of claim 2, wherein the interworking control module
Receiving a sharing request message from the other device through the event transmission / reception module, extracting motion information of a time slot requested in a sharing request message among the motion information stored in the buffer, and transmitting the motion information to the other device Lt; / RTI > The method according to claim 1,
Further comprising an interlock device management module for managing status information of another device including at least one of a remaining battery level, a data transmission frequency, a movement pattern, and a wear status. The method according to claim 1,
A battery management module for checking the remaining battery power of the device; And
The system resource of the apparatus is managed to extract state information including at least one of the battery remaining amount, the data transmission frequency, the movement pattern, and the wear state, and transmits the extracted state information to the other apparatus, And a schedule management module for managing the schedule management module. Collecting sensed sensed values from at least one sensor that senses a physical change;
Calculating motion information from the collected sensing values;
Transmitting the calculated motion information or sensing value to another device, or receiving motion information or a sensing value detected by another device; And
If it is determined that there is a possibility of erroneous recognition by judging the possibility of misunderstanding by using the calculated motion information, motion information or sensed value detected by the other device, together with the calculated motion information, Recognizing a predefined motion using the motion;
And a recording medium storing the program. 9. The method of claim 8,
And the calculated motion information is stored in the buffer for a predetermined period of time.

Images