JP7449838B2 - Behavior analysis system and behavior analysis method using it - Google Patents

Description

The present invention relates to a behavior analysis system for analyzing the behavior of a user wearing a wearable terminal and a behavior analysis method using the same, and particularly relates to a technique for aligning the axis direction before analysis.

In order to measure the behavior of a user wearing a wearable terminal and analyze the behavior based on the measured data, it is necessary to align the axial directions of sensors such as acceleration sensors included in the wearable terminal in advance.

Patent Document 1 describes a method based on measurement data measured by each sensor when a user performs a rotational movement such as a practice swing with the sensors attached to the user's hands and arms and the equipment used by the user, such as a golf club. Accordingly, it is disclosed that the axial directions of each sensor are aligned.

Patent No. 6168279

However, in Patent Document 1, no consideration is given to aligning the axial directions of the sensors using measurement data measured at different timings. In other words, since Patent Document 1 is based on measurement data measured at the same time, for example, the axial direction of the sensor at the time of development and the time of use are determined based on the measurement data at the time of development and the measurement data at the time of use of the wearable terminal. It is difficult to match the axial direction of the sensor.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide a behavior analysis system and a behavior analysis method using the system that can align the axial directions of a plurality of sensors even if the measurement data is measured at different timings.

In order to achieve the above object, the present invention provides a behavior analysis system that analyzes user behavior, and includes a setting unit that sets a reference interval for first measurement data measured in a plurality of axial directions by a first sensor. an extraction unit that extracts a corresponding section corresponding to the reference section from second measurement data measured in a plurality of axial directions by a second sensor at a timing different from the first measurement data; a generation unit that generates an axial direction conversion model that is a model for converting the axial direction based on the first measurement data and the second measurement data of the corresponding section; The sensor is characterized by comprising a converter that converts the axial direction to the axial direction of the first sensor.

The present invention also provides a behavior analysis method using a behavior analysis system for analyzing user behavior, in which a setting step of setting a reference interval for first measurement data measured in a plurality of axial directions by a first sensor. an extraction step of extracting a corresponding section corresponding to the reference section from second measurement data measured in a plurality of axial directions by a second sensor at a timing different from the first measurement data; a generation step of generating an axial direction conversion model, which is a model for converting the axial direction, based on the first measurement data and the second measurement data of the corresponding section; The method is characterized by comprising a conversion step of converting the axial direction to the axial direction of the first sensor.

According to the present invention, it is possible to provide a behavior analysis system and a behavior analysis method using the same that can align the axial directions of a plurality of sensors even if measurement data is measured at different timings.

1 is a diagram illustrating an example of the overall configuration of a behavior analysis system according to a first embodiment. It is a figure showing an example of the data structure of sensor data for development. FIG. 3 is a diagram showing an example of a data structure of sensor data used by a user. FIG. 3 is a diagram showing an example of a data structure of an axial transformation model. It is a figure which shows an example of the data structure of sensor data after provision. 3 is a diagram showing an example of the flow of processing for generating development sensor data in Example 1. FIG. It is a figure showing an example of the data structure of a sensor table. FIG. 3 is a diagram showing an example of a data structure of a user table. It is a figure which shows an example of the data structure of a mounting part table. It is a figure which shows an example of the data structure of a mounting direction table. 3 is a diagram showing an example of the flow of processing for generating an axial transformation model in Example 1. FIG. FIG. 3 is a diagram showing an example of the flow of processing for converting the axial direction in the first embodiment. 3 is a diagram showing an example of a normal screen displayed in the first embodiment. FIG. 5 is a diagram showing an example of a warning screen displayed in Example 1. FIG. 2 is a diagram illustrating an example of the overall configuration of a behavior analysis system according to a second embodiment. FIG. FIG. 7 is a diagram illustrating an example of the flow of processing for behavior analysis in Example 2; 7 is a diagram showing an example of a screen displayed in Example 2. FIG.

Preferred embodiments of the behavior analysis system according to the present invention will be described below with reference to the accompanying drawings. In the following description and accompanying drawings, components having the same functional configuration are designated by the same reference numerals, thereby omitting redundant explanation.

The overall configuration of the behavior analysis system of Example 1 will be explained using FIG. 1. The behavior analysis system includes a sensor terminal 1 used by the user or a sensor terminal 6 for development that is attached to a part of the user's body, a PC 2 or a smartphone 3, and a server 5. The sensor terminal 1 used by the user or the sensor terminal 6 for development is connected to the PC 2 or the smartphone 3 so that data can be transmitted and received wirelessly. The PC 2 or smartphone 3 and the server 5 are connected via a network 4 so that data can be sent and received.

The sensor terminal 1 used by the user and the sensor terminal 6 for development are wearable terminals that are worn by the user to measure behavior, and include, for example, acceleration sensors that measure acceleration in each of three orthogonal axes. Note that the user-used sensor terminal 1 is a wearable terminal used by an individual user, and the development sensor terminal 6 is a wearable terminal used by a developer. Further, the object to which the sensor terminal 1 for use by the user or the sensor terminal for development 6 is attached is not limited to humans, but may be any moving object including animals, robots, and the like.

The PC 2 and the smartphone 3 are computers dedicated to the user, and transmit measurement data sent from the sensor terminal 1 used by the user or the sensor terminal 6 for development to the server 5 along with metadata, and receive data sent from the server 5. and display it. Metadata is data that describes the meaning of measurement data measured by the sensor terminal 1 used by the user or the sensor terminal 6 for development, and includes, for example, a user ID indicating the user, a terminal name indicating the wearable terminal used for measurement, etc. including.

The server 5 is a so-called computer, and includes a memory 11, a communication section 12, a CPU (Central Processing Unit) 13, and a database 14. The memory 11 is a device that temporarily stores data and programs, and temporarily stores, for example, a sensor data input/output program 21 and an axial direction change program 22. The communication unit 12 is a device that transmits and receives data via a network 4 such as a LAN, telephone line, or the Internet, and is connected to the CPU 13 . The CPU 13 is a device that executes various programs and calculations and controls the operation of each part, and executes, for example, a sensor data input/output program 21 and an axial direction change program 22 loaded into the memory 11.

The database 14 is a device that stores various data, and stores, for example, data processed by the sensor data input/output program 21 and the axial direction change program 22. Specifically, user-used sensor data 41, development sensor data 42, axial direction conversion model 43, post-application sensor data 44, sensor table 46, user table 47, attachment site table 48, and the like are stored in database 14. Further, the mounting direction table 45 may be stored in the database 14. Structures of various data will be described later.

The sensor data input/output program 21 is a program that transmits and receives data to and from the PC 2 and the smartphone 3 via the communication unit 12 and the network 4, and presents data stored in the database 14 to the user or input by the user. accept data.

The axial direction change program 22 is a program for changing the axial direction of measurement data measured by the sensor terminal 1 used by the user or the sensor terminal 6 for development. Contains a conversion program 33.

The metadata addition program 31 is a program that adds metadata transmitted together with the measurement data to the measurement data of the sensor terminal 1 used by the user or the sensor terminal 6 for development. By adding metadata to the measurement data of the sensor terminal 1 used by the user, the sensor data 41 used by the user is generated. Further, the development sensor data 42 is generated by adding metadata to the measurement data of the development sensor terminal 6. Note that since the user-used sensor terminal 1 is used in various scenes, development sensor data 42 corresponding to each scene is generated.

An example of the data structure of the development sensor data 42 and the user-used sensor data 41 will be described with reference to FIGS. 2A and 2B. The development sensor data 42 is composed of measurement data and metadata. The measurement data includes, for example, time 901, X-axis acceleration 902, Y-axis acceleration 903, and Z-axis acceleration 904. The metadata includes at least a user ID 905, a sensor terminal name 906, and a mounting site 907. The attachment site 907 stores a body site to which the development sensor terminal 6 is attached, such as a left wrist or a right ankle. Metadata may be provided for each measurement data at each time, or may be provided for measurement data at multiple times as illustrated in FIG. 2A.

Further, the metadata may further include a wearing direction 908 and a reference behavior 909. The mounting direction 908 stores a direction based on the forward direction when the development sensor terminal 6 is mounted. For example, when the development sensor terminal 6 is attached in the same direction as the predetermined forward direction, "forward direction" is stored, and when it is attached in the opposite direction to the forward direction, "reverse direction" is stored. Ru. Note that the development sensor terminal 6 may be left unstored on the premise that the direction in which it is attached is the same as the predetermined direction. Further, data regarding the difference in the mounting direction with respect to the forward direction may be stored in the form of Euler angles, quaternions, or rotation matrices. The standard behavior 909 stores a behavior that serves as a standard when measurement data is measured. For example, when the measurement data is measured while walking, "walking" is stored, and when the measurement data is measured during radio gymnastics, "radio gymnastics" is stored.

The user-used sensor data 41, like the development sensor data 42, is composed of measurement data and metadata. The measurement data includes, for example, time 911, X-axis acceleration 912, Y-axis acceleration 913, and Z-axis acceleration 914. The metadata includes at least a user ID 915, a sensor terminal name 916, and a mounting site 917. Further, the metadata may further include a mounting direction 918 and extraction conditions 919. The extraction condition 919 is a condition used when extracting a corresponding section, which will be described later, from the user-used sensor data 41.

Returning to the explanation of FIG. The axial direction conversion model creation program 32 is a program that generates an axial direction conversion model 43, which is a model for converting the axial direction, based on the user-used sensor data 41 and the development sensor data 42. The axial direction conversion program 33 is a program that uses the axial direction conversion model 43 to generate added sensor data 44 from the user-used sensor data 41 .

An example of the data structure of the axial direction conversion model 43 and the assigned sensor data 44 will be described using FIGS. 3A and 3B. The axial conversion model 43 includes, for example, an axial conversion model ID 951, a reference sensor terminal name 952, a target sensor terminal name 953, a mounting site 954, and a conversion formula 955. The axial direction conversion model ID 951 stores the ID of the axial direction conversion model 43 created by the axial direction conversion model creation program 32. The reference sensor terminal name 952 stores the terminal name of the development sensor terminal 6 that is used to measure the development sensor data 42 and serves as a reference in the axial direction. The target sensor terminal name 953 stores the terminal name of the user-used sensor terminal 1 that is used to measure the user-used sensor data 41 and whose axial direction is changed. The attachment site 954 stores the site where the development sensor terminal 6 is attached. The conversion formula 955 stores a mathematical formula used to change the axial direction.

Further, the axial direction conversion model 43 may further include a mounting direction 956, extraction conditions 957, one reference axis 958, and two other axes 959. The mounting direction 956 stores, for example, the direction in which the development sensor terminal 6 is mounted. The extraction condition 957 stores conditions used when extracting a corresponding section, which will be described later, from the user-used sensor data 41. The reference axis 958 stores which axis among the three axes of the development sensor terminal 6 and the user-used sensor terminal 1 is associated as the reference axis. The other two axes 959 store rotation angles for associating two axes other than the reference axis.

The applied sensor data 44 is composed of measurement data, metadata, and axis conversion data. The measurement data includes time 931, X-axis acceleration 932, Y-axis acceleration 933, and Z-axis acceleration 934. The metadata includes at least a user ID 935, a sensor terminal name 936, a mounting site 937, and may further include a mounting direction 938. The axis transformation data includes an axis transformation 939 and an axial direction transformation model ID 940. The axis conversion 939 stores data indicating whether or not axis conversion has been performed. FIG. 3B exemplifies "unfinished" indicating that the axis conversion has not been performed yet. Note that if axis conversion has not been performed, the measurement data of the user-used sensor data 41 is stored in the measurement data of the assigned sensor data 44, and if axis conversion has been performed, the measurement data with axis conversion is stored in the measurement data of the attached sensor data 44. It is stored in the measurement data of the sensor data 44. The axial direction conversion model ID 940 stores the ID of the axial direction conversion model 43 used in the axial direction conversion program 33.

Prior to analyzing the user's behavior, the behavior analysis system of the first embodiment determines the axial direction of the development sensor terminal 6, which is a wearable terminal used when developing the behavior analysis system, and the wearable terminal used by the individual user. Align with the axial direction of the sensor terminal 1 used by a certain user. The development sensor data 42 including measurement data of the development sensor terminal 6 is generated so as to be compatible with various situations in which the user-use sensor terminal 1 is used.

An example of the flow of processing for generating the development sensor data 42 will be explained for each processing step using FIG. 4 .

(S401)
Measurement data is collected by the development sensor terminal 6, and the collected measurement data is transmitted to the server 5 together with metadata.

(S402)
By executing the metadata adding program 31, metadata is added to the measurement data collected in S401, and development sensor data 42 is generated. The generated development sensor data 42 is stored in the database 14. The metadata includes at least the identifier of the development sensor terminal 6, the user's identifier, and the part to which the development sensor terminal 6 is attached. Note that the sensor table 46, user table 47, attachment site table 48, and attachment direction table 45 stored in the database 14 may be used to provide the metadata.

An example of the data structure of the sensor table 46, user table 47, attachment site table 48, and attachment direction table 45 will be described using FIGS. 5A, 5B, 5C, and 5D. The sensor table 46 includes, for example, a reference sensor terminal name 961, a reference sensor attachment part 962, a conversion destination sensor terminal name 964, a conversion destination sensor attachment part 965, and an axial conversion model ID 968. The reference sensor terminal name 961 stores the terminal name of the development sensor terminal 6 that is the reference in the axial direction. The reference sensor attachment portion 962 stores a portion to which the development sensor terminal 6, which serves as an axial reference, is attached. The conversion destination sensor terminal name 964 stores the terminal name of the user-used sensor terminal 1 whose axial direction is to be converted. The conversion destination sensor attachment site 965 stores a site where the user-used sensor terminal 1 whose axial direction is to be converted is attached. The axial direction conversion model ID 968 stores the ID of the axial direction conversion model 43 created by the axial direction conversion model creation program 32. Note that the axial direction conversion model ID 968 may not be stored before the axial direction conversion model 43 is created.

The sensor table 46 may further include a reference sensor mounting direction 963, a conversion destination sensor mounting direction 966, and an extraction condition 967. The reference sensor mounting direction 963 stores the direction in which the development sensor terminal 6, which serves as an axial reference, is mounted. The conversion destination sensor mounting direction 966 stores the direction in which the sensor terminal 1 used by the user whose axial direction is to be converted is mounted. The extraction condition 967 stores conditions used when extracting a corresponding section, which will be described later, from the user-used sensor data 41.

The user table 47 includes, for example, a user ID 971, a used sensor terminal name 972, and a mounting site 973. Further, the user table 47 may include a mounting direction 974, an axial direction conversion model ID 977, a usage start time 975, and a usage end time 976.

The attachment site table 48 includes, for example, the name of the sensor terminal used 981 and the allowable attachment site 982. The allowable mounting region 982 stores the region where the sensor terminal 1 used by the user is allowed to be mounted.

The mounting direction table 45 includes, for example, a used sensor terminal name 991 and a permissible mounting direction 992. The permissible mounting direction 992 stores the direction in which the sensor terminal 1 used by the user is allowed to be mounted.

Returning to the explanation of FIG. 4.

(S403)
A reference interval is set for the development sensor data 42 generated in S402. The reference interval may be set by the start time and end time determined by the user's button operation while measuring the measurement data, or may be set by a predetermined time after the detection of the behavior that becomes the reference when the measurement data is measured. It may be set as a period of time until the period of time elapses.

(S404)
Measurement data is collected for all conditions corresponding to various situations in which the sensor terminal 1 used by the user is used, and it is determined whether the reference section has been set. If reference intervals have been set for all conditions, the process flow ends; if not, the process returns to S401 and measurement data for uncollected conditions is collected. Note that the conditions may include the type of wearable terminal.

Through the above process flow, the development sensor data 42 is generated for all conditions corresponding to various situations in which the user sensor terminal 1 is used, and the development sensor data 42 is generated as a standard for each of the development sensor data 42. The interval is set. The axial conversion model creation program 32 creates an axial conversion model 43 using the development sensor data 42 in which the reference section has been set.

An example of the flow of processing for creating the axial transformation model 43 will be explained for each processing step using FIG. 6 .

(S601)
Measurement data is collected by the sensor terminal 1 used by the user, and the collected measurement data is transmitted to the server 5 together with metadata.

(S602)
By executing the metadata adding program 31, metadata is added to the measurement data collected in S601, and user-used sensor data 41 is generated. The metadata includes at least the identifier of the sensor terminal 1 used by the user, the identifier of the user, and the part to which the sensor terminal 1 used by the user is attached. The metadata may further include the direction in which the sensor terminal 1 used by the user is attached. Furthermore, the sensor table 46, user table 47, attachment site table 48, and attachment direction table 45 stored in the database 14 may be used to provide metadata.

(S603)
A corresponding section is extracted from the user-used sensor data 41 generated in S602. The corresponding section is a section corresponding to a reference section set for the development sensor data 42, and is extracted based on extraction conditions. The extraction conditions may be input by the user or set in advance according to criteria selected by the user. For example, the reference actions to be selected include walking, radio calisthenics, and a stationary state, and if walking is selected, a threshold value for a frequency power of a predetermined frequency band, for example, 2 Hz, is set as an extraction condition in one of the three axes. , the section in which the frequency power exceeds the threshold is defined as the corresponding section. Additionally, when radio calisthenics is selected, time-series pattern data representing a specific movement performed with a specific rhythm is set as an extraction condition, and a section with a higher degree of match with the time-series pattern data is set as a corresponding section. . Further, when a stationary state is selected, gravitational acceleration is set as an extraction condition, and a section in which gravitational acceleration continues to be maximum in one of the three axes is set as a corresponding section.

(S604)
It is determined whether the corresponding section extracted in S603 is appropriate. If appropriate, the process advances to S605; if inappropriate, the process returns to S601 and measurement data is collected again. Whether or not the corresponding section is appropriate is determined by, for example, whether or not a corresponding section that lasts a certain period of time or more is extracted. In other words, when the standard behavior of walking, radio calisthenics, or standing still is detected for a certain period of time or more, it is determined that the corresponding section has been appropriately extracted.

(S605)
The development sensor data 42 including the reference section corresponding to the corresponding section extracted in S603 is read. Since the database 14 stores development sensor data 42 corresponding to various situations in which the user sensor terminal 1 is used, the corresponding development sensor data 42 is read out from therein based on the metadata. Note that the metadata of the user-used sensor data 41 and the metadata of the development sensor data 42 do not necessarily have to completely match. For example, if the attachment site 917 of the user-used sensor data 41 is the left wrist, the attachment site 907 of the development sensor data 42 may be the right wrist. In other words, the attachment site is allowed to be a bilaterally symmetrical site, and the symmetry of the body is utilized in subsequent processing.

(S606)
Pre-processing is performed on the user-used sensor data 41 of the corresponding section extracted in S603, so that, for example, the sampling rate and dynamic range of the user-used sensor data 41 are aligned with those of the development sensor data 42. Furthermore, if the attachment site 917 of the user-used sensor data 41 and the attachment site 907 of the development sensor data 42 are bilaterally symmetrical, the user-use sensor data 41 may be processed based on the symmetry, for example, when the measurement data in the horizontal direction is A process of exchanging the positive and negative is performed.

(S607)
One axis of each of the three axes of the generated development sensor data 42 read in S605 and the user-use sensor data 41 subjected to preprocessing in S606 is associated with each other. For example, if the standard behavior of both data is walking, the frequency power is calculated by frequency analysis using Fourier transform for each of the three axes of both data, and the one axis with the maximum frequency power among the three axes is Can be matched. Further, when the reference action is radio gymnastics, the degree of similarity between the time series data of each of the three axes of both data is calculated, and the one axis having the maximum degree of similarity is associated with each other. For example, a dynamic time warping method or the like is used to calculate the similarity between time series data. Furthermore, when both data are measured in a stationary state, the one axis with the maximum gravitational acceleration is associated with each other. The associated single axes are used as reference axes for both data, and a uniaxial conversion model RefAx is calculated, which is a model for rotating the user-used sensor data 41 so that both reference axes coincide, including the positive and negative directions.

(S608)
Of the three axes of the development sensor data 42 and the user-used sensor data 41, two axes other than the reference axis are associated with each other. An example of a specific procedure for associating two axes will be described.

First, using the uniaxial conversion model RefAx calculated in S607, the user-used sensor data 41 is rotated so that the directions of both reference axes coincide. The three axes of the development sensor data 42 are [Xdev, Ydev, Zdev], the three axes of the user sensor data 41 are [x, y, z], and the three axes of the user sensor data 41 after rotation are [x2, y2]. , z2], then [x2, y2, z2]=RefAx·[x, y, z] using the uniaxial transformation model RefAx.

Then, the rotated user-used sensor data 41 is further rotated around the reference axis to associate the two axes other than the reference axis. When Zdev and z are associated as reference axes, z2 is parallel to the reference axis Zdev of the development sensor data 42, so by rotating [x2, y2, z2] around z2, The two axes of are associated with each other. In other words, Rot(θ)・[x2, y2, z2]=Rot(θ)・RefAx・[x, y, z] using the rotation matrix Rot(θ) regarding the rotation angle θ are the three axes of the development sensor data 42. It is associated with [Xdev, Ydev, Zdev]. Therefore, the rotation angle θ' that minimizes the error between the two is calculated, and Rot(θ') using θ' becomes the other two-axis conversion model, which is a model that associates two axes other than the reference axis. Note that if the minimum value of both errors is larger than a predetermined tolerance, a warning may be issued that there is a problem in the attachment of the sensor terminal 1 used by the user.

Furthermore, if the acceleration sensors included in the development sensor terminal 6 and the user-used sensor terminal 1 are oriented in the same direction and the two terminals are attached to the same location, the degree of freedom in the direction in which both terminals are attached is limited. Therefore, we limited the search range of the rotation angle θ to four types: 0°, 90°, 180°, and 270°, calculated the error with the development sensor data 42 for each angle, and calculated the θ that minimizes the error. ' may be used to calculate another two-axis transformation model Rot(θ'). Furthermore, if the user-used sensor data 41 and the development sensor data 42 include the mounting direction, and the mounting directions of both match, the rotation angle θ=0°, so the rotation angle θ' is predetermined to minimize the error. It may be determined whether the mounting direction is correct or not by comparing it with the determined tolerance ε. In other words, if |θ′|≦ε, it is determined that the mounting direction is approximately correct; if |θ′|>ε, it is determined that the mounting direction is incorrect, and even if a warning to that effect is issued. good.

Further, to minimize the error with the development sensor data 42, frequency power when the reference action is walking or similarity between time series data when the reference action is radio gymnastics may be used. Furthermore, the correspondence between two axes other than the reference axis is not limited to the rotation matrix Rot(θ), but may also be a rotation vector, a quaternion, or an Euler angle.

(S609)
The axial direction conversion model 43 is generated based on the uniaxial conversion model RefAx calculated in S607 and the other two-axis conversion model Rot(θ') calculated in S608. That is, the axial transformation model 43 is generated as Rot(θ')·RefAx.

Through the above processing flow, the axial direction conversion model 43, which is a model for converting the axial direction, is generated. The generation of the axial transformation model 43 is executed automatically at the timing when the server 5 receives the sensor data 41 used by the user, periodically, or based on a user's instruction. The generated axial direction conversion model 43 is used in the process of changing the axial direction of ' to the axial direction of the development sensor terminal 6.

An example of the process flow for changing the axial direction of the sensor terminal 1 used by the user to the axial direction of the sensor terminal 6 for development will be explained for each processing step using FIG. 7 .

(S701)
Measurement data is collected by the sensor terminal 1 used by the user, and the collected measurement data is transmitted to the server 5 together with metadata.

(S702)
By executing the metadata adding program 31, metadata is added to the measurement data collected in S701, and user-used sensor data 41 is generated.

(S703)
Pre-processing is performed on the user-used sensor data 41 generated in S702, such as aligning the sampling rate and dynamic range of the user-used sensor data 41 with those of the development sensor data 42, or processing based on symmetry. It will be implemented. Note that processing related to the sampling rate and dynamic range is not essential because it is performed in the process of generating the axial transformation model 43.

(S704)
The axial direction of the user-used sensor data 41 that has been preprocessed in S703 is changed, and post-applied sensor data 44 is generated. An axial direction conversion model 43 is used to change the axial direction. The metadata of the assigned sensor data 44 may include a flag indicating whether or not the axis conversion has been performed or not, and the ID of the axis direction conversion model 43. If axis conversion has not been performed, the measurement data of the assigned sensor data 44 remains the measurement data of the user-used sensor data 41, and axis conversion can be performed at any timing by referring to the ID of the axis direction conversion model 43. It may be done.

(S705)
It is determined whether or not to continue collecting measurement data by the sensor terminal 1 used by the user. If the process is to be continued, the process returns to S701; if the process is not to be continued, the process flow ends.

Through the above process flow, the axial direction of the sensor terminal 1 used by the user is changed to the axial direction of the sensor terminal 6 for development. The change result in the axial direction may be displayed on the PC 2 or the smartphone 3.

An example of a screen displayed on the smartphone 3 will be explained using FIGS. 8 and 9. The screen illustrated in FIG. 8 is divided into an upper stage, a middle stage, and a lower stage, and a sensor data display section 101 during measurement, a start button 102, and an end button 103 are arranged in the upper stage of the screen. The measuring sensor data display section 101 displays measurement data that is three-axis acceleration data that is measured by the sensor terminal 1 used by the user and whose axial direction has been changed by the axial direction conversion model 43. Note that the display may be switched to the display of measurement data before the axial direction is changed by the axis conversion switching unit 109, which will be described later. The start button 102 and the end button 103 are buttons for instructing the user's sensor terminal 1 to start and end measurement. In FIG. 8, since measurement is in progress, the start button 102 is grayed out.

A first feature amount display section 104 and a second feature amount display section 105 are arranged in the middle of the screen. The first feature amount display section 104 displays the feature amount extracted from the development sensor data 42, and the second feature amount display section 105 displays the feature amount extracted from the user-used sensor data 41. In FIG. 8, frequency power calculated from measurement data during walking is illustrated as a feature amount, and a walking frequency band of around 2 Hz is used as a reference. Furthermore, time-series data patterns obtained from the development sensor data 42 and the user-used sensor data 41 during radio exercise may be displayed on the first feature amount display section 104 and the second feature amount display section 105. By comparing the feature amounts displayed on the first feature amount display section 104 and the second feature amount display section 105, the user can qualitatively confirm that the axial directions are aligned.

At the bottom of the screen, a user ID input/display section 106, sensor terminal name input/display section 107, attachment site input/display section 108, axis conversion switching section 109, and reference axis extraction condition input/display section 110 are arranged. . The user ID is displayed on the user ID input/display section 106, and the user ID may be re-entered as necessary. The sensor terminal name is displayed in the sensor terminal name input/display section 107, and the sensor terminal name may be re-inputted if necessary. The attachment site is displayed on the attachment site input/display section 108, and the attachment site may be re-inputted as necessary. The axis conversion switching unit 109 displays "completed" or "uncompleted" indicating whether or not the axis conversion has been performed, and the two can be switched as necessary. If the axis conversion switching unit 109 is “completed”, the measurement data whose axis direction has been changed is displayed on the sensor data display unit 101 during measurement, and if it is “uncompleted”, the measurement data before the axis direction is changed is displayed. Measurement data will be displayed. The reference axis extraction condition input/display unit 110 displays the reference axis extraction condition, which is a condition for extracting the corresponding section, and the reference axis extraction condition may be input again as necessary.

Note that if the axial directions cannot be aligned for some reason, a warning screen illustrated in FIG. 9 may be displayed. In the warning screen of FIG. 9, the frequency power calculated from the measurement data during walking is maximum around 6 Hz, and a warning 115 is displayed urging the user to reduce the walking speed which is too fast. The warning 115 may display a prompt to change the extraction conditions of the corresponding section.

According to the first embodiment described above, even if the user-used sensor data 41 and the development-use sensor data 42 are measured at different timings, the axial directions of the user-use sensor terminal 1 and the development-use sensor terminal 6 can be aligned. Further, the screen illustrated in FIGS. 8 and 9 allows the user to confirm that the axial directions have been aligned or that the axial directions have not been aligned for some reason.

In the first embodiment, the axial directions of the user-used sensor terminal 1 and the development-purpose sensor terminal 6 are aligned based on the user-used sensor data 41 and the development-use sensor data 42 measured at different timings. In the second embodiment, a description will be given of analyzing a user's behavior using a behavior analysis system in which the axis directions of the sensor terminal 1 used by the user and the sensor terminal 6 for development are aligned. Note that the same components as in the first embodiment are given the same reference numerals, and the description thereof will be omitted.

The overall configuration of the behavior analysis system of Example 2 will be described using FIG. 10. The behavior analysis system of the second embodiment has a behavior analysis program 23, a behavior analysis model 49, and behavior analysis result data 50 added to the first embodiment. That is, the CPU 13 executes the behavior analysis program 23 loaded into the memory 11 along with the sensor data input/output program 21 and the axis direction change program 22. The database 14 also stores a behavior analysis model 49 and behavior analysis result data 50.

The behavior analysis program 23 is a program that analyzes user behavior based on the behavior analysis model 49. The behavior analysis program 23 determines, for example, sleep state or sleep level, or detects movements such as walking, swinging an arm, or hitting a hammer.

The behavior analysis model 49 is a model that outputs the user's behavior from the measurement data measured by the sensor terminal 1 used by the user, and is created using the measurement data measured under various conditions by the development sensor terminal 6. . Machine learning may be used to create the behavior analysis model 49, and learning is performed using, for example, development sensor data 42 measured during a specific behavior as teacher data. Behavior analysis result data 50 is data output from the behavior analysis model 49.

An example of the flow of behavior analysis processing by the behavior analysis system of the second embodiment will be explained step by step with reference to FIG. 11 .

(S1101)
After the provision, the sensor data 44 is input to the behavior analysis program 23, and the user's behavior is analyzed. In behavior analysis, behaviors performed during work, such as detection of walking motion, picking and delivery of goods, are determined. The analyzed results are stored in the database 14 as behavioral analysis result data 50.

In S1101, the assigned sensor data 44 converted into the axial direction of the development sensor data 42 measured by the development sensor terminal 6 is input, so the behavior analysis model 49 is used without being affected by the axial direction. be able to. That is, when the user-used sensor data 41 measured by the user-used sensor terminal 1 is inputted as is, the analysis accuracy decreases due to the influence in the axial direction, but is improved by inputting the assigned sensor data 44. The improvement effect is particularly significant when the behavior analysis model 49 is configured based on a neural network.

(S1102)
It is determined whether or not to continue behavioral analysis. If the process is to be continued, the process returns to S1101, and if it is not to be continued, the process is to proceed to S1103.

(S1103)
It is determined whether or not to relearn the behavior analysis model 49. If re-learning is to be performed, the process proceeds to S1104, and if re-learning is not to be performed, the process flow ends.

(S1104)
Relearning of the behavior analysis model 49 is performed using the sensor data 44 after the addition. For example, relearning using only the assigned sensor data 44 is performed using the neural network weights of the existing behavioral analysis model 49 as a starting point. Alternatively, data obtained by adding sensor data 44 after assignment to the development sensor data 42 used to create the existing behavior analysis model 49 is used as training data, and relearning is performed after weighting the neural network. good. Since the assigned sensor data 44 has the same axis direction as the development sensor data 42, it can be easily used for relearning the behavior analysis model 49.

Through the above processing flow, highly accurate behavior analysis is executed. The results of the behavior analysis may be displayed on the PC 2 or the smartphone 3.

An example of a screen displayed on the smartphone 3 will be described using FIG. 12. In the screen illustrated in FIG. 12, a walking pitch display section 111 and an article picking display section 112 are displayed instead of the first feature display section 104 and the second feature display section 105 arranged in the middle of the screen in FIG. Placed. The walking pitch display section 111 displays the transition of the user's walking pitch, and the article picking display section 112 displays the determination result of article picking.

According to the above-described second embodiment, by using the assigned sensor data 44, behavior analysis can be performed without being affected by the difference in the axial direction between the development sensor data 42 and the user-used sensor data 41. Accurate analysis results can be obtained. Furthermore, since the post-application sensor data 44 is axially aligned with the development sensor data 42, the behavior analysis model 49 can be retrained using the post-application sensor data 44.

A plurality of embodiments of the present invention have been described above. The present invention is not limited to the above-described embodiments, but can be embodied by modifying the constituent elements without departing from the gist of the invention. Further, a plurality of components disclosed in the above embodiments may be combined as appropriate. Furthermore, some components may be deleted from all the components shown in the above embodiments.

Part or all of the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in hardware by designing, for example, an integrated circuit. Further, each of the above configurations, functions, etc. may be realized by software by a processor interpreting and executing a program for realizing each function. Information such as programs, tables, files, etc. that realize each function is stored in storage devices such as memory, hard disk drive, SSD (Solid State Drive), or computer-readable non-temporary data such as IC cards, SD cards, and DVDs. It can be stored on a storage medium.

In addition, the drawings show control lines and information lines that are considered necessary for explaining the embodiments, and do not necessarily show all control lines and information lines included in an actual product to which the present invention is applied. It doesn't necessarily mean that In reality, almost all configurations may be considered to be interconnected.

1: Sensor terminal used by user, 2: PC, 3: Smartphone, 4: Network, 5: Server, 6: Sensor terminal for development, 11: Memory, 12: Communication department, 13: CPU, 14: Database, 21: Sensor Data input/output program, 22: Axis direction change program, 23: Behavior analysis program, 31: Metadata addition program, 32: Axis direction conversion model creation program, 33: Axis direction conversion program, 41: Sensor data used by user, 42: Development sensor data, 43: Axial direction conversion model, 44: Sensor data after provision, 45: Attachment direction table, 46: Sensor table, 47: User table, 48: Attachment site table, 49: Behavior analysis model, 50: Behavior Analysis result data, 101: Sensor data display section during measurement, 102: Start button, 103: End button, 104: First feature amount display section, 105: Second feature amount display section, 106: User ID input/display section, 107: Sensor terminal name input/display section, 108: Mounting site input/display section, 109: Axis conversion switching section, 110: Reference axis extraction condition input/display section, 111: Walking pitch display section, 112: Article picking display section , 115: Warning, 901: Time, 902: X-axis acceleration, 903: Y-axis acceleration, 904: Z-axis acceleration, 905: User ID, 906: Sensor terminal name, 907: Attachment site, 908: Attachment direction, 909: Reference behavior, 911: Time, 912: X-axis acceleration, 913: Y-axis acceleration, 914: Z-axis acceleration, 915: User ID, 916: Sensor terminal name, 917: Attachment site, 918: Attachment direction, 919: Extraction condition , 931: Time, 932: X-axis acceleration, 933: Y-axis acceleration, 934: Z-axis acceleration, 935: User ID, 936: Sensor terminal name, 937: Attachment site, 938: Attachment direction, 939: Axis conversion, 940 : Axial direction conversion model ID, 951: Axial direction conversion model ID, 952: Reference sensor terminal name, 953: Target sensor terminal name, 954: Mounting site, 955: Conversion formula, 956: Mounting direction, 957: Extraction condition, 958 : Reference one axis, 959: Other two axes, 961: Reference sensor terminal name, 962: Reference sensor mounting part, 963: Reference sensor mounting direction, 964: Conversion destination sensor terminal name, 965: Conversion destination sensor mounting part, 966: Conversion Previous sensor attachment direction, 967: Extraction condition, 968: Axial direction conversion model ID, 971: User ID, 972: Used sensor terminal name, 973: Attachment site, 974: Attachment direction, 975: Usage start time, 976: Usage end Time, 977: Axial direction conversion model ID, 981: Name of sensor terminal used, 982: Allowable mounting location, 991: Name of sensor terminal used, 992: Allowable mounting direction

Claims (11)

A behavior analysis system that analyzes user behavior,
a setting unit that sets a reference section for each of various first measurement data measured in a plurality of axial directions by the first sensor and assigned a reference behavior ;
A corresponding section corresponding to the reference section is set in advance according to a criterion by which the user selects a corresponding section from among second measurement data measured in a plurality of axial directions by a second sensor at a timing different from the first measurement data. an extraction unit that extracts based on extraction conditions ;
The axial direction is determined based on the first measurement data of the reference section and the second measurement data of the corresponding section, which are read out from the various first measurement data based on the reference behavior corresponding to the extraction condition. a generation unit that generates an axial transformation model that is a model to be transformed;
A behavior analysis system comprising: a conversion unit that converts the axial direction of the second sensor to the axial direction of the first sensor using the axial direction conversion model. The behavior analysis system according to claim 1,
The behavior analysis system is characterized in that the extraction unit extracts the corresponding section based on a setting condition when the reference section is set. The behavior analysis system according to claim 2,
The behavior analysis system is characterized in that the setting conditions include one of walking, radio calisthenics, and a resting state as a standard behavior. The behavior analysis system according to claim 1,
The generation unit associates one axis of the plurality of axes of the first measurement data and the second measurement data with each other as a reference axis, and then sets one of the first measurement data and the second measurement data as the reference axis. A behavior analysis system characterized in that the axial direction transformation model is generated by rotating the model around an axis and associating it with axes other than the reference axis. The behavior analysis system according to claim 4,
The behavior analysis system is characterized in that the generation unit associates, as the reference axis, one axis in which the frequency power calculated for each of a plurality of axes of the first measurement data and the second measurement data is maximum. The behavior analysis system according to claim 4,
The generation unit obtains time series data for each of a plurality of axes of the first measurement data and the second measurement data, and associates one axis with a maximum degree of similarity between the time series data as the reference axis. A behavior analysis system featuring: The behavior analysis system according to claim 1,
When the part to which the first sensor is attached and the part to which the second sensor is attached are bilaterally symmetrical, the generation section replaces the positive and negative of the left and right data of the second measurement data, and then generates the second measurement data. A behavior analysis system characterized by generating an axial transformation model. The behavior analysis system according to claim 1,
A behavior analysis system characterized in that the first measurement data and the second measurement data include a wearing direction as metadata. The behavior analysis system according to claim 1,
a storage unit that stores a behavior analysis model that is a model for analyzing the behavior and is created by learning the first measurement data;
The behavior analysis system further includes an analysis unit that outputs an analysis result of the behavior by inputting converted data generated by converting the second measurement data by the conversion unit into the behavior analysis model. . The behavior analysis system according to claim 9,
A behavior analysis system further comprising a relearning unit that retrains the behavior analysis model using the converted data. A behavior analysis method using a behavior analysis system that analyzes user behavior,
a setting step of setting a reference interval for each of various first measurement data measured in a plurality of axial directions by the first sensor and assigned a reference action ;
A corresponding section corresponding to the reference section is set in advance according to a criterion by which the user selects a corresponding section from among second measurement data measured in a plurality of axial directions by a second sensor at a timing different from the first measurement data. an extraction step for extracting based on extraction conditions ;
The axial direction is determined based on the first measurement data of the reference section and the second measurement data of the corresponding section, which are read out from the various first measurement data based on the reference behavior corresponding to the extraction condition. a generation step of generating an axial transformation model that is a model to be transformed;
A behavior analysis method comprising a conversion step of converting the axial direction of the second sensor to the axial direction of the first sensor using the axial direction conversion model.

Images