US20170143226A1

US20170143226A1 - Action Recognition Method and Device Based on Surface Electromyography Signal

Info

Publication number: US20170143226A1
Application number: US15/428,230
Authority: US
Inventors: Qiang Ding; Xiaorong Gao; Ke Lin
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2014-08-13
Filing date: 2017-02-09
Publication date: 2017-05-25
Also published as: CN105326500B; EP3167799A1; EP3167799A4; WO2016023349A1; CN105326500A

Abstract

An action recognition method based on a surface electromyography signal includes obtaining surface electromyography signals of multiple channels, determining a valid surface electromyography signal according to the surface electromyography signals of the multiple channels, determining a frequency of the valid surface electromyography signal, and determining, according to the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels. A frequency of a surface electromyography signal is irrelevant to a feature such as signal strength, therefore, the method can significantly improve accuracy of action recognition based on a surface electromyography signal. Moreover, with a frequency being used as a recognition feature, a user does not need to conduct an action with a large range, which brings better user experience.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of International Application No. PCT/CN2015/073369, filed on Feb. 27, 2015, which claims priority to Chinese Patent Application No. 201410396422.5, filed on Aug. 13, 2014. The disclosures of the aforementioned applications are hereby incorporated by reference in their entireties.

TECHNICAL FIELD

Embodiments of the present disclosure relate to action recognition technologies, and in particular, to an action recognition method and device based on a surface electromyography signal.

BACKGROUND

A surface electromyography signal is a type of bioelectric signal related to a neuromuscular activity, and the surface electromyography signal can reflect information such as muscle contraction mode and contraction strength. Different body actions are corresponding to different surface electromyography signals, and a specific action corresponding to a surface electromyography signal may be determined by analyzing the surface electromyography signal. Therefore, a surface electromyography signal is widely applied to many fields such as clinical medicine, sport medicine, biomedicine, and rehabilitation engineering.
An amplitude feature of a surface electromyography signal is used as a recognition parameter to recognize a body action. According to the method, sliding window processing is performed on a collected surface electromyography signal, to obtain a window sequence of each sliding window, then, a window sequence amplitude of each window is calculated to obtain an amplitude feature, and the amplitude feature is compared with an amplitude feature of a surface electromyography signal corresponding to a body action that is obtained by means of pre-training to determine a body action corresponding to the amplitude feature.
The method has the following problems Because another user action may also generate a surface electromyography signal, when a user conducts another motion (running, typing, or the like), a generated surface electromyography interference signal may be overlapped with a target surface electromyography signal, and consequently, recognition accuracy is low, and in addition, a magnitude of a surface electromyography signal amplitude is proportional to a user body action amplitude, and to reach a relatively high signal-to-noise ratio, the user needs to conduct a relatively intense body action, which makes the user feel tired during a long-time operation.

SUMMARY

Embodiments of the present disclosure provide an action recognition method and device based on a surface electromyography signal, so that accuracy of action recognition based on a surface electromyography signal can be improved.
A first aspect of the present disclosure provides an action recognition method based on a surface electromyography signal, including obtaining surface electromyography signals of multiple channels, determining a valid surface electromyography signal according to the surface electromyography signals of the multiple channels, determining a frequency of the valid surface electromyography signal, and determining, according to the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.
With reference to the first aspect of the present disclosure, in a first possible implementation manner of the first aspect of the present disclosure, the determining a valid surface electromyography signal according to the surface electromyography signals of the multiple channels includes overlapping the surface electromyography signals of the multiple channels, and dividing the overlapped surface electromyography signals by a quantity of the channels to obtain a single-channel surface electromyography signal, sliding, starting from a start time of the single-channel surface electromyography signal, the single-channel surface electromyography signal at each sliding moment to obtain a window corresponding to each sliding moment, determining a window sequence corresponding to each sliding moment, and calculating an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, where there is one sliding interval between every two adjacent sliding moments, the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, and the window sequence corresponding to the sliding moment includes a total of N consecutive windows a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, where N is a positive integer greater than or equal to 2, and if an average surface electromyography signal amplitude of a window sequence corresponding to a sliding moment T in all sliding moments is not less than a preset amplitude, using a start time of the window sequence corresponding to the sliding moment T as a start time of the valid surface electromyography signal, adding a preset time to the start time of the valid surface electromyography signal to obtain an end time of the valid surface electromyography signal, and capturing and using a surface electromyography signal of the multiple channels that is located between the start time and the end time of the valid surface electromyography signal as the valid surface electromyography signal.
With reference to the first possible implementation manner of the first aspect of the present disclosure, in a second possible implementation manner of the first aspect of the present disclosure, the preset amplitude is an average absolute value of amplitudes of the surface electromyography signals obtained by overlapping the surface electromyography signals of the multiple channels.
With reference to the first aspect of the present disclosure and the first and second possible implementation manners of the first aspect of the present disclosure, in a third possible implementation manner of the first aspect of the present disclosure, the determining a frequency of the valid surface electromyography signal includes calculating a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices, where the sine-cosine matrix includes fundamental-frequency and multiplied-frequency sine functions and cosine functions, and all of the sine-cosine matrices have different fundamental frequencies, determining whether a greatest correlation coefficient in the correlation coefficients between the valid surface electromyography signal and the multiple sine-cosine matrices is greater than a preset correlation coefficient, and if the greatest correlation coefficient is greater than the preset correlation coefficient, using a fundamental frequency of a sine-cosine matrix corresponding to the greatest correlation coefficient as the frequency of the valid surface electromyography signal.
A second aspect of the present disclosure provides an action recognition method based on a surface electromyography signal, including obtaining surface electromyography signals of multiple channels, determining a valid surface electromyography signal according to the surface electromyography signals of the multiple channels, determining a frequency of the valid surface electromyography signal, extracting an amplitude feature of the valid surface electromyography signal, and determining, according to the amplitude feature of the valid surface electromyography signal and the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.
With reference to the second aspect of the present disclosure, in a first possible implementation manner of the second aspect of the present disclosure, the determining a valid surface electromyography signal according to the surface electromyography signals of the multiple channels includes overlapping the surface electromyography signals of the multiple channels, and dividing the overlapped surface electromyography signals by a quantity of the channels to obtain a single-channel surface electromyography signal, sliding, starting from a start time of the single-channel surface electromyography signal, the single-channel surface electromyography signal at each sliding moment to obtain a window corresponding to each sliding moment, determining a window sequence corresponding to each sliding moment, and calculating an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, where there is one sliding interval between every two adjacent sliding moments, the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, and the window sequence corresponding to the sliding moment includes a total of N consecutive windows a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, where N is a positive integer greater than or equal to 2, and when an average surface electromyography signal amplitude of a window sequence corresponding to a sliding moment T in all sliding moments is not less than a preset amplitude, using a start time of the window sequence corresponding to the sliding moment T as a start time of the valid surface electromyography signal, adding a preset time to the start time of the valid surface electromyography signal to obtain an end time of the valid surface electromyography signal, and capturing and using a surface electromyography signal of the multiple channels that is located between the start time and the end time of the valid surface electromyography signal as the valid surface electromyography signal.
With reference to the first possible implementation manner of the second aspect of the present disclosure, in a second possible implementation manner of the second aspect of the present disclosure, the preset amplitude is an average absolute value of amplitudes of the surface electromyography signals obtained by overlapping the surface electromyography signals of the multiple channels.
With reference to the second aspect of the present disclosure and the first and second possible implementation manners of the second aspect of the present disclosure, in a third possible implementation manner of the second aspect of the present disclosure, the determining a frequency of the valid surface electromyography signal includes calculating a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices, where the sine-cosine matrix includes fundamental-frequency and multiplied-frequency sine functions and cosine functions, and all of the sine-cosine matrices have different fundamental frequencies, determining whether a greatest correlation coefficient in the correlation coefficients between the valid surface electromyography signal and the multiple sine-cosine matrices is greater than a preset correlation coefficient, and if the greatest correlation coefficient is greater than the preset correlation coefficient, using a fundamental frequency of a sine-cosine matrix corresponding to the greatest correlation coefficient as the frequency of the valid surface electromyography signal.
With reference to the third possible implementation manner of the second aspect of the present disclosure, in a fourth possible implementation manner of the second aspect of the present disclosure, the extracting an amplitude feature of the valid surface electromyography signal includes separately performing sliding window processing on surface electromyography signals of each channel of the valid surface electromyography signal, and calculating an average amplitude of each sliding window of the surface electromyography signals of each channel of the valid surface electromyography signal, where the average amplitude of each sliding window is an average absolute value of amplitudes of surface electromyography signals within each sliding window, and using the average amplitude of each sliding window of the valid surface electromyography signal as the amplitude feature of the valid surface electromyography signal.
With reference to the second aspect of the present disclosure and the first and second possible implementation manners of the second aspect of the present disclosure, in a fifth possible implementation manner of the second aspect of the present disclosure, the determining, according to the amplitude feature of the valid surface electromyography signal and the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels includes determining, according to the frequency of the valid surface electromyography signal, multiple alternative body actions corresponding to the surface electromyography signals of the multiple channels, and performing matching between the amplitude feature of the valid surface electromyography signal and an amplitude feature of the multiple alternative body actions that is obtained by means of pre-training, to obtain a body action matched with the amplitude feature of the valid surface electromyography signal, and using the body action matched with the amplitude feature of the valid surface electromyography signal as the body action corresponding to the surface electromyography signals of the multiple channels.
A third aspect of the present disclosure provides an action recognition device based on a surface electromyography signal, including an obtaining module, configured to obtain surface electromyography signals of multiple channels, a first determining module, configured to determine a valid surface electromyography signal according to the surface electromyography signals of the multiple channels, a second determining module, configured to determine a frequency of the valid surface electromyography signal, and a recognition module, configured to determine, according to the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.
With reference to the third aspect of the present disclosure, in a first possible implementation manner of the third aspect of the present disclosure, the first determining module is configured to overlap the surface electromyography signals of the multiple channels, and divide the overlapped surface electromyography signals by a quantity of the channels to obtain a single-channel surface electromyography signal, slide, starting from a start time of the single-channel surface electromyography signal, the single-channel surface electromyography signal at each sliding moment to obtain a window corresponding to each sliding moment, determine a window sequence corresponding to each sliding moment, and calculate an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, where there is one sliding interval between every two adjacent sliding moments, the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, and the window sequence corresponding to the sliding moment includes a total of N consecutive windows a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, where N is a positive integer greater than or equal to 2, and if an average surface electromyography signal amplitude of a window sequence corresponding to a sliding moment T in all sliding moments is not less than a preset amplitude, use a start time of the window sequence corresponding to the sliding moment T as a start time of the valid surface electromyography signal, add a preset time to the start time of the valid surface electromyography signal to obtain an end time of the valid surface electromyography signal, and capture and use a surface electromyography signal of the multiple channels that is located between the start time and the end time of the valid surface electromyography signal as the valid surface electromyography signal.
With reference to the first possible implementation manner of the third aspect of the present disclosure, in a second possible implementation manner of the third aspect of the present disclosure, the preset amplitude is an average absolute value of amplitudes of the surface electromyography signals obtained by overlapping the surface electromyography signals of the multiple channels.
With reference to the third aspect of the present disclosure and the first and second possible implementation manners of the third aspect of the present disclosure, in a third possible implementation manner of the third aspect of the present disclosure, the second determining module is configured to calculate a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices, where the sine-cosine matrix includes fundamental-frequency and multiplied-frequency sine functions and cosine functions, and all of the sine-cosine matrices have different fundamental frequencies, determine whether a greatest correlation coefficient in the correlation coefficients between the valid surface electromyography signal and the multiple sine-cosine matrices is greater than a preset correlation coefficient, and if the greatest correlation coefficient is greater than the preset correlation coefficient, use a fundamental frequency of a sine-cosine matrix corresponding to the greatest correlation coefficient as the frequency of the valid surface electromyography signal.
A fourth aspect of the present disclosure provides an action recognition device based on a surface electromyography signal, including an obtaining module, configured to obtain surface electromyography signals of multiple channels, a first determining module, configured to determine a valid surface electromyography signal according to the surface electromyography signals of the multiple channels, a second determining module, configured to determine a frequency of the valid surface electromyography signal, an extraction module, configured to extract an amplitude feature of the valid surface electromyography signal, and a recognition module, configured to determine, according to the amplitude feature of the valid surface electromyography signal and the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.
With reference to the fourth aspect of the present disclosure, in a first possible implementation manner of the fourth aspect of the present disclosure, the first determining module is configured to overlap the surface electromyography signals of the multiple channels, and divide the overlapped surface electromyography signals by a quantity of the channels to obtain a single-channel surface electromyography signal, slide, starting from a start time of the single-channel surface electromyography signal, the single-channel surface electromyography signal at each sliding moment to obtain a window corresponding to each sliding moment, determine a window sequence corresponding to each sliding moment, and calculate an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, where there is one sliding interval between every two adjacent sliding moments, the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, and the window sequence corresponding to the sliding moment includes a total of N consecutive windows a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, where N is a positive integer greater than or equal to 2, and when an average surface electromyography signal amplitude of a window sequence corresponding to a sliding moment T in all sliding moments is not less than a preset amplitude, use a start time of the window sequence corresponding to the sliding moment T as a start time of the valid surface electromyography signal, adding a preset time to the start time of the valid surface electromyography signal to obtain an end time of the valid surface electromyography signal, and capture and use a surface electromyography signal of the multiple channels that is located between the start time and the end time of the valid surface electromyography signal as the valid surface electromyography signal.
With reference to the first possible implementation manner of the fourth aspect of the present disclosure, in a second possible implementation manner of the fourth aspect of the present disclosure, the preset amplitude is an average absolute value of amplitudes of the surface electromyography signals obtained by overlapping the surface electromyography signals of the multiple channels.
With reference to the fourth aspect of the present disclosure and the first and second possible implementation manners of the fourth aspect of the present disclosure, in a third possible implementation manner of the fourth aspect of the present disclosure, the second determining module is configured to calculate a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices, where the sine-cosine matrix includes fundamental-frequency and multiplied-frequency sine functions and cosine functions, and all of the sine-cosine matrices have different fundamental frequencies, determine whether a greatest correlation coefficient in the correlation coefficients between the valid surface electromyography signal and the multiple sine-cosine matrices is greater than a preset correlation coefficient, and if the greatest correlation coefficient is greater than the preset correlation coefficient, use a fundamental frequency of a sine-cosine matrix corresponding to the greatest correlation coefficient as the frequency of the valid surface electromyography signal.
With reference to the third possible implementation manner of the fourth aspect of the present disclosure, in a fourth possible implementation manner of the fourth aspect of the present disclosure, the extraction module is configured to separately perform sliding window processing on surface electromyography signals of each channel of the valid surface electromyography signal, and calculate an average amplitude of each sliding window of the surface electromyography signals of each channel of the valid surface electromyography signal, where the average amplitude of each sliding window is an average absolute value of amplitudes of surface electromyography signals within each sliding window, and use the average amplitude of each sliding window of the valid surface electromyography signal as the amplitude feature of the valid surface electromyography signal.
With reference to the fourth aspect of the present disclosure and the first and second possible implementation manners of the fourth aspect of the present disclosure, in a fifth possible implementation manner of the fourth aspect of the present disclosure, the recognition module is configured to determine, according to the frequency of the valid surface electromyography signal, multiple alternative body actions corresponding to the surface electromyography signals of the multiple channels, and perform matching between the amplitude feature of the valid surface electromyography signal and an amplitude feature of the multiple alternative body actions that is obtained by means of pre-training, to obtain a body action matched with the amplitude feature of the valid surface electromyography signal, and use the body action matched with the amplitude feature of the valid surface electromyography signal as the body action corresponding to the surface electromyography signals of the multiple channels.
A fifth aspect of the present disclosure provides an action recognition device based on a surface electromyography signal, including a processor, a memory, and a system bus, where by using the system bus, the processor and the memory are connected and implement mutual communication, the memory is configured to store a computer execution instruction, and the processor is configured to run the computer execution instruction and enable the action recognition device to execute the method according to the first aspect of the present disclosure and the first to third implementation manners of the first aspect of the present disclosure.
A sixth aspect of the present disclosure provides an action recognition device based on a surface electromyography signal, including a processor, a memory, and a system bus, where by using the system bus, the processor and the memory are connected and implement mutual communication, the memory is configured to store a computer execution instruction, and the processor is configured to run the computer execution instruction and enable the action recognition device to execute the method according to the second aspect of the present disclosure and the first to fifth implementation manners of the second aspect of the present disclosure.
According to the action recognition method and device based on a surface electromyography signal provided in the embodiments of the present disclosure, surface electromyography signals of multiple channels are obtained, and a valid surface electromyography signal is determined according to the surface electromyography signals of the multiple channels, then, a frequency of the valid surface electromyography signal is determined, and a body action corresponding to the surface electromyography signals of the multiple channels is determined according to the frequency of the valid surface electromyography signal. A frequency of a surface electromyography signal is irrelevant to signal strength or other features, therefore, the method in the embodiments can significantly improve accuracy of action recognition based on a surface electromyography signal. Moreover, with a frequency being used as a recognition feature, a user does not need to conduct an action with a large range, which brings better user experience. In addition, according to the method in the embodiments of the present disclosure, a body action corresponding to a surface electromyography signal may be further recognized with reference to a frequency and amplitude feature of the surface electromyography signal, which can not only improve accuracy of recognizing a surface electromyography signal, but also can increase recognition types of body actions.

BRIEF DESCRIPTION OF DRAWINGS

To describe the technical solutions in the embodiments of the present disclosure more clearly, the following briefly describes the accompanying drawings required for describing the embodiments. The accompanying drawings in the following description show some embodiments of the present disclosure, and a person of ordinary skill in the art may still derive other drawings from these accompanying drawings without creative efforts.

FIG. 1 is a flowchart of an action recognition method based on a surface electromyography signal according to Embodiment 1 of the present disclosure.

FIG. 2 is a flowchart of a method for determining a frequency of a valid surface electromyography signal according to Embodiment 2 of the present disclosure.

FIG. 3 is a flowchart of an action recognition method based on a surface electromyography signal according to Embodiment 3 of the present disclosure.

FIG. 4 is an overall block diagram of an action recognition method based on a surface electromyography signal according to Embodiment 4 of the present disclosure.

FIG. 5 is a schematic structural diagram of an action recognition device based on a surface electromyography signal according to Embodiment 5 of the present disclosure.

FIG. 6 is a schematic structural diagram of an action recognition device based on a surface electromyography signal according to Embodiment 6 of the present disclosure.

FIG. 7 is a schematic structural diagram of an action recognition device based on a surface electromyography signal according to Embodiment 7 of the present disclosure.

FIG. 8 is a schematic structural diagram of an action recognition device based on a surface electromyography signal according to Embodiment 8 of the present disclosure.

DESCRIPTION OF EMBODIMENTS

To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the following clearly describes the technical solutions in the embodiments of the present disclosure with reference to the accompanying drawings in the embodiments of the present disclosure. The described embodiments are some but not all of the embodiments of the present disclosure. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present disclosure without creative efforts shall fall within the protection scope of the present disclosure.
FIG. 1 is a flowchart of an action recognition method based on a surface electromyography signal according to Embodiment 1 of the present disclosure, and the method in this embodiment may be execute by an action recognition device based on a surface electromyography signal. The action recognition device based on a surface electromyography signal may be specifically a smartphone, a tablet, and another terminal device. As shown in FIG. 1, the method in this embodiment may include the following steps.
Step 101. Obtain surface electromyography signals of multiple channels.
The obtaining, by the action recognition device based on a surface electromyography signal, surface electromyography signals of multiple channels includes receiving the surface electromyography signals, of the multiple channels, sent by a collection apparatus, namely, placing the collection apparatus on a muscle group surface whose signal is to be collected, to collect the surface electromyography signals. In the method of this embodiment, a corresponding body action is determined according to a frequency of a surface electromyography signal. Therefore, a user is required to repeatedly conduct a same body action at a specific frequency when the surface electromyography signals are collected. The body action may be not only a rhythmic gesture of a human forearm, but also a rhythmic action of a leg, or even a rhythmic action of a neck, belly or another body part, and the rhythmic action herein is an action repeated at a frequency.
The collection apparatus includes multiple sensing nodes, data collected by one sensing node is used as a surface electromyography signal of one channel, and the collection apparatus may be built in a smart watch or band or another wearable device. When the user conducts a body action, multiple muscle groups may be driven to act. Therefore, it is necessary to collect surface electromyography signals of the multiple muscle groups, and the collected signals of the multiple channels comprehensively reflect the user body action.
Step 102. Determine a valid surface electromyography signal according to the surface electromyography signals of the multiple channels.
Because a surface electromyography signal has a small amplitude and a low signal-to-noise ratio, the obtained surface electromyography signals of the multiple channels need to be preprocessed before the valid surface electromyography signal is determined, where the preprocessing the surface electromyography signals includes performing signal amplification, power frequency filtering, high-pass filtering, and other processing on the surface electromyography signals.
There may be an interference signal in surface electromyography signals collected by the collection apparatus in the most beginning. For example, the user does not conduct a corresponding body action in time when the collection apparatus starts working. In this case, although the user does not conduct a body action, the collection apparatus can still collect a weak surface electromyography signal. The weak surface electromyography signal is an interference signal. Therefore, it is necessary to remove a possible interference signal to obtain the valid surface electromyography signal.
The valid surface electromyography signal may be determined according to the following method.
Step 1: Overlap the preprocessed surface electromyography signals of the multiple channels, and divide the overlapped surface electromyography signals by a quantity of the channels to obtain a single-channel surface electromyography signal.
Step 2: Slide, starting from a start time of the single-channel surface electromyography signal, the single-channel surface electromyography signal at each sliding moment to obtain a window corresponding to each sliding moment, determine a window sequence corresponding to each sliding moment, and calculate an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, where there is one sliding interval between every two adjacent sliding moments, the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, and the window sequence corresponding to the sliding moment includes a total of N consecutive windows a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, where N is a positive integer greater than or equal to 2.
The start time of the single-channel surface electromyography signal refers to a time when a channel signal starts to exist, the sliding the single-channel surface electromyography signal at each sliding moment is moving the single-channel surface electromyography signal backwards by one window in terms of time at each sliding moment, and a time difference between two adjacent sliding moments is a sliding interval. For example, a window width is a time period t, a sliding interval is t/2, where t=1 millisecond (ms), each window sequence includes four windows, there are a total of N windows, and window serial numbers are respectively 1, 2, 3 . . . N. From the start time of the single-channel surface electromyography signal, a window corresponding to a first sliding moment is window No. 4, a window sequence corresponding to the first sliding moment is 0-4 ms, and there are a total of four windows. At a second sliding moment, a single-channel surface electromyography signal is slid backwards by 0.5 ms, to obtain a second window sequence, the second window sequence is 0.5-4.5 ms, and by analogy, the window sequence corresponding to each sliding moment is obtained. To calculate an average surface electromyography signal amplitude of a window sequence corresponding to the first sliding moment, absolute values of amplitudes of single-channel surface electromyography signals within the window sequence are first obtained, then the absolute values of the single-channel surface electromyography signals within the window sequence are added to obtain an average value, thereby obtaining the average surface electromyography signal amplitude of the window sequence.
Step 3: If an average surface electromyography signal amplitude of a window sequence corresponding to a sliding moment T in all sliding moments is not less than a preset amplitude, use a start time of the window sequence corresponding to the sliding moment T as a start time of the valid surface electromyography signal, add a preset time to the start time of the valid surface electromyography signal to obtain an end time of the valid surface electromyography signal, and capture and use a surface electromyography signal of the multiple channels that is located between the start time and the end time of the valid surface electromyography signal as the valid surface electromyography signal.
When the average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment is obtained by calculation, it is determined whether the average surface electromyography signal amplitude of the window sequence corresponding to the sliding moment is less than the preset amplitude. If the average surface electromyography signal amplitude of the window sequence corresponding to the sliding moment T is not less than the preset amplitude, that is, the average surface electromyography signal amplitude of the window sequence corresponding to the sliding moment T is greater than or equal to the preset amplitude, the start time of the window sequence corresponding to the sliding moment T is used as the start time of the valid surface electromyography signal, and the preset time is added to the start time of the valid surface electromyography signal to obtain the end time of the valid surface electromyography signal. After the start time and the end time of the valid surface electromyography signal is obtained, a surface electromyography signal of the multiple channels that is located between the start time and the end time of the valid surface electromyography signal are captured and used as the valid surface electromyography signal. The preset amplitude may be an average absolute value of amplitudes of the surface electromyography signals obtained by overlapping the surface electromyography signals of the multiple channels.
Step 103. Determine a frequency of the valid surface electromyography signal.
There are multiple methods for determining the frequency of the valid surface electromyography signal, and the following enumerates several commonly used methods. In a first method, a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices is calculated. Assuming that possible body action frequencies are f1, . . . , fn, sine-cosine matrices with f1, . . . , fn as fundamental frequencies are selected. Canonical correlation analysis (CCA) operation is separately performed on the valid surface electromyography signal and each sine-cosine matrix, to obtain a greatest correlation coefficient between the valid surface electromyography signal and the sine-cosine matrices. If the greatest correlation coefficient is greater than a preset correlation coefficient, a fundamental frequency of a sine-cosine matrix corresponding to the greatest correlation coefficient is used as the frequency of the valid surface electromyography signal. In a second method, a Fast Fourier Transform (FFT) is performed on the valid surface electromyography signal, and frequency distribution of the valid surface electromyography signal is determined according to a transform result, thereby obtaining the frequency of the valid surface electromyography signal. In a third method, the frequency of the surface electromyography signal is calculated according to two zero-crossing time intervals of the valid surface electromyography signal.
Step 104. Determine, according to the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.
A correspondence between a frequency of a surface electromyography signal and a body action needs to be pre-established. For example, if a body action corresponding to a 1 (Hertz) HZ frequency is “clenching a fist”, a body action corresponding to a 2 HZ frequency is “bending an elbow”, and a body action corresponding to a 3 HZ frequency is an “OK gesture”, when the user conducts the action of clenching a fist at the 1 HZ frequency, it is determined, according to the obtained frequency of the valid surface electromyography signal, that the body action of the user is clenching a fist.
In this embodiment of the present disclosure, surface electromyography signals of multiple channels are obtained, and a valid surface electromyography signal is determined according to the surface electromyography signals of multiple channels, then, a frequency of the valid surface electromyography signal is determined, and finally, a body action corresponding to the surface electromyography signals of multiple channels is determined according to the frequency of the valid surface electromyography signal. A frequency of a surface electromyography signal is irrelevant to signal strength or other features, therefore, the method in the embodiments can significantly improve accuracy of action recognition based on a surface electromyography signal. Moreover, with a frequency being used as a recognition feature, a user does not need to conduct an action with a large range, which brings better user experience.
In addition, with good anti-interference performance, the method in this embodiment can effectively resist electromyography noise interference generated by an irrelevant action, and the method can be applied when the user is in a non-stationary condition, such as running, driving, or doing housework. The method also provides high stability and is not affected by factors, such as a change of skin moisture, a change of an electrode contact status, and a muscle fatigue degree during a use process. The method provides the user with use convenience, user training data does not need to be pre-collected, and retraining is not required each time before use.
Embodiment 2 of the present disclosure elaborates Step 103 in Embodiment 1 of the present disclosure. FIG. 2 is a flowchart of a method for determining a frequency of a valid surface electromyography signal according to Embodiment 2 of the present disclosure. As shown in FIG. 2, the method in this embodiment includes the following steps.
Step 201. Calculate a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices, where the sine-cosine matrix includes fundamental-frequency and multiplied-frequency sine functions and cosine functions, and all of the sine-cosine matrices have different fundamental frequencies.
Assuming that possible body action frequencies are f1, . . . , fn, sine-cosine matrices with f1, . . . , fn as fundamental frequencies are selected for contrast. CCA operation is separately performed on the valid surface electromyography signal and the sine-cosine matrices, to obtain the correlation coefficients between the valid surface electromyography signal and each sine-cosine matrix.
Valid surface electromyography signals of n channels are recorded as x=(x₁, x₂, x₃, . . . , x_n)^T, where x₁, x₂, x₃, . . . , x_nrespectively represent a signal of each channel, and a sine-cosine matrix is y=[cos(2πf₁t), sin(2πf₁t), cos(4πf₁t), sin(4πf₁t)]^T, where f1 is a fundamental frequency. In this case, the CCA may be defined as the following problem separately finding vectors w_xand w_y, so that a correlation value between projections X=x^Tw_xand Y=y^Tw_ythat are of x and y on the vectors w_xand w_yis the greatest, namely, p in the following formula is the greatest.
$ρ = \frac{E [XY]}{\sqrt{E [X^{2}] E [Y^{2}]}} = \frac{E [w_{x}^{T} {xy}^{T} w_{y}]}{\sqrt{E [w_{x}^{T} {xx}^{T} w_{x}] E [w_{y}^{T} {yy}^{T} w_{y}]}} = \frac{w_{x}^{T} E [{xy}^{T}] w_{y}}{\sqrt{w_{x}^{T} E [{xx}^{T}] w_{x} w_{y}^{T} E [{yy}^{T}] w_{y}}} = \frac{w_{x}^{T} C_{xy} w_{y}}{\sqrt{w_{x}^{T} C_{xx} w_{x} w_{y}^{T} C_{yy} w_{y}}}$
C_xyindicates a cross correlation matrix of x and y, C_xxindicates an autocorrelation matrix of x, and C_yyindicates an autocorrelation matrix of y. A method for finding the vectors w_xand w_yis as follows
Step 1. Construct the following Lagrangian by using the Lagrange algorithm
$L = w_{x}^{T} C_{xy} w_{y} - \frac{λ x}{2} (w_{x}^{T} C_{xx} w_{x} - 1) - \frac{λ y}{2} (w_{y}^{T} C_{yy} w_{y} - 1) .$
Step 2. Respectively seek for partial derivatives of L with respect to w_xand w_y, to obtain
C _xy w _y−λ_x C _xx w _x=0
C _yx w _y−λ_y C _yy w _y=0
In this way, formula 1 and formula 2 are obtained.
$\begin{matrix} w_{y} = \frac{C_{yy}^{- 1} C_{yx} w_{x}}{λ} & (1) \\ λ = λ_{x} = λ_{y}, C_{xx}^{- 1} C_{xy} C_{yy}^{- 1} C_{yx} w_{x} = λ^{2} w_{x} & (2) \end{matrix}$
w_xcan be obtained according to formula 2, and w_ycan be obtained according to formula 1. It can be learned from formula 2 that a process of obtaining w_xaccording to formula 2 is converted into an Eigen value decomposition problem, and w_xand w_yare put into the definition of p to obtain a correlation coefficient.
Step 202. Determine whether a greatest correlation coefficient in the correlation coefficients between the valid surface electromyography signal and the multiple sine-cosine matrices is greater than a preset correlation coefficient.
After correlation coefficients between the valid surface electromyography signal and all sine-cosine matrices are calculated, the greatest correlation coefficient is found out, to determine whether the greatest correlation coefficient is greater than the preset correlation coefficient.
Step 203. If the greatest correlation coefficient is greater than the preset correlation coefficient, use a fundamental frequency of a sine-cosine matrix corresponding to the greatest correlation coefficient as the frequency of the valid surface electromyography signal.
If the greatest correlation coefficient is greater than the preset correlation coefficient, it indicates that the valid surface electromyography signal has the frequency of the sine-cosine matrix corresponding to the greatest correlation coefficient. The fundamental frequency of the sine-cosine matrix corresponding to the greatest correlation coefficient is used as the frequency of the valid surface electromyography signal.
FIG. 3 is a flowchart of an action recognition method based on a surface electromyography signal according to Embodiment 3 of the present disclosure. A difference between the method in this embodiment and the method in Embodiment 1 includes in this embodiment, a body action corresponding to a surface electromyography signal is determined with reference to a frequency and an amplitude feature of the surface electromyography signal.
Step 301. Obtain surface electromyography signals of multiple channels.
Step 302. Determine a valid surface electromyography signal according to the surface electromyography signals of the multiple channels.
The determining a valid surface electromyography signal according to the surface electromyography signals of the multiple channels includes overlapping the surface electromyography signals of the multiple channels, and dividing the overlapped surface electromyography signals by a quantity of the channels to obtain a single-channel surface electromyography signal, sliding, starting from a start time of the single-channel surface electromyography signal, the single-channel surface electromyography signal at each sliding moment to obtain a window corresponding to each sliding moment, determining a window sequence corresponding to each sliding moment, and calculating an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, where there is one sliding interval between every two adjacent sliding moments, the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, and the window sequence corresponding to the sliding moment includes a total of N consecutive windows a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, where N is a positive integer greater than or equal to 2, and when an average surface electromyography signal amplitude of a window sequence corresponding to a sliding moment T in all sliding moments is not less than a preset amplitude, using a start time of the window sequence corresponding to the sliding moment T as a start time of the valid surface electromyography signal, adding a preset time to the start time of the valid surface electromyography signal to obtain an end time of the valid surface electromyography signal, and capturing and using a surface electromyography signal of the multiple channels that is located between the start time and the end time of the valid surface electromyography signal as the valid surface electromyography signal, where the preset amplitude may be an average absolute value of amplitudes of the signals obtained by overlapping the surface electromyography signals of the multiple channels.
Step 303. Determine a frequency of the valid surface electromyography signal.
The determining a frequency of the valid surface electromyography signal includes calculating a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices, where the sine-cosine matrix includes fundamental-frequency and multiplied-frequency sine functions and cosine functions, and all of the sine-cosine matrices have different fundamental frequencies, determining whether a greatest correlation coefficient in the correlation coefficients between the valid surface electromyography signal and the multiple sine-cosine matrices is greater than a preset correlation coefficient, and if the greatest correlation coefficient is greater than the preset correlation coefficient, using a fundamental frequency of a sine-cosine matrix corresponding to the greatest correlation coefficient as the frequency of the valid surface electromyography signal.
For specific implementation manners of steps 301 to 303, reference may be made to related descriptions in Embodiment 1 and Embodiment 2, and details are not described herein.
Step 304. Extract an amplitude feature of the valid surface electromyography signal.
The extracting an amplitude feature of the valid surface electromyography signal includes first, separately performing sliding window processing on surface electromyography signals of each channel of the valid surface electromyography signal to obtain multiple sliding widows, where for example, a sliding window width is 100 ms, and a sliding interval is 100 ms, and then, calculating an average amplitude of each sliding window of the surface electromyography signals of each channel of the valid surface electromyography signal, where the average amplitude of each sliding window is an average absolute value of amplitudes of surface electromyography signals within each sliding window, and using the average amplitude of each sliding window of the valid surface electromyography signal as the amplitude feature of the valid surface electromyography signal.
It should be noted that in this embodiment, there is no execution sequence between step 303 and step 304, and step 304 may be performed before step 303.
Step 305. Determine, according to the amplitude feature of the valid surface electromyography signal and the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.
Multiple alternative body actions corresponding to the surface electromyography signals of the multiple channels are first determined according to the frequency of the valid surface electromyography signal. In this embodiment, each frequency may be corresponding to multiple body actions, for example, a 1 HZ frequency may be separately corresponding to the following three body actions “clenching a fist”, “OK gesture”, and “bending an elbow”. In this case, when it is determined that a frequency of a surface electromyography signal is 1 HZ, the surface electromyography signal is corresponding to the foregoing three alternative body actions. Then, matching is performed between the amplitude feature of the valid surface electromyography signal and an amplitude feature of the multiple alternative body actions that is obtained by means of pre-training, to obtain a body action matched with the amplitude feature of the valid surface electromyography signal, and the body action matched with the amplitude feature of the valid surface electromyography signal is used as the body action corresponding to the surface electromyography signals of the multiple channels. A body action corresponding to a surface electromyography signal may be recognized by using the amplitude feature of the alternative body actions as a template and using a linear discriminant analysis (LDA) classifier.
According to the method in this embodiment, a frequency and an amplitude feature of a surface electromyography signal are obtained, and a body action corresponding to the surface electromyography signal is recognized according to the frequency and amplitude feature of the surface electromyography signal, which not only can improve accuracy of action recognition based on a surface electromyography signal, but also can increase recognition types of surface electromyography signals. Because rhythmic actions, of a same body action, conducted at different frequencies may be regarded as different body actions, in this embodiment, a quantity of recognition types is significantly increased.
When a body action corresponding to a surface electromyography signal is determined with reference to a frequency and amplitude feature of the surface electromyography signal, the surface electromyography signal needs to be pre-trained to obtain amplitude features of various types of body actions. FIG. 4 is an overall block diagram of a recognition method based on a surface electromyography signal according to Embodiment 4 of the present disclosure. According to the recognition method in this embodiment, a recognition process based on a surface electromyography signal is classified into two parts a recognition method based on an amplitude feature and a recognition method based on a frequency.
In the recognition method based on an amplitude feature, a surface electromyography signal first needs to be trained, to obtain a training template. A process of training a surface electromyography signal includes the following steps.
Step 1 Collect surface electromyography signals of multiple channels of various body actions.
A user is required to repeat different body actions (for example, “clenching a fist” and “OK gesture”) for multiple times. Surface electromyography signals of a main muscle group related to a body action are collected by using a collection apparatus, the collected surface electromyography signals include multiple channels, and a time of each collection is recorded.
Step 2 Preprocess the surface electromyography signals of the multiple channels of each body action.
For example, a 50 HZ power frequency interference trap is performed on the collected surface electromyography signals of the multiple channels, and an FIR filter is used to perform high-pass filtering, to obtain preprocessed surface electromyography signals.
Step 3 Extract amplitude features of the surface electromyography signal of the multiple channels of each body action.
For a specific extraction method, reference may be made to a description of step 304 in Embodiment 3, and details are not described herein.
For example, the user repeats an action of “clenching a fist” for 30 times, start times of all collections are [t1, t2, . . . , t30], and collected surface electromyography signals include signals of 8 channels. When amplitude features of the surface electromyography signals are collected, the signals of each channel in the preprocessed surface electromyography signals are slid at a 100 ms sliding window interval successively in a total of 30 time segments (t1, t1+300 ms), (t2, t2+300 ms), . . . , and (t3, t3+300 ms), where a sliding window width is also 100 ms. An average absolute value M(n) of signals of each window in the 30 time segments is calculated, and M(n) is used as the amplitude feature of the action of clenching a fist. Amplitude features of all body actions may be obtained according to the same method.
Step 4 Prepare a training template, that is, establish a correspondence between each body action and an amplitude feature, and then, send the prepared training template to a classifier, so that the classifier recognizes a body action according to the training template.
The foregoing four steps are a training phase, and in a recognition phase, an amplitude feature of a surface electromyography signal also needs to be extracted, and then, the amplitude feature is inputted into a classifier for recognition.
In the frequency-based recognition method, a surface electromyography signal does not need to be trained. In the training phase, at step 1, surface electromyography signals of multiple channels are collected, at step 2, the surface electromyography signals of the multiple channels are preprocessed, at step 3, a valid surface electromyography signal is determined, at step 4, CCA operation is performed, that is, a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices is calculated to obtain a greatest correlation coefficient of the valid surface electromyography signal, and at step 5, a frequency of the valid surface electromyography signal is determined. For a specific implementation manner of the foregoing step 1 to step 5, reference may be made to descriptions of Embodiment 1 and Embodiment 2, and details are not described herein. After the frequency of the valid surface electromyography signal is determined, an alternative body action is determined according to the frequency of the valid surface electromyography signal.
In this embodiment, after a valid surface electromyography signal is determined, an amplitude feature of the valid surface electromyography signal is extracted, the amplitude feature of the valid surface electromyography signal is inputted into a classifier, the classifier uses an amplitude feature of an alternative body action as a template, and feature matching is performed according to the inputted amplitude feature of the valid surface electromyography signal and the amplitude feature of the alternative body action, to obtain a body action corresponding to the valid surface electromyography signal.
The method in each embodiment of the present disclosure is applied to a wide range of scenarios (1) use in an enabling command of a wearable device, for example, to enable an electromyography bracelet by flexing a wrist for multiple times at a specific frequency, thereby avoiding a misoperation, (2) use in a non-stationary state, such as driving or running, for example, for adjusting music volume, switching a song, answering a call, or the like, (3) application to triggering a control instruction due to high recognition accuracy, for example, a handicapped person controls a direction and speed of a wheelchair by using gestures of different rhythms, (4) air mouse, for example, granting finger rotations at different frequencies with corresponding mouse operations, (5) simple game control, for example, two actions fast and slow actions of a finger can be respectively corresponding to acceleration and brake in a racing game, and (6) use in a rehabilitation process to test physical coordination and a control ability of a user.
Matching is performed between the amplitude feature of the valid surface electromyography signal and the amplitude feature of the multiple alternative body actions that is obtained by means of pre-training, to obtain a body action matched with the amplitude feature of the valid surface electromyography signal, and the body action matched with the amplitude feature of the valid surface electromyography signal is used as a body action corresponding to the surface electromyography signals of the multiple channels.
FIG. 5 is a schematic structural diagram of an action recognition device based on a surface electromyography signal according to Embodiment 5 of the present disclosure. As shown in FIG. 5, the action recognition device based on a surface electromyography signal in this embodiment includes an obtaining module 11, a first determining module 12, a second determining module 13, and a recognition module 14.
The obtaining module 11 is configured to obtain surface electromyography signals of multiple channels, the first determining module 12 is configured to determine a valid surface electromyography signal according to the surface electromyography signals of the multiple channels, the second determining module 13 is configured to determine a frequency of the valid surface electromyography signal, and the recognition module 14 is configured to determine, according to the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.
Optionally, the first determining module 12 is configured to overlap the surface electromyography signals of the multiple channels, and divide the overlapped surface electromyography signals by a quantity of the channels to obtain a single-channel surface electromyography signal, slide, starting from a start time of the single-channel surface electromyography signal, the single-channel surface electromyography signal at each sliding moment to obtain a window corresponding to each sliding moment, determine a window sequence corresponding to each sliding moment, and calculate an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, where there is one sliding interval between every two adjacent sliding moments, the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, and the window sequence corresponding to the sliding moment includes a total of N consecutive windows a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, where N is a positive integer greater than or equal to 2, and if an average surface electromyography signal amplitude of a window sequence corresponding to a sliding moment T in all sliding moments is not less than a preset amplitude, use a start time of the window sequence corresponding to the sliding moment T as a start time of the valid surface electromyography signal, add a preset time to the start time of the valid surface electromyography signal to obtain an end time of the valid surface electromyography signal, and capture and use a surface electromyography signal of the multiple channels that is located between the start time and the end time of the valid surface electromyography signal as the valid surface electromyography signal, where the preset amplitude may be an average absolute value of amplitudes of the surface electromyography signals obtained by overlapping the surface electromyography signals of the multiple channels.
Optionally, the second determining module 13 is configured to first, calculate a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices, where the sine-cosine matrix includes fundamental-frequency and multiplied-frequency sine functions and cosine functions, and all of the sine-cosine matrices have different fundamental frequencies, then, determine whether a greatest correlation coefficient in the correlation coefficients between the valid surface electromyography signal and the multiple sine-cosine matrices is greater than a preset correlation coefficient, and if the greatest correlation coefficient is greater than the preset correlation coefficient, use a fundamental frequency of a sine-cosine matrix corresponding to the greatest correlation coefficient as the frequency of the valid surface electromyography signal.
The device in this embodiment of the present disclosure can be configured to execute the solutions in method Embodiment 1 and Embodiment 2. Specific implementation manners and technical effects are similar, and details are not described herein.
FIG. 6 is a schematic structural diagram of an action recognition device based on a surface electromyography signal according to Embodiment 6 of the present disclosure. As shown in FIG. 6, the action recognition device based on a surface electromyography signal provided in this embodiment includes an obtaining module 21, a first determining module 22, a second determining module 23, an extraction module 24, and a recognition module 25.
The obtaining module 21 is configured to obtain surface electromyography signals of multiple channels, the first determining module 22 is configured to determine a valid surface electromyography signal according to the surface electromyography signals of the multiple channels, the second determining module 23 is configured to determine a frequency of the valid surface electromyography signal, the extraction module 24 is configured to extract an amplitude feature of the valid surface electromyography signal, and the recognition module 25 is configured to determine, according to the amplitude feature of the valid surface electromyography signal and the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.
Optionally, the first determining module 22 is configured to overlap the surface electromyography signals of the multiple channels, and divide the overlapped surface electromyography signals by a quantity of the channels to obtain a single-channel surface electromyography signal, slide, starting from a start time of the single-channel surface electromyography signal, the single-channel surface electromyography signal at each sliding moment to obtain a window corresponding to each sliding moment, determine a window sequence corresponding to each sliding moment, and calculate an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, where there is one sliding interval between every two adjacent sliding moments, the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, and the window sequence corresponding to the sliding moment includes a total of N consecutive windows a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, where N is a positive integer greater than or equal to 2, and when an average surface electromyography signal amplitude of a window sequence corresponding to a sliding moment T in all sliding moments is not less than a preset amplitude, use a start time of the window sequence corresponding to the sliding moment T as a start time of the valid surface electromyography signal, adding a preset time to the start time of the valid surface electromyography signal to obtain an end time of the valid surface electromyography signal, and capture and use a surface electromyography signal of the multiple channels that is located between the start time and the end time of the valid surface electromyography signal as the valid surface electromyography signal, where the preset amplitude may be an average absolute value of amplitudes of the surface electromyography signals obtained by overlapping the surface electromyography signals of the multiple channels.
Optionally, the second determining module 23 is configured to first, calculate a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices, where the sine-cosine matrix includes fundamental-frequency and multiplied-frequency sine functions and cosine functions, and all of the sine-cosine matrices have different fundamental frequencies, then, determine whether a greatest correlation coefficient in the correlation coefficients between the valid surface electromyography signal and the multiple sine-cosine matrices is greater than a preset correlation coefficient, and if the greatest correlation coefficient is greater than the preset correlation coefficient, use a fundamental frequency of a sine-cosine matrix corresponding to the greatest correlation coefficient as the frequency of the valid surface electromyography signal.
Optionally, the extraction module 24 is configured to separately perform sliding window processing on surface electromyography signals of each channel of the valid surface electromyography signal, and calculate an average amplitude of each sliding window of the surface electromyography signals of each channel of the valid surface electromyography signal, where the average amplitude of each sliding window is an average absolute value of amplitudes of surface electromyography signals within each sliding window, and use the average amplitude of each sliding window of the valid surface electromyography signal as the amplitude feature of the valid surface electromyography signal.
Optionally, the recognition module 25 is configured to determine, according to the frequency of the valid surface electromyography signal, multiple alternative body actions corresponding to the surface electromyography signals of the multiple channels, and perform matching between the amplitude feature of the valid surface electromyography signal and an amplitude feature of the multiple alternative body actions that is obtained by means of pre-training, to obtain a body action matched with the amplitude feature of the valid surface electromyography signal, and use the body action matched with the amplitude feature of the valid surface electromyography signal as the body action corresponding to the surface electromyography signals of the multiple channels.
The device in this embodiment may be configured to execute the technical solution provided in Embodiment 3. Specific implementation manners and technical effects are similar, and details are not described herein.
FIG. 7 is a schematic structural diagram of an action recognition device based on a surface electromyography signal according to Embodiment 7 of the present disclosure. As shown in FIG. 7, an action recognition device 300 based on a surface electromyography signal in this embodiment includes a processor 31, a memory 32, and a system bus 33. By using the system bus 33, the processor 31 and the memory 32 are connected and implement mutual communication, the memory 32 is configured to store a computer execution instruction 321, the processor 31 is configured to run the computer execution instruction 321, to execute the following method obtaining surface electromyography signals of multiple channels, determining a valid surface electromyography signal according to the surface electromyography signals of the multiple channels, determining a frequency of the valid surface electromyography signal, and determining, according to the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.
Optionally, when determining the valid surface electromyography signal according to the surface electromyography signals of the multiple channels, the processor 31 is configured to overlap the surface electromyography signals of the multiple channels, and divide the overlapped surface electromyography signals by a quantity of the channels to obtain a single-channel surface electromyography signal, slide, starting from a start time of the single-channel surface electromyography signal, the single-channel surface electromyography signal at each sliding moment to obtain a window corresponding to each sliding moment, determine a window sequence corresponding to each sliding moment, and calculate an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, where there is one sliding interval between every two adjacent sliding moments, the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, and the window sequence corresponding to the sliding moment includes a total of N consecutive windows a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, where N is a positive integer greater than or equal to 2, and if an average surface electromyography signal amplitude of a window sequence corresponding to a sliding moment T in all sliding moments is not less than a preset amplitude, use a start time of the window sequence corresponding to the sliding moment T as a start time of the valid surface electromyography signal, add a preset time to the start time of the valid surface electromyography signal to obtain an end time of the valid surface electromyography signal, and capture and use a surface electromyography signal of the multiple channels that is located between the start time and the end time of the valid surface electromyography signal as the valid surface electromyography signal, where the preset amplitude may be an average absolute value of amplitudes of the surface electromyography signals obtained by overlapping the surface electromyography signals of the multiple channels.
Optionally, when determining the frequency of the valid surface electromyography signal, the processor 31 is configured to calculate a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices, where the sine-cosine matrix includes fundamental-frequency and multiplied-frequency sine functions and cosine functions, and all of the sine-cosine matrices have different fundamental frequencies, determine whether a greatest correlation coefficient in the correlation coefficients between the valid surface electromyography signal and the multiple sine-cosine matrices is greater than a preset correlation coefficient, and if the greatest correlation coefficient is greater than the preset correlation coefficient, use a fundamental frequency of a sine-cosine matrix corresponding to the greatest correlation coefficient as the frequency of the valid surface electromyography signal.
The device in this embodiment may be configured to execute the technical solutions in Embodiment 1 and Embodiment 2. Specific implementation manners and technical effects are similar, and details are not described herein.
FIG. 8 is a schematic structural diagram of an action recognition device based on a surface electromyography signal according to Embodiment 8 of the present disclosure. As shown in FIG, 8, an action recognition device 400 based on a surface electromyography signal in this embodiment includes a processor 41, a memory 42, and a system bus 43. By using the system bus 43, the processor 41 and the memory 42 are connected and implement mutual communication, the memory 42 is configured to store a computer execution instruction 421, the processor 41 is configured to run the computer execution instruction 421, to execute the following method obtaining surface electromyography signals of multiple channels, determining a valid surface electromyography signal according to the surface electromyography signals of the multiple channels, determining a frequency of the valid surface electromyography signal, extracting an amplitude feature of the valid surface electromyography signal, and determining, according to the amplitude feature of the valid surface electromyography signal and the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.
Optionally, when determining the valid surface electromyography signal according to the surface electromyography signals of the multiple channels, the processor 41 is configured to overlap the surface electromyography signals of the multiple channels, and divide the overlapped surface electromyography signals by a quantity of the channels to obtain a single-channel surface electromyography signal, slide, starting from a start time of the single-channel surface electromyography signal, the single-channel surface electromyography signal at each sliding moment to obtain a window corresponding to each sliding moment, determine a window sequence corresponding to each sliding moment, and calculate an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, where there is one sliding interval between every two adjacent sliding moments, the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, and the window sequence corresponding to the sliding moment includes a total of N consecutive windows a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, where N is a positive integer greater than or equal to 2, and when an average surface electromyography signal amplitude of a window sequence corresponding to a sliding moment T in all sliding moments is not less than a preset amplitude, use a start time of the window sequence corresponding to the sliding moment T as a start time of the valid surface electromyography signal, adding a preset time to the start time of the valid surface electromyography signal to obtain an end time of the valid surface electromyography signal, and capture and use a surface electromyography signal of the multiple channels that is located between the start time and the end time of the valid surface electromyography signal as the valid surface electromyography signal, where the preset amplitude may be an average absolute value of amplitudes of the surface electromyography signals obtained by overlapping the surface electromyography signals of the multiple channels.
Optionally, when determining the frequency of the valid surface electromyography signal, the processor 41 is configured to calculate a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices, where the sine-cosine matrix includes fundamental-frequency and multiplied-frequency sine functions and cosine functions, and all of the sine-cosine matrices have different fundamental frequencies, determine whether a greatest correlation coefficient in the correlation coefficients between the valid surface electromyography signal and the multiple sine-cosine matrices is greater than a preset correlation coefficient, and if the greatest correlation coefficient is greater than the preset correlation coefficient, use a fundamental frequency of a sine-cosine matrix corresponding to the greatest correlation coefficient as the frequency of the valid surface electromyography signal.
Optionally, when extracting the amplitude feature of the valid surface electromyography signal, the processor 41 is configured to separately perform sliding window processing on surface electromyography signals of each channel of the valid surface electromyography signal, and calculate an average amplitude of each sliding window of the surface electromyography signals of each channel of the valid surface electromyography signal, where the average amplitude of each sliding window is an average absolute value of amplitudes of surface electromyography signals within each sliding window, and use the average amplitude of each sliding window of the valid surface electromyography signal as the amplitude feature of the valid surface electromyography signal.
Optionally, when determining, according to the amplitude feature of the valid surface electromyography signal and the frequency of the valid surface electromyography signal, the body action corresponding to the surface electromyography signals of the multiple channels, the processor 41 is configured to determine, according to the frequency of the valid surface electromyography signal, multiple alternative body actions corresponding to the surface electromyography signals of the multiple channels, and perform matching between the amplitude feature of the valid surface electromyography signal and an amplitude feature of the multiple alternative body actions that is obtained by means of pre-training, to obtain a body action matched with the amplitude feature of the valid surface electromyography signal, and use the body action matched with the amplitude feature of the valid surface electromyography signal as the body action corresponding to the surface electromyography signals of the multiple channels.
The device in this embodiment may be configured to execute the technical solution in Embodiment 3. Specific implementation manners and technical effects are similar, and details are not described herein.
Persons of ordinary skill in the art may understand that all or some of the steps of the method embodiments may be implemented by a program instructing relevant hardware. The program may be stored in a computer-readable storage medium. When the program runs, the steps of the method embodiments are performed. The foregoing storage medium includes any medium that can store program code, such as a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disc.
Finally, it should be noted that the foregoing embodiments are merely intended for describing the technical solutions of the present disclosure, but not for limiting the present disclosure. Although the present disclosure is described in detail with reference to the foregoing embodiments, persons of ordinary skill in the art should understand that they may still make modifications to the technical solutions described in the foregoing embodiments or make equivalent replacements to some or all technical features thereof, without departing from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

What is claimed is:

1. An action recognition method based on a surface electromyography signal, comprising:

obtaining surface electromyography signals of multiple channels;

determining a valid surface electromyography signal according to the surface electromyography signals of the multiple channels;

determining a frequency of the valid surface electromyography signal; and

determining, according to the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.

2. The method according to claim 1, wherein determining the valid surface electromyography signal according to the surface electromyography signals of the multiple channels comprises:

overlapping the surface electromyography signals of the multiple channels;

dividing the overlapped surface electromyography signals by a quantity of the channels to obtain a single-channel surface electromyography signal;

sliding, starting from a start time of the single-channel surface electromyography signal, the single-channel surface electromyography signal at each sliding moment to obtain a window corresponding to each sliding moment;

determining a window sequence corresponding to each sliding moment;

calculating an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, wherein there is one sliding interval between every two adjacent sliding moments, wherein the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, wherein the window sequence corresponding to the sliding moment comprises a total of N consecutive windows, wherein the N consecutive windows comprise a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, and wherein N is a positive integer greater than or equal to 2;

using a start time of a window sequence corresponding to a sliding moment T as a start time of the valid surface electromyography signal;

adding a preset time to the start time of the valid surface electromyography signal to obtain an end time of the valid surface electromyography signal; and

capturing and using a surface electromyography signal of the multiple channels that is located between the start time and the end time of the valid surface electromyography signal as the valid surface electromyography signal when an average surface electromyography signal amplitude of the window sequence corresponding to the sliding moment T in all sliding moments is not less than a preset amplitude.

3. The method according to claim 2, wherein the preset amplitude is an average absolute value of amplitudes of the surface electromyography signals obtained by overlapping the surface electromyography signals of the multiple channels.

4. The method according to claim 1, wherein the determining the frequency of the valid surface electromyography signal comprises:

calculating a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices, wherein the sine-cosine matrix comprises fundamental-frequency and multiplied-frequency sine functions and cosine functions, and wherein all of the sine-cosine matrices have different fundamental frequencies;

determining whether a greatest correlation coefficient in the correlation coefficients between the valid surface electromyography signal and the multiple sine-cosine matrices is greater than a preset correlation coefficient; and

using a fundamental frequency of a sine-cosine matrix corresponding to the greatest correlation coefficient as the frequency of the valid surface electromyography signal when the greatest correlation coefficient is greater than the preset correlation coefficient.

5. An action recognition method based on a surface electromyography signal, comprising:

obtaining surface electromyography signals of multiple channels;

determining a frequency of the valid surface electromyography signal;

extracting an amplitude feature of the valid surface electromyography signal; and

determining, according to the amplitude feature of the valid surface electromyography signal and the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.

6. The method according to claim 5, wherein determining the valid surface electromyography signal according to the surface electromyography signals of the multiple channels comprises:

overlapping the surface electromyography signals of the multiple channels;

determining a window sequence corresponding to each sliding moment;

calculating an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, wherein there is one sliding interval between every two adjacent sliding moments, wherein the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, wherein the window sequence corresponding to the sliding moment comprises a total of N consecutive windows, wherein the N consecutive windows comprise a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, and wherein N is a positive integer greater than or equal to 2; and

capturing and using a surface electromyography signal of the multiple channel that is located between the start time and the end time of the valid surface electromyography signal as the valid surface electromyography signal when an average surface electromyography signal amplitude of the window sequence corresponding to the sliding moment T in all sliding moments is not less than a preset amplitude.

7. The method according to claim 6, wherein the preset amplitude is an average absolute value of amplitudes of the surface electromyography signals obtained by overlapping the surface electromyography signals of the multiple channels.

8. The method according to claim 5, wherein determining the frequency of the valid surface electromyography signal comprises:

9. The method according to claim 8, wherein extracting the amplitude feature of the valid surface electromyography signal comprises:

separately performing sliding window processing on surface electromyography signals of each channel of the valid surface electromyography signal;

calculating an average amplitude of each sliding window of the surface electromyography signals of each channel of the valid surface electromyography signal, wherein the average amplitude of each sliding window is an average absolute value of amplitudes of surface electromyography signals within each sliding window; and

using the average amplitude of each sliding window of the valid surface electromyography signal as the amplitude feature of the valid surface electromyography signal.

10. The method according to claim 5, wherein determining, according to the amplitude feature of the valid surface electromyography signal and the frequency of the valid surface electromyography signal, the body action corresponding to the surface electromyography signals of the multiple channels comprises:

determining, according to the frequency of the valid surface electromyography signal, multiple alternative body actions corresponding to the surface electromyography signals of the multiple channels;

performing matching between the amplitude feature of the valid surface electromyography signal and an amplitude feature of the multiple alternative body actions that is obtained by a pre-training to obtain the body action matched with the amplitude feature of the valid surface electromyography signal; and

using the body action matched with the amplitude feature of the valid surface electromyography signal as the body action corresponding to the surface electromyography signals of the multiple channels.

11. An action recognition device based on a surface electromyography signal, comprising:

a processor;

a memory; and

a system bus,

wherein by using the system bus, the processor and the memory are connected and implement mutual communication,

wherein the memory is configured to store a computer execution instruction, and

wherein the processor is configured to run the computer execution instruction to:

obtain surface electromyography signals of multiple channels;

determine a valid surface electromyography signal according to the surface electromyography signals of the multiple channels;

determine a frequency of the valid surface electromyography signal; and

determine, according to the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.

12. The device according to claim 11, wherein the processor is further configured to:

overlap the surface electromyography signals of the multiple channels;

divide the overlapped surface electromyography signals by a quantity of the channels to obtain a single-channel surface electromyography signal;

slide, starting from a start time of the single-channel surface electromyography signal, the single-channel surface electromyography signal at each sliding moment to obtain a window corresponding to each sliding moment;

determine a window sequence corresponding to each sliding moment;

calculate an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, wherein there is one sliding interval between every two adjacent sliding moments, wherein the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, wherein the window sequence corresponding to the sliding moment comprises a total of N consecutive windows, wherein the total of N consecutive windows comprise a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, wherein N is a positive integer greater than or equal to 2;

use a start time of the window sequence corresponding to the sliding moment T as a start time of the valid surface electromyography signal when an average surface electromyography signal amplitude of a window sequence corresponding to a sliding moment T in all sliding moments is not less than a preset amplitude;

add a preset time to the start time of the valid surface electromyography signal to obtain an end time of the valid surface electromyography signal; and

capture and use a surface electromyography signal of the multiple channels that is located between the start time and the end time of the valid surface electromyography signal as the valid surface electromyography signal.

13. The device according to claim 12, wherein the preset amplitude is an average absolute value of amplitudes of the surface electromyography signals obtained by overlapping the surface electromyography signals of the multiple channels.

14. The device according to claim 11, wherein the processor is further configured to:

calculate a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices, wherein the sine-cosine matrix comprises fundamental-frequency and multiplied-frequency sine functions and cosine functions, and all of the sine-cosine matrices have different fundamental frequencies;

determine whether a greatest correlation coefficient in the correlation coefficients between the valid surface electromyography signal and the multiple sine-cosine matrices is greater than a preset correlation coefficient; and

use a fundamental frequency of a sine-cosine matrix corresponding to the greatest correlation coefficient as the frequency of the valid surface electromyography signal when the greatest correlation coefficient is greater than the preset correlation coefficient.

15. An action recognition device based on a surface electromyography signal, comprising:

a processor;

a memory; and

a system bus,

wherein the memory is configured to store a computer execution instruction, and

obtain surface electromyography signals of multiple channels;

determine a frequency of the valid surface electromyography signal;

extract an amplitude feature of the valid surface electromyography signal; and

determine, according to the amplitude feature of the valid surface electromyography signal and the frequency of the valid surface electromyography signal, a body action corresponding to the surface electromyography signals of the multiple channels.

16. The device according to claim 15, wherein the processor is further configured to:

overlap the surface electromyography signals of the multiple channels;

determine a window sequence corresponding to each sliding moment;

calculate an average surface electromyography signal amplitude of the window sequence corresponding to each sliding moment, wherein there is one sliding interval between every two adjacent sliding moments, wherein the average surface electromyography signal amplitude of the window sequence is an average absolute value of amplitudes of surface electromyography signals within the window sequence, wherein the window sequence corresponding to the sliding moment comprises a total of N consecutive windows, wherein the N consecutive windows comprise a window corresponding to the sliding moment and N−1 windows corresponding to N−1 sliding moments before the sliding moment, and wherein N is a positive integer greater than or equal to 2;

17. The device according to claim 16, wherein the preset amplitude is an average absolute value of amplitudes of the surface electromyography signals obtained by overlapping the surface electromyography signals of the multiple channels.

18. The device according to claim 15, wherein the processor is configured to:

calculate a correlation coefficient between the valid surface electromyography signal and each of multiple sine-cosine matrices, wherein the sine-cosine matrix comprises fundamental-frequency and multiplied-frequency sine functions and cosine functions, and wherein all of the sine-cosine matrices have different fundamental frequencies;

19. The device according to claim 18, wherein the processor is further configured to:

separately perform sliding window processing on surface electromyography signals of each channel of the valid surface electromyography signal;

calculate an average amplitude of each sliding window of the surface electromyography signals of each channel of the valid surface electromyography signal, wherein the average amplitude of each sliding window is an average absolute value of amplitudes of surface electromyography signals within each sliding window;

use the average amplitude of each sliding window of the valid surface electromyography signal as the amplitude feature of the valid surface electromyography signal.

20. The device according to claim 15, wherein the processor is configured to:

determine, according to the frequency of the valid surface electromyography signal, multiple alternative body actions corresponding to the surface electromyography signals of the multiple channels;

perform matching between the amplitude feature of the valid surface electromyography signal and an amplitude feature of the multiple alternative body actions that is obtained by a pre-training to obtain a body action matched with the amplitude feature of the valid surface electromyography signal;

use the body action matched with the amplitude feature of the valid surface electromyography signal as the body action corresponding to the surface electromyography signals of the multiple channels.