[go: up one dir, main page]

CN107506041A - A kind of wearable mouse control method based on motion sensor - Google Patents

A kind of wearable mouse control method based on motion sensor Download PDF

Info

Publication number
CN107506041A
CN107506041A CN201710812802.6A CN201710812802A CN107506041A CN 107506041 A CN107506041 A CN 107506041A CN 201710812802 A CN201710812802 A CN 201710812802A CN 107506041 A CN107506041 A CN 107506041A
Authority
CN
China
Prior art keywords
action
data
cursor
head
gyroscope
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201710812802.6A
Other languages
Chinese (zh)
Inventor
王志波
金博楠
李熠劼
龚银超
庞晓艺
王骞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan University WHU
Original Assignee
Wuhan University WHU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan University WHU filed Critical Wuhan University WHU
Priority to CN201710812802.6A priority Critical patent/CN107506041A/en
Publication of CN107506041A publication Critical patent/CN107506041A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012Head tracking input arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Position Input By Displaying (AREA)

Abstract

本发明公开了一种基于运动传感器的可穿戴式鼠标控制方法,通过数据采集、动作定义、数据预处理、特征提取、动作分类、动作分割、数据与训练个错误矫正等八步骤,利用从可穿戴式设备中的传感器传送来的头部动作运动数据,在确保不影响正常视物和操作简便的同时,实现了使用头部动作对鼠标的高精度控制。相较于市面上已经出现的眼控类、脚控类和呼气控制类鼠标,本发明操作更简便,识别更精准。另外,本发明应用广泛,不仅可以方便无臂人士使用,也可以帮助一些正常人解放双手。本发明具有普遍的适用性,不仅可以用于控制电脑,还可以控制其他任何电子设备。

The invention discloses a wearable mouse control method based on a motion sensor, through eight steps of data collection, action definition, data preprocessing, feature extraction, action classification, action segmentation, data and training error correction, etc. The head motion motion data transmitted by the sensor in the wearable device, while ensuring that it does not affect normal vision and easy operation, realizes high-precision control of the mouse using head motion. Compared with the eye-controlled, foot-controlled and exhalation-controlled mice that have appeared on the market, the present invention is easier to operate and more accurate to recognize. In addition, the invention has wide application, not only can be used conveniently by armless people, but also can help some normal people to free their hands. The invention has universal applicability and can be used not only to control computers, but also to control any other electronic equipment.

Description

一种基于运动传感器的可穿戴式鼠标控制方法A wearable mouse control method based on motion sensor

技术领域technical field

本发明属于移动感知计算领域,具体涉及一种基于运动传感器的可穿戴式鼠标控制方法。相较于市面上已经出现的眼控类、脚控类和呼气控制类鼠标,本发明操作更简便,识别更精准。另外,本发明应用广泛,不仅可以方便无臂人士使用,也可以帮助一些正常人解放双手。本发明具有普遍的适用性,不仅可以用于控制电脑,还可以控制其他任何电子设备。The invention belongs to the field of mobile perception computing, and in particular relates to a wearable mouse control method based on a motion sensor. Compared with the eye-controlled, foot-controlled and exhalation-controlled mice that have appeared on the market, the present invention is easier to operate and more accurate to recognize. In addition, the invention has wide application, not only can be used conveniently by armless people, but also can help some normal people to free their hands. The invention has universal applicability and can be used not only to control computers, but also to control any other electronic equipment.

背景技术Background technique

随着互联网技术的发展,在人们的生活中电脑、智能手机等设备越来越普及,鼠标也成为最常见的人机交互系统。但是,无臂人士和一些手臂残疾人士在生活中却无法使用鼠标、触摸屏来操纵设备。与此同时,可穿戴式设备在当今也越来越普及,例如智能手表、智能手环等。传感器如今也越来越多样化并具有相当高的可用性:加速计、陀螺仪等设备可以提供设备精准的运动信息。市场上已经出现了一些使用穿戴式设备,如通过眼球的移动来控制鼠标的眼控类,通过脚趾代替手指来对鼠标进行控制的脚控类,和通过吸气呼气的时间长短和强弱的不同组合来实现上下左右和点击等鼠标功能的呼气控制类,但它们都存在有操作不便、控制困难的不足之处。而使用头部动作来进行控制能做到操作简便、识别准确,可以解决上述问题。With the development of Internet technology, computers, smart phones and other devices are becoming more and more popular in people's lives, and the mouse has become the most common human-computer interaction system. However, armless people and some arm disabled people cannot use a mouse or a touch screen to manipulate devices in their lives. At the same time, wearable devices, such as smart watches and smart bracelets, are becoming more and more popular today. Sensors are also increasingly diverse and highly available: devices such as accelerometers and gyroscopes can provide precise information about the movement of a device. Some wearable devices have appeared on the market, such as eye-controlled devices that control the mouse through the movement of the eyeballs, foot-controlled devices that use the toes instead of fingers to control the mouse, and the length and strength of inhalation and exhalation. different combinations to realize the exhalation control class of mouse functions such as up, down, left, and right, and clicking, but they all have the disadvantages of inconvenient operation and difficult control. The use of head movements for control can achieve easy operation and accurate recognition, which can solve the above problems.

本发明致力于利用从可穿戴式设备中的传感器传送来的头部动作运动数据,研究和探索一种基于运动传感器的可穿戴式鼠标控制方法。The present invention is committed to researching and exploring a wearable mouse control method based on a motion sensor by using the head movement data transmitted from the sensor in the wearable device.

动作识别的准确度与传感器数据的采样率有关,若采样率过高,会导致传输成本太大,且容易造成拥塞,使数据传输造成相当大的延迟,且容易丢包。若采样率过低,则会造成动作特征不明显、识别不精确的结果。因此,为了保证数据正常传输且动作识别可以足够精确,需要对其采样率进行选择。The accuracy of action recognition is related to the sampling rate of sensor data. If the sampling rate is too high, the transmission cost will be too high, and it will easily cause congestion, causing considerable delay in data transmission and easy packet loss. If the sampling rate is too low, it will result in indistinct motion features and imprecise recognition results. Therefore, in order to ensure normal data transmission and accurate enough action recognition, it is necessary to select its sampling rate.

传感器数据的传输通过蓝牙或者wifi进行。在使用过程中,会出现头部不自然抖动、网络情况不理想等情况。所以需要对数据的处理和传输的方式进行精细的设计。在实际应用中,噪声波干扰、数据传输的缺失、附加信号的不可感知性与信息传递的鲁棒性之间的权衡等因素,都需要加以考量。The transmission of sensor data is carried out via bluetooth or wifi. During use, unnatural shaking of the head and unsatisfactory network conditions may occur. Therefore, the way of data processing and transmission needs to be carefully designed. In practical applications, factors such as noise wave interference, lack of data transmission, and the trade-off between the imperceptibility of additional signals and the robustness of information transmission need to be considered.

发明内容Contents of the invention

为了解决上述技术问题,本发明提供了一种基于运动传感器的可穿戴式鼠标控制方法。In order to solve the above technical problems, the present invention provides a wearable mouse control method based on a motion sensor.

本发明所采用的技术方案是:The technical scheme adopted in the present invention is:

一种基于运动传感器的可穿戴式鼠标控制方法,其特征在于,包括以下步骤:A kind of wearable mouse control method based on motion sensor, is characterized in that, comprises the following steps:

步骤1:进行数据采集,具体包括:Step 1: Conduct data collection, including:

步骤1.1:选择蓝牙或Wifi热点连接嵌入式穿戴式设备和被控制的电脑,建立Socket连接来传递数据。端口设置为8088。Step 1.1: Select a Bluetooth or Wifi hotspot to connect the embedded wearable device and the controlled computer, and establish a Socket connection to transfer data. The port is set to 8088.

步骤1.2:将从加速计和陀螺仪得到的三个方向的运动数据分别依次存储。当接收到的数据数大于窗口长度时则去除最先的数据。采样率设置为每秒40次。Step 1.2: Store the motion data in three directions obtained from the accelerometer and gyroscope in sequence. When the number of received data is greater than the window length, the first data will be removed. The sampling rate is set to 40 times per second.

步骤2:动作定义,具体包括以下子步骤:Step 2: Action definition, specifically including the following sub-steps:

步骤2.1:定义使用者一开始正对着屏幕,将初始的头部所在位置成为中间区域。Step 2.1: Define that the user is facing the screen at the beginning, and the initial head position becomes the middle area.

步骤2.2:定义单击操作。将头部快速向下,然后快速返回到中间区域的动作定义为单击操作。Step 2.2: Define the click action. The action of snapping the head down and then snapping back to the middle area is defined as a click operation.

步骤2.3:定义方向动作。将头部从中间区域转向左,保持一段时间至光标达到期望位置后转回中间区域定义为左移动作。将头部从中间区域转向右,保持一段时间至光标达到期望位置后转回中间区域定义为右移动作。将头部从中间区域转向上,保持一段时间至光标达到期望位置后转回中间区域定义为上移动作。Step 2.3: Define the direction action. Turning the head from the middle area to the left, keeping it for a while until the cursor reaches the desired position, and then turning back to the middle area is defined as a left movement. Turning the head from the middle area to the right, keeping it for a while until the cursor reaches the desired position, and then turning back to the middle area is defined as a right movement. Turning the head upward from the middle area, keeping it for a while until the cursor reaches the desired position, and then turning back to the middle area is defined as an up movement.

将头部从中间区域转向下,保持一段时间至光标达到期望位置后转回中间区域定义为下移动作。Turning the head down from the middle area, keeping it for a while until the cursor reaches the desired position, and then turning back to the middle area is defined as a down movement.

步骤3:消除噪声的去除齿波的数据预处理;Step 3: data preprocessing for eliminating noise and removing tooth waves;

步骤4:特征提取,具体包括以下子步骤:Step 4: feature extraction, specifically including the following sub-steps:

步骤4.1:因在加速计的x轴方向,单击操作比另外四个方向的动作特征更为显著,所以可以使用加速计x轴方向的值来将单击动作和方向动作区分开来。Step 4.1: Because in the x-axis direction of the accelerometer, the click action is more prominent than the other four directions, so the value of the x-axis direction of the accelerometer can be used to distinguish the click action from the direction action.

步骤4.2:因在陀螺仪的x轴方向,左右移动比上下移动动作特征更为显著,所以可以使用陀螺仪x轴方向的值来将左右动作和上下动作区分开来。又因左右动作特征正好相反,故也可将左右动作区分开来。Step 4.2: Because in the x-axis direction of the gyroscope, the left-right movement is more characteristic than the up-down movement, so the value in the x-axis direction of the gyroscope can be used to distinguish the left-right movement from the up-down movement. And because the characteristics of the left and right movements are just opposite, it is also possible to distinguish the left and right movements.

步骤4.3:因在陀螺仪的z轴方向,上下移动比左右移动动作特征更为显著,所以可以使用陀螺仪z轴方向的值来将上下动作和左右动作区分开来。又因左右动作特征正好相反,故也可将上下动作区分开来。Step 4.3: Because in the z-axis direction of the gyroscope, the up and down motion is more significant than the left and right motion, so the value in the z-axis direction of the gyroscope can be used to distinguish the up and down motion from the left and right motion. And because the characteristics of the left and right movements are just opposite, it is also possible to distinguish the up and down movements.

步骤5:动作分类,具体实现包括以下子步骤:Step 5: action classification, the specific implementation includes the following sub-steps:

步骤5.1:定义atn x,gtn x,gtx x,gtn z,gtx z分别为加速计x轴方向的下阈值、陀螺仪x、z轴方向的上下阈值。阈值的值由预训练步骤决定,作为决策树的判定数据。定义ai x,gi x,gi z为当前时间i的加速计x轴方向、陀螺仪x轴方向、陀螺仪z轴方向的值。Step 5.1: define a tn x , g tn x , g tx x , g tn z , and g tx z as the lower threshold of the accelerometer in the x-axis direction and the upper and lower thresholds of the gyroscope in the x-axis and z-axis directions, respectively. The value of the threshold is determined by the pre-training step as the decision data of the decision tree. Define a i x , g i x , and g i z as the values of the accelerometer x-axis direction, gyroscope x-axis direction, and gyroscope z-axis direction at the current time i.

检查当前时间加速计x轴方向的值,如果ai x<atn x,则头部开始点击动作,否则,进入步骤5.2。Check the value of the x-axis direction of the accelerometer at the current time, if a i x <a tn x , the head starts to click, otherwise, go to step 5.2.

步骤5.2:检查当前时间陀螺仪x轴方向的值,如果gi x<gtn x或者gi x>gtx x,头部则开始左或右动作,进入步骤5.3。否则,进入步骤5.4。Step 5.2: Check the value of the x-axis direction of the gyroscope at the current time. If g i x <g tn x or g i x >g tx x , the head will start to move left or right, and go to step 5.3. Otherwise, go to step 5.4.

步骤5.3:如果gi x<gtn x,则头部开始向左移动,屏幕上的光标向左移动。否则,头部向右移动,屏幕上的光标也向右移。Step 5.3: If g i x <g tn x , the head starts to move to the left and the cursor on the screen moves to the left. Otherwise, the head moves to the right, and the cursor on the screen moves to the right.

步骤5.4:检查当前时间陀螺仪z轴方向的值,如果gi z<gtn z或者gi z>gtx z,头部则开始上或下动作,进入步骤5.5。否则,在当前时间没有动作发生。Step 5.4: Check the value of the z-axis direction of the gyroscope at the current time. If g i z <g tn z or g i z >g tx z , the head will start to move up or down, and go to step 5.5. Otherwise, no action occurs at the current time.

步骤5.5:如果gi z<gtn z,则头部开始向上移动,屏幕上的光标向上移动。否则,头部开始向下移动,屏幕上的光标也向右移。Step 5.5: If g i z <g tn z , the head starts to move up and the cursor on the screen moves up. Otherwise, the head starts moving down and the on-screen cursor moves to the right.

步骤5.6:一旦动作被识别了,光标便会在屏幕上执行相应动作直至动作的终止,Step 5.6: Once the action is recognized, the cursor will execute the corresponding action on the screen until the end of the action,

步骤6:动作分割,得到的原始数据是时间连续数据,动作分割将这些时间连续的数据分割成块,每一块包含的数据代表一个动作。依据动作的定义,每一个动作都包含两个相反的部分。例如,向左动作首先将头部从中间区域转向左,随后再由左转回中间区域。结果是,在同一方向上有相反的两个波形。定义在一个方向上有两个阈值Tn,Tp来决定动作的开始和结束。其中,Tn是下阈值,Tp是上阈值。Step 6: Action segmentation. The obtained original data is time-continuous data. Action segmentation divides these time-continuous data into blocks, and the data contained in each block represents an action. According to the definition of action, each action contains two opposite parts. For example, a leftward movement first turns the head from the middle area to the left, and then from the left back to the middle area. The result is that there are two opposite waveforms in the same direction. Define two thresholds T n and T p in one direction to determine the start and end of the action. Wherein, T n is the lower threshold and T p is the upper threshold.

步骤7:数据预训练,具体实现包括以下子步骤:Step 7: Data pre-training, the specific implementation includes the following sub-steps:

步骤7.1:用户在使用之前依次做5个定义的动作5次。Step 7.1: The user does 5 defined actions sequentially 5 times before use.

步骤7.2:提取得到的数据的5对极值,将得到的最低波峰和最高波谷乘以某一系数k作为训练出的个性化阈值。Step 7.2: Extract 5 pairs of extreme values of the obtained data, and multiply the obtained lowest peak and highest trough by a certain coefficient k as the trained personalized threshold.

步骤8:错误矫正,当用户使用时,用户可以看见屏幕上光标的实时移动。当错误发生时,用户可以知道发生了一个错误,并可以立即开始矫正。如果光标向错误的方向移动,用户可以执行同样的动作,然后进行接触。例如,当用户向左时,发现光标向右移动,用户可以也执行一次向右动作来矫正这个错误。用这个方法,当前一个动作结束了,后续动作不会受之前的错误影响。Step 8: Error correction, when the user uses it, the user can see the real-time movement of the cursor on the screen. When an error occurs, the user can know that an error has occurred and can immediately start correcting it. If the cursor moves in the wrong direction, the user can perform the same action and then make a contact. For example, when the user moves to the left and finds that the cursor moves to the right, the user can also perform a right action to correct this error. In this way, the previous action is finished, and the subsequent action will not be affected by the previous error.

在上述的基于运动传感器的可穿戴式鼠标控制方法,步骤3的具体实现包括以下子步骤:In the above-mentioned wearable mouse control method based on the motion sensor, the specific realization of step 3 includes the following sub-steps:

步骤3.1:使用阈值去噪的方式消除噪声。定义一个噪声阈值Eth,当接收到的数据大于该阈值时才向设备传输数据,否则置为0。Step 3.1: Use threshold denoising to remove noise. Define a noise threshold E th , and only transmit data to the device when the received data is greater than the threshold, otherwise it is set to 0.

步骤3.2:使用最大值滤波的方式去除齿波。最大值滤波即把窗口内所有值的最大值作为当前滤波后的值。定义滤波窗口为τ,则当前时间滤波后的值rt可表示为:Step 3.2: Use the maximum value filter to remove the tooth wave. Maximum value filtering is to use the maximum value of all values in the window as the current filtered value. Define the filtering window as τ, then the filtered value r t at the current time can be expressed as:

rt=max{rt-τ/2,...,rt-1,rt,rt+1,...,rt+τ/2}r t =max{r t-τ/2 ,...,r t-1 ,r t ,r t+1 ,...,r t+τ/2 }

在上述的基于运动传感器的可穿戴式鼠标控制方法,步骤5.6中动作的终止定义如下:In the above-mentioned wearable mouse control method based on the motion sensor, the termination of the action in step 5.6 is defined as follows:

对于左动作,当实时数据gi x≥gtx x,光标停止移动,随后识别到实时数据gi x≤gtn x,左动作终止。For the left action, when the real-time data g i x ≥ g tx x , the cursor stops moving, and then the real-time data g i x ≤ g tn x is recognized, and the left action is terminated.

对于右动作,当实时数据ai x≤atn x,光标停止移动,随后识别到实时数据ai x≥atn x,右动作终止。For the right action, when the real-time data a i x ≤ a tn x , the cursor stops moving, and then the real-time data a i x ≥ a tn x is recognized, and the right action is terminated.

对于上动作,当实时数据gi z≥gtx x,光标停止移动,随后识别到实时数据gi z≤gtx z,上动作终止。For the up action, when the real-time data g i z ≥ g tx x , the cursor stops moving, and then the real-time data g i z ≤ g tx z is recognized, and the up action is terminated.

对于下动作,当实时数据gi z≤gtn x,光标停止移动,随后识别到实时数据gi z≥gtn z,下动作终止。For the down action, when the real-time data g i z ≤ g tn x , the cursor stops moving, and then the real-time data g i z ≥ g tn z is recognized, and the down action is terminated.

对于单击动作,其动作与下动作在一个方向上,当实时数据gi z≤gtn x,光标停止移动,随后识别到实时数据gi z≥gtn z,单击动作终止。For the click action, its action is in the same direction as the down action. When the real-time data g i z ≤ g tn x , the cursor stops moving, and then the real-time data g i z ≥ g tn z is recognized, and the click action is terminated.

在上述的基于运动传感器的可穿戴式鼠标控制方法,步骤6中,使用如下规则来分割数据:In the aforementioned motion sensor-based wearable mouse control method, in step 6, the following rules are used to segment the data:

定义头部最初在中间区域正对屏幕,当陀螺仪的数据被识别到第一次到达某一阈值时,该点被标志为动作开始的点,记为Ps。当陀螺仪数据在相反方向第二次到达某阈值时,该点被标志为动作结束的点,记为Pe。则Ps和Pe中间的一段被分割出来作为一个动作。It is defined that the head is facing the screen in the middle area initially, when the gyroscope data is recognized and reaches a certain threshold for the first time, this point is marked as the point where the action starts, denoted as P s . When the gyroscope data reaches a certain threshold for the second time in the opposite direction, this point is marked as the point where the action ends, which is denoted as P e . Then the section between P s and P e is divided as an action.

在上述的基于运动传感器的可穿戴式鼠标控制方法,步骤7的具体实现包括以下子步骤:In the above-mentioned wearable mouse control method based on the motion sensor, the specific realization of step 7 includes the following sub-steps:

步骤7.1:用户在使用之前依次做5个定义的动作5次。Step 7.1: The user does 5 defined actions sequentially 5 times before use.

步骤7.2:提取得到的数据的5对极值,将得到的最低波峰和最高波谷乘以某一系数k作为训练出的个性化阈值。Step 7.2: Extract 5 pairs of extreme values of the obtained data, and multiply the obtained lowest peak and highest trough by a certain coefficient k as the trained personalized threshold.

相对于现有技术,本发明的有益成果是:控制更精确,操作简单,不影响正常事物,具有可扩展性,而且具有普遍的适用性。Compared with the prior art, the invention has the advantages of more precise control, simple operation, no influence on normal things, scalability and universal applicability.

附图说明Description of drawings

图1为本发明的系统框架图。Fig. 1 is a system frame diagram of the present invention.

图2a为提取的波形信号示例图。Figure 2a is an example diagram of the extracted waveform signal.

图2b为提取的波形信号预处理后结果图。Figure 2b is a graph of the extracted waveform signal after preprocessing.

图3a为用户在普适阈值下测试结果的准确率。Figure 3a shows the accuracy rate of user test results under the universal threshold.

图3b为用户在普适阈值下测试结果的召回率。Figure 3b shows the recall rate of user test results under the universal threshold.

图4a为各动作识别准确率随k值变化曲线。Figure 4a is the variation curve of each action recognition accuracy rate with k value.

图4b为各动作识别召回率随k值变化曲线。Figure 4b is the curve of the recall rate of each action recognition with the value of k.

图5为个性化阈值的判断精度。Figure 5 shows the judgment accuracy of the personalized threshold.

具体实施方式detailed description

为了便于本领域普通技术人员理解和实施本发明,下面结合附图及实施例对本发明作进一步的详细描述,应当理解,此处所描述的实施示例仅用于说明和解释本发明,并不用于限定本发明。In order to facilitate those of ordinary skill in the art to understand and implement the present invention, the present invention will be described in further detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described here are only used to illustrate and explain the present invention, and are not intended to limit this invention.

本发明主要基于穿戴式设备和传感器技术,考虑到头部动作的运动特征,提出了一种基于运动传感器的可穿戴式鼠标控制方法。本方法充分利用头部动作的运动数据,在不影响正常视物的同时,实现了易于操作且高精度的鼠标控制。本发明可用于为无臂人士使用,让无臂或者手臂残疾人士也可以使用鼠标,同时,正常人也可以使用本发明来解放双手。The present invention is mainly based on wearable equipment and sensor technology, considering the motion characteristics of head movements, and proposes a wearable mouse control method based on motion sensors. The method makes full use of the movement data of the head movement, and realizes easy-to-operate and high-precision mouse control without affecting normal vision. The present invention can be used for armless persons, so that armless or arm disabled persons can also use the mouse, and at the same time, normal people can also use the present invention to free their hands.

本发明提供的方法能够用计算机软件技术实现流程。参见图1,本发明提供的一种基于运动传感器的可穿戴式鼠标控制方法,包括以下步骤:The method provided by the invention can use computer software technology to realize the process. Referring to Fig. 1, a kind of wearable mouse control method based on motion sensor provided by the present invention comprises the following steps:

步骤1:从传感器采集动作数据,具体实现过程是:Step 1: Collect action data from the sensor. The specific implementation process is:

步骤1.1:选择蓝牙或Wifi热点的方式连接嵌入式穿戴式设备和被控制的电脑,建立Socket连接来传递数据。端口设置为8088。Step 1.1: Select Bluetooth or Wifi hotspot to connect the embedded wearable device and the controlled computer, and establish a Socket connection to transfer data. The port is set to 8088.

步骤1.2:将从加速计和陀螺仪得到的三个方向的运动数据分别依次存储。当接收到的数据数大于窗口长度时则去除最先的数据。采样率设置为每秒40次。Step 1.2: Store the motion data in three directions obtained from the accelerometer and gyroscope in sequence. When the number of received data is greater than the window length, the first data will be removed. The sampling rate is set to 40 times per second.

实施例具体的实施过程说明如下:The specific implementation process of the embodiment is described as follows:

首先将穿戴式设备与控制设备(如电脑)通过蓝牙或Wifi热点进行连接,建立Socket连接后传送传感器数据,端口设置为8088.First, connect the wearable device and the control device (such as a computer) through Bluetooth or Wifi hotspot, establish a Socket connection and transmit sensor data, and set the port to 8088.

然后,将从加速计和陀螺仪得到的三个方向的运动数据分别依次存储。当接收到的数据数大于定义的窗口数时则去除最先的数据,然后将剩下的数据放在窗口内交与电脑端程序处理。采样率设置为每秒40次。当采样率过大时,成本过大,且可能会造成拥塞。当采样率过小时,动作特征会不够明显。Then, the motion data in three directions obtained from the accelerometer and the gyroscope are respectively stored sequentially. When the number of received data is greater than the number of defined windows, the first data will be removed, and then the remaining data will be placed in the window and handed over to the computer program for processing. The sampling rate is set to 40 times per second. When the sampling rate is too large, the cost is too large and may cause congestion. When the sampling rate is too small, the motion features will not be obvious enough.

步骤2:定义动作。对鼠标的每一个操作,都定义一个头部动作与之对应。Step 2: Define the action. For each operation of the mouse, a corresponding head movement is defined.

步骤2.1:假定使用者一开始正对着屏幕,将初始的头部所在位置成为中间区域。Step 2.1: Assuming that the user is facing the screen at the beginning, the initial head position is defined as the middle area.

步骤2.2:定义单击操作。将头部快速向下,然后快速返回到中间区域的动作定义为单击操作。Step 2.2: Define the click action. The action of snapping the head down and then snapping back to the middle area is defined as a click operation.

步骤2.3:定义方向动作。将头部从中间区域转向左,保持一段时间至光标达到期望位置后转回中间区域定义为左移动作。将头部从中间区域转向右,保持一段时间至光标达到期望位置后转回中间区域定义为右移动作。将头部从中间区域转向上,保持一段时间至光标达到期望位置后转回中间区域定义为上移动作。将头部从中间区域转向下,保持一段时间至光标达到期望位置后转回中间区域定义为下移动作。Step 2.3: Define the direction action. Turning the head from the middle area to the left, keeping it for a while until the cursor reaches the desired position, and then turning back to the middle area is defined as a left movement. Turning the head from the middle area to the right, keeping it for a while until the cursor reaches the desired position, and then turning back to the middle area is defined as a right movement. Turning the head upward from the middle area, keeping it for a while until the cursor reaches the desired position, and then turning back to the middle area is defined as an up movement. Turning the head down from the middle area, keeping it for a while until the cursor reaches the desired position, and then turning back to the middle area is defined as a down movement.

具体实现过程是:The specific implementation process is:

以左动作为例。左动作的定义为屏幕上的光标向左移动。一旦头部从中间区域向左移动,光标也开始向左移动。头部会在左区域保持一段距离,此时光标会保持向左移动。一旦头部摆回中间区域,光标即停止移动。Take left action as an example. A left action is defined as moving the cursor on the screen to the left. Once the head moves to the left from the middle area, the cursor starts moving to the left as well. The head will keep a certain distance in the left area, and the cursor will keep moving to the left. Once the head is swung back to the middle area, the cursor stops moving.

步骤3:对数据进行预处理。在不影响动作特征提取的前提下对原始数据进行去噪操作。Step 3: Preprocess the data. The denoising operation is performed on the original data without affecting the extraction of motion features.

步骤3.1:使用阈值去噪的方式消除噪声。定义一个噪声阈值Eth,当接收到的数据大于该阈值时才向设备传输数据,否则置为0。Step 3.1: Use threshold denoising to remove noise. Define a noise threshold E th , and only transmit data to the device when the received data is greater than the threshold, otherwise it is set to 0.

步骤3.2:使用最大值滤波的方式去除齿波。最大值滤波即把窗口内所有值的最大值作为当前滤波后的值。定义滤波窗口为τ,则当前时间滤波后的值rt可表示为:Step 3.2: Use the maximum value filter to remove the tooth wave. Maximum value filtering is to use the maximum value of all values in the window as the current filtered value. Define the filtering window as τ, then the filtered value r t at the current time can be expressed as:

rt=max{rt-τ/2,...,rt-1,rt,rt+1,...,rt+τ/2}r t =max{r t-τ/2 ,...,r t-1 ,r t ,r t+1 ,...,r t+τ/2 }

具体实现过程是:The specific implementation process is:

定义一个噪声阈值Eth,原始数据中小于该阈值的值将会被置为0,只有接收到的数据大于该阈值时才会向设备传输数据。将经过阈值去噪后的数据再使用最大值滤波的方法来去除齿波。最大值滤波将当前位置的左右值放在一起考虑,将其中的最大值最为该位置滤波后的值。Define a noise threshold E th , the value in the original data that is smaller than the threshold will be set to 0, and only when the received data is greater than the threshold will the data be transmitted to the device. The data after the threshold denoising is used to remove the tooth wave by the method of maximum value filtering. Maximum value filtering considers the left and right values of the current position together, and takes the maximum value as the filtered value of the position.

步骤4:特征提取。Step 4: Feature extraction.

具体实现过程是:The specific implementation process is:

因在加速计的x轴方向,单击操作比另外四个方向的动作特征更为显著,而上下左右四个动作在加速计上没有明显的区别,所以可以使用加速计x轴方向的值来将单击动作和方向动作区分开来。Because in the x-axis direction of the accelerometer, the click operation is more significant than the other four directions, and there is no obvious difference in the four actions of the accelerometer, so you can use the value of the x-axis direction of the accelerometer to Distinguish between click actions and direction actions.

因在陀螺仪的x轴方向,左右移动比上下移动动作特征更为显著,所以可以使用陀螺仪x轴方向的值来将左右动作和上下动作区分开来。又因左右动作特征正好相反,故也可将左右动作区分开来。Because in the x-axis direction of the gyroscope, the left and right movements are more significant than the up and down movements, so the value in the x-axis direction of the gyroscope can be used to distinguish the left and right movements from the up and down movements. And because the characteristics of the left and right movements are just opposite, it is also possible to distinguish the left and right movements.

因在陀螺仪的z轴方向,上下移动比左右移动动作特征更为显著,所以可以使用陀螺仪z轴方向的值来将上下动作和左右动作区分开来。又因左右动作特征正好相反,故也可将上下动作区分开来。Because in the z-axis direction of the gyroscope, the up and down motion is more significant than the left and right motion, so the value in the z-axis direction of the gyroscope can be used to distinguish the up and down motion from the left and right motion. And because the characteristics of the left and right movements are just opposite, it is also possible to distinguish the up and down movements.

步骤5:动作分类。Step 5: Action Classification.

具体实现过程是:The specific implementation process is:

步骤5.1:定义atn x,gtn x,gtx x,gtn z,gtx z分别为加速计x轴方向的下阈值、陀螺仪x、z轴方向的上下阈值。阈值的值由预训练步骤决定,作为决策树的判定数据。定义ai x,gi x,gi z为当前时间i的加速计x轴方向、陀螺仪x轴方向、陀螺仪z轴方向的值。Step 5.1: define a tn x , g tn x , g tx x , g tn z , and g tx z as the lower threshold of the accelerometer in the x-axis direction and the upper and lower thresholds of the gyroscope in the x-axis and z-axis directions, respectively. The value of the threshold is determined by the pre-training step as the decision data of the decision tree. Define a i x , g i x , and g i z as the values of the accelerometer x-axis direction, gyroscope x-axis direction, and gyroscope z-axis direction at the current time i.

检查当前时间加速计x轴方向的值,如果ai x<atn x,则头部开始点击动作,否则,进入步骤5.2。Check the value of the x-axis direction of the accelerometer at the current time, if a i x <a tn x , the head starts to click, otherwise, go to step 5.2.

步骤5.2:检查当前时间陀螺仪x轴方向的值,如果gi x<gtn x或者gi x>gtx x,头部则开始左或右动作,进入步骤5.3。否则,进入步骤5.4。Step 5.2: Check the value of the x-axis direction of the gyroscope at the current time. If g i x <g tn x or g i x >g tx x , the head will start to move left or right, and go to step 5.3. Otherwise, go to step 5.4.

步骤5.3:如果gi x<gtn x,则头部开始向左移动,屏幕上的光标向左移动。否则,头部向右移动,屏幕上的光标也向右移。Step 5.3: If g i x <g tn x , the head starts to move to the left and the cursor on the screen moves to the left. Otherwise, the head moves to the right, and the cursor on the screen moves to the right.

步骤5.4:检查当前时间陀螺仪z轴方向的值,如果gi z<gtn z或者gi z>gtx z,头部则开始上或下动作,进入步骤5.5。否则,在当前时间没有动作发生。Step 5.4: Check the value of the z-axis direction of the gyroscope at the current time. If g i z <g tn z or g i z >g tx z , the head will start to move up or down, and go to step 5.5. Otherwise, no action occurs at the current time.

步骤5.5:如果gi z<gtn z,则头部开始向上移动,屏幕上的光标向上移动。否则,头部开始向下移动,屏幕上的光标也向右移。Step 5.5: If g i z <g tn z , the head starts to move up and the cursor on the screen moves up. Otherwise, the head starts moving down and the on-screen cursor moves to the right.

步骤5.6:一旦动作被识别了,光标便会在屏幕上执行相应动作直至动作的终止,动作的终止定义如下:Step 5.6: Once the action is recognized, the cursor will execute the corresponding action on the screen until the end of the action. The end of the action is defined as follows:

对于左动作,当实时数据gi x≥gtx x,光标停止移动,随后识别到实时数据gi x≤gtn x,左动作终止。For the left action, when the real-time data g i x ≥ g tx x , the cursor stops moving, and then the real-time data g i x ≤ g tn x is recognized, and the left action is terminated.

对于右动作,当实时数据ai x≤atn x,光标停止移动,随后识别到实时数据ai x≥atn x,右动作终止。For the right action, when the real-time data a i x ≤ a tn x , the cursor stops moving, and then the real-time data a i x ≥ a tn x is recognized, and the right action is terminated.

对于上动作,当实时数据gi z≥gtx x,光标停止移动,随后识别到实时数据gi z≤gtx z,上动作终止。For the up action, when the real-time data g i z ≥ g tx x , the cursor stops moving, and then the real-time data g i z ≤ g tx z is recognized, and the up action is terminated.

对于下动作,当实时数据gi z≤gtn x,光标停止移动,随后识别到实时数据gi z≥gtn z,下动作终止。For the down action, when the real-time data g i z ≤ g tn x , the cursor stops moving, and then the real-time data g i z ≥ g tn z is recognized, and the down action is terminated.

对于单击动作,其动作与下动作在一个方向上,当实时数据gi z≤gtn x,光标停止移动,随后识别到实时数据gi z≥gtn z,单击动作终止。For the click action, its action is in the same direction as the down action. When the real-time data g i z ≤ g tn x , the cursor stops moving, and then the real-time data g i z ≥ g tn z is recognized, and the click action is terminated.

步骤6:动作分割Step 6: Action Segmentation

具体实现过程是:The specific implementation process is:

我们得到的原始数据是时间连续数据,动作分割将这些时间连续的数据分割成块,每一块包含的数据代表一个动作。依据动作的定义,每一个动作都包含两个相反的部分。例如,向左动作首先将头部从中间区域转向左,随后再由左转回中间区域。结果是,在同一方向上有相反的两个波形。定义在一个方向上有两个阈值Tn,Tp来决定动作的开始和结束。其中,Tn是下阈值,Tp是上阈值。使用如下规则来分割数据:The raw data we get is time-continuous data, and action segmentation divides these time-continuous data into blocks, and each block contains data representing an action. According to the definition of action, each action contains two opposite parts. For example, a leftward movement first turns the head from the middle area to the left, and then from the left back to the middle area. The result is that there are two opposite waveforms in the same direction. Define two thresholds T n and T p in one direction to determine the start and end of the action. Wherein, T n is the lower threshold and T p is the upper threshold. Use the following rules to split the data:

假设头部最初在中间区域正对屏幕,当陀螺仪的数据被识别到第一次到达某一阈值时,该点被标志为动作开始的点,记为Ps。当陀螺仪数据在相反方向第二次到达某阈值时,该点被标志为动作结束的点,记为Pe。则Ps和Pe中间的一段被分割出来作为一个动作。Assuming that the head is facing the screen in the middle area initially, when the gyroscope data is recognized and reaches a certain threshold for the first time, this point is marked as the point at which the action starts, denoted as P s . When the gyroscope data reaches a certain threshold for the second time in the opposite direction, this point is marked as the point where the action ends, which is denoted as P e . Then the section between P s and P e is divided as an action.

步骤7:数据预训练Step 7: Data pre-training

具体实现过程是:The specific implementation process is:

因不同的人的动作习惯不同,产生的动作特征波形也不同。所以上步骤定义的分类器非普适,需要为不同的用户训练个性化分类器。Due to the different action habits of different people, the resulting action characteristic waveforms are also different. Therefore, the classifier defined in the above step is not universal, and a personalized classifier needs to be trained for different users.

步骤7.1:用户在使用之前依次做5个定义的动作5次。Step 7.1: The user does 5 defined actions sequentially 5 times before use.

步骤7.2:提取得到的数据的5对极值,将得到的最低波峰和最高波谷乘以某一系数k作为训练出的个性化阈值。将该个性化阈值作为分类器的阈值将分类器个性化。Step 7.2: Extract 5 pairs of extreme values of the obtained data, and multiply the obtained lowest peak and highest trough by a certain coefficient k as the trained personalized threshold. Personalize the classifier by using this personalized threshold as the threshold of the classifier.

步骤8:错误矫正Step 8: Error Correction

具体实现过程是:The specific implementation process is:

当用户使用时,用户可以看见屏幕上光标的实时移动。当错误发生时,用户可以知道发生了一个错误,并可以立即开始矫正。如果光标向错误的方向移动,用户可以执行同样的动作,然后进行接触。例如,当用户向左时,发现光标向右移动,用户可以也执行一次向右动作来矫正这个错误。用这个方法,当前一个动作结束了,后续动作不会受之前的错误影响。When the user uses it, the user can see the real-time movement of the cursor on the screen. When an error occurs, the user can know that an error has occurred and can immediately start correcting it. If the cursor moves in the wrong direction, the user can perform the same action and then make a contact. For example, when the user moves to the left and finds that the cursor moves to the right, the user can also perform a right action to correct this error. In this way, the previous action is finished, and the subsequent action will not be affected by the previous error.

应当理解的是,本说明书未详细阐述的部分均属于现有技术。It should be understood that the parts not described in detail in this specification belong to the prior art.

应当理解的是,上述针对较佳实施例的描述较为详细,并不能因此而认为是对本发明专利保护范围的限制,本领域的普通技术人员在本发明的启示下,在不脱离本发明权利要求所保护的范围情况下,还可以做出替换或变形,均落入本发明的保护范围之内,本发明的请求保护范围应以所附权利要求为准。It should be understood that the above-mentioned descriptions for the preferred embodiments are relatively detailed, and should not therefore be considered as limiting the scope of the patent protection of the present invention. Within the scope of protection, replacements or modifications can also be made, all of which fall within the protection scope of the present invention, and the scope of protection of the present invention should be based on the appended claims.

Claims (5)

1.一种基于运动传感器的可穿戴式鼠标控制方法,其特征在于,包括以下步骤:1. A wearable mouse control method based on motion sensor, is characterized in that, comprises the following steps: 步骤1:进行数据采集,具体包括:Step 1: Conduct data collection, including: 步骤1.1:选择蓝牙或Wifi热点连接嵌入式穿戴式设备和被控制的电脑,建立Socket连接来传递数据;端口设置为8088;Step 1.1: Select a Bluetooth or Wifi hotspot to connect the embedded wearable device and the controlled computer, and establish a Socket connection to transfer data; the port is set to 8088; 步骤1.2:将从加速计和陀螺仪得到的三个方向的运动数据分别依次存储;当接收到的数据数大于窗口长度时则去除最先的数据;采样率设置为每秒40次;Step 1.2: Store the motion data in three directions obtained from the accelerometer and gyroscope in sequence; when the number of received data is greater than the window length, remove the first data; set the sampling rate to 40 times per second; 步骤2:动作定义,具体包括以下子步骤:Step 2: Action definition, specifically including the following sub-steps: 步骤2.1:定义使用者一开始正对着屏幕,将初始的头部所在位置成为中间区域;Step 2.1: Define that the user is facing the screen at the beginning, and the initial head position becomes the middle area; 步骤2.2:定义单击操作;将头部快速向下,然后快速返回到中间区域的动作定义为单击操作;Step 2.2: Define the click operation; the action of moving the head down quickly and then quickly returning to the middle area is defined as the click operation; 步骤2.3:定义方向动作;将头部从中间区域转向左,保持一段时间至光标达到期望位置后转回中间区域定义为左移动作;将头部从中间区域转向右,保持一段时间至光标达到期望位置后转回中间区域定义为右移动作;将头部从中间区域转向上,保持一段时间至光标达到期望位置后转回中间区域定义为上移动作;Step 2.3: Define the direction action; turn the head from the middle area to the left, and keep it for a while until the cursor reaches the desired position, then turn back to the middle area to define the left movement; turn the head from the middle area to the right, and keep it for a while until the cursor reaches the desired position Turning back to the middle area after the desired position is defined as moving right; turning the head from the middle area upwards, keeping it for a period of time until the cursor reaches the desired position and then turning back to the middle area is defined as moving up; 将头部从中间区域转向下,保持一段时间至光标达到期望位置后转回中间区域定义为下移动作;Turn the head from the middle area down, keep it for a period of time until the cursor reaches the desired position, and then turn back to the middle area to define the downward movement; 步骤3:消除噪声的去除齿波的数据预处理;Step 3: data preprocessing for eliminating noise and removing tooth waves; 步骤4:特征提取,具体包括以下子步骤:Step 4: feature extraction, specifically including the following sub-steps: 步骤4.1:因在加速计的x轴方向,单击操作比另外四个方向的动作特征更为显著,所以可以使用加速计x轴方向的值来将单击动作和方向动作区分开来;Step 4.1: Because in the x-axis direction of the accelerometer, the click operation is more significant than the other four directions, so the value of the x-axis direction of the accelerometer can be used to distinguish the click action from the direction action; 步骤4.2:因在陀螺仪的x轴方向,左右移动比上下移动动作特征更为显著,所以可以使用陀螺仪x轴方向的值来将左右动作和上下动作区分开来;又因左右动作特征正好相反,故也可将左右动作区分开来;Step 4.2: Because in the x-axis direction of the gyroscope, the left and right movement is more significant than the up and down movement, so the value in the x-axis direction of the gyroscope can be used to distinguish the left and right movements from the up and down movements; and because the left and right movement characteristics are exactly On the contrary, it is also possible to distinguish left and right actions; 步骤4.3:因在陀螺仪的z轴方向,上下移动比左右移动动作特征更为显著,所以可以使用陀螺仪z轴方向的值来将上下动作和左右动作区分开来;又因左右动作特征正好相反,故也可将上下动作区分开来;Step 4.3: Because in the z-axis direction of the gyroscope, the up-and-down movement is more significant than the left-right movement, so the value in the z-axis direction of the gyroscope can be used to distinguish the up-down movement from the left-right movement; and because the left-right movement characteristics are exactly On the contrary, it is also possible to distinguish the up and down movements; 步骤5:动作分类,具体实现包括以下子步骤:Step 5: Action classification, the specific implementation includes the following sub-steps: 步骤5.1:定义atn x,gtn x,gtx x,gtn z,gtx z分别为加速计x轴方向的下阈值、陀螺仪x、z轴方向的上下阈值;阈值的值由预训练步骤决定,作为决策树的判定数据;定义ai x,gi x,gi z为当前时间i的加速计x轴方向、陀螺仪x轴方向、陀螺仪z轴方向的值;Step 5.1: define a tn x , g tn x , g tx x , g tn z , and g tx z as the lower threshold of the accelerometer in the x-axis direction and the upper and lower thresholds of the gyroscope in the x-axis and z-axis directions; the threshold values are determined by the preset Determined by the training step, as the decision data of the decision tree; define a i x , g i x , and g i z as the values of the accelerometer x-axis direction, gyroscope x-axis direction, and gyroscope z-axis direction at the current time i; 检查当前时间加速计x轴方向的值,如果ai x<atn x,则头部开始点击动作,否则,进入步骤5.2;Check the value of the x-axis direction of the accelerometer at the current time, if a i x <a tn x , the head starts to click, otherwise, go to step 5.2; 步骤5.2:检查当前时间陀螺仪x轴方向的值,如果gi x<gtn x或者gi x>gtx x,头部则开始左或右动作,进入步骤5.3;否则,进入步骤5.4;Step 5.2: Check the value of the x-axis direction of the gyroscope at the current time. If g i x <g tn x or g i x >g tx x , the head will start to move left or right, and go to step 5.3; otherwise, go to step 5.4; 步骤5.3:如果gi x<gtn x,则头部开始向左移动,屏幕上的光标向左移动;否则,头部向右移动,屏幕上的光标也向右移;Step 5.3: If g i x <g tn x , then the head starts to move to the left, and the cursor on the screen moves to the left; otherwise, the head moves to the right, and the cursor on the screen also moves to the right; 步骤5.4:检查当前时间陀螺仪z轴方向的值,如果gi z<gtn z或者gi z>gtx z,头部则开始上或下动作,进入步骤5.5;否则,在当前时间没有动作发生;Step 5.4: Check the value of the z-axis direction of the gyroscope at the current time. If g i z <g tn z or g i z >g tx z , the head will start to move up or down, and go to step 5.5; otherwise, there is no action occurs; 步骤5.5:如果gi z<gtn z,则头部开始向上移动,屏幕上的光标向上移动;否则,头部开始向下移动,屏幕上的光标也向右移;Step 5.5: If g i z <g tn z , then the head starts to move up, and the cursor on the screen moves up; otherwise, the head starts to move down, and the cursor on the screen also moves to the right; 步骤5.6:一旦动作被识别了,光标便会在屏幕上执行相应动作直至动作的终止,Step 5.6: Once the action is recognized, the cursor will execute the corresponding action on the screen until the end of the action, 步骤6:动作分割,得到的原始数据是时间连续数据,动作分割将这些时间连续的数据分割成块,每一块包含的数据代表一个动作;依据动作的定义,每一个动作都包含两个相反的部分;例如,向左动作首先将头部从中间区域转向左,随后再由左转回中间区域;结果是,在同一方向上有相反的两个波形;定义在一个方向上有两个阈值Tn,Tp来决定动作的开始和结束;其中,Tn是下阈值,Tp是上阈值;Step 6: Action segmentation. The obtained original data is time-continuous data. Action segmentation divides these time-continuous data into blocks, and the data contained in each block represents an action; according to the definition of action, each action contains two opposite Partial; for example, a leftward movement first turns the head from the middle area to the left, and then turns from the left back to the middle area; as a result, there are two opposite waveforms in the same direction; define two thresholds T in one direction n , T p to determine the start and end of the action; among them, T n is the lower threshold, T p is the upper threshold; 步骤7:数据预训练,具体实现包括以下子步骤:Step 7: Data pre-training, the specific implementation includes the following sub-steps: 步骤7.1:用户在使用之前依次做5个定义的动作5次;Step 7.1: The user performs 5 defined actions sequentially 5 times before use; 步骤7.2:提取得到的数据的5对极值,将得到的最低波峰和最高波谷乘以某一系数k作为训练出的个性化阈值;Step 7.2: Extract 5 pairs of extreme values of the obtained data, and multiply the obtained lowest peak and highest trough by a certain coefficient k as the trained personalized threshold; 步骤8:错误矫正,当用户使用时,用户可以看见屏幕上光标的实时移动;当错误发生时,用户可以知道发生了一个错误,并可以立即开始矫正;如果光标向错误的方向移动,用户可以执行同样的动作,然后进行接触;例如,当用户向左时,发现光标向右移动,用户可以也执行一次向右动作来矫正这个错误;用这个方法,当前一个动作结束了,后续动作不会受之前的错误影响。Step 8: Error correction, when the user is using it, the user can see the real-time movement of the cursor on the screen; when an error occurs, the user can know that an error has occurred, and can start to correct it immediately; if the cursor moves to the wrong direction, the user can Perform the same action, and then make a contact; for example, when the user moves to the left and finds that the cursor moves to the right, the user can also perform a right action to correct this error; in this way, the current action is over, and subsequent actions will not Affected by previous errors. 2.根据权利要求1所述的基于运动传感器的可穿戴式鼠标控制方法,其特征在于,步骤3的具体实现包括以下子步骤:2. the wearable mouse control method based on motion sensor according to claim 1, is characterized in that, the concrete realization of step 3 comprises the following sub-steps: 步骤3.1:使用阈值去噪的方式消除噪声;定义一个噪声阈值Eth,当接收到的数据大于该阈值时才向设备传输数据,否则置为0;Step 3.1: Use threshold denoising to eliminate noise; define a noise threshold E th , and only transmit data to the device when the received data is greater than the threshold, otherwise set it to 0; 步骤3.2:使用最大值滤波的方式去除齿波;最大值滤波即把窗口内所有值的最大值作为当前滤波后的值;定义滤波窗口为τ,则当前时间滤波后的值rt可表示为:Step 3.2: Use the maximum value filter to remove the tooth wave; the maximum value filter takes the maximum value of all values in the window as the current filtered value; define the filter window as τ, then the current time filtered value r t can be expressed as : rt=max{rt-τ/2,...,rt-1,rt,rt+1,...,rt+τ/2}。r t =max{r t−τ/2 , . . . , r t−1 , r t , r t+1 , . . . , r t+τ/2 }. 3.根据权利要求1所述的基于运动传感器的可穿戴式鼠标控制方法,其特征在于,步骤5.6中动作的终止定义如下:3. the wearable mouse control method based on motion sensor according to claim 1, is characterized in that, the termination definition of action in step 5.6 is as follows: 对于左动作,当实时数据gi x≥gtx x,光标停止移动,随后识别到实时数据gi x≤gtn x,左动作终止;For the left action, when the real-time data g i x ≥ g tx x , the cursor stops moving, and then the real-time data g i x ≤ g tn x is recognized, and the left action is terminated; 对于右动作,当实时数据ai x≤atn x,光标停止移动,随后识别到实时数据ai x≥atn x,右动作终止;For the right action, when the real-time data a i x ≤ a tn x , the cursor stops moving, and then the real-time data a i x ≥ a tn x is recognized, and the right action is terminated; 对于上动作,当实时数据gi z≥gtx x,光标停止移动,随后识别到实时数据gi z≤gtx z,上动作终止;For the up action, when the real-time data g i z ≥ g tx x , the cursor stops moving, and then the real-time data g i z ≤ g tx z is recognized, and the up action is terminated; 对于下动作,当实时数据gi z≤gtn x,光标停止移动,随后识别到实时数据gi z≥gtn z,下动作终止;For the down action, when the real-time data g i z ≤ g tn x , the cursor stops moving, and then the real-time data g i z ≥ g tn z is recognized, and the down action is terminated; 对于单击动作,其动作与下动作在一个方向上,当实时数据gi z≤gtn x,光标停止移动,随后识别到实时数据gi z≥gtn z,单击动作终止。For the click action, its action is in the same direction as the down action. When the real-time data g i z ≤ g tn x , the cursor stops moving, and then the real-time data g i z ≥ g tn z is recognized, and the click action is terminated. 4.根据权利要求1所述的基于运动传感器的可穿戴式鼠标控制方法,其特征在于,步骤6中,使用如下规则来分割数据:4. the wearable mouse control method based on motion sensor according to claim 1, is characterized in that, in step 6, uses following rule to divide data: 定义头部最初在中间区域正对屏幕,当陀螺仪的数据被识别到第一次到达某一阈值时,该点被标志为动作开始的点,记为Ps;当陀螺仪数据在相反方向第二次到达某阈值时,该点被标志为动作结束的点,记为Pe;则Ps和Pe中间的一段被分割出来作为一个动作。Define that the head is initially facing the screen in the middle area. When the gyroscope data is recognized and reaches a certain threshold for the first time, this point is marked as the point where the action starts, denoted as P s ; when the gyroscope data is in the opposite direction When a certain threshold is reached for the second time, this point is marked as the point where the action ends, which is recorded as P e ; then the section between P s and P e is divided as an action. 5.根据权利要求1所述的基于运动传感器的可穿戴式鼠标控制方法,其特征在于,步骤7的具体实现包括以下子步骤:5. the wearable mouse control method based on motion sensor according to claim 1, is characterized in that, the concrete realization of step 7 comprises the following sub-steps: 步骤7.1:用户在使用之前依次做5个定义的动作5次;Step 7.1: The user performs 5 defined actions sequentially 5 times before use; 步骤7.2:提取得到的数据的5对极值,将得到的最低波峰和最高波谷乘以某一系数k作为训练出的个性化阈值。Step 7.2: Extract 5 pairs of extreme values of the obtained data, and multiply the obtained lowest peak and highest trough by a certain coefficient k as the trained personalized threshold.
CN201710812802.6A 2017-09-11 2017-09-11 A kind of wearable mouse control method based on motion sensor Pending CN107506041A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710812802.6A CN107506041A (en) 2017-09-11 2017-09-11 A kind of wearable mouse control method based on motion sensor

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710812802.6A CN107506041A (en) 2017-09-11 2017-09-11 A kind of wearable mouse control method based on motion sensor

Publications (1)

Publication Number Publication Date
CN107506041A true CN107506041A (en) 2017-12-22

Family

ID=60695399

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710812802.6A Pending CN107506041A (en) 2017-09-11 2017-09-11 A kind of wearable mouse control method based on motion sensor

Country Status (1)

Country Link
CN (1) CN107506041A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108815845A (en) * 2018-05-15 2018-11-16 百度在线网络技术(北京)有限公司 The information processing method and device of human-computer interaction, computer equipment and readable medium
CN109782904A (en) * 2018-12-26 2019-05-21 南昌大学 A kind of wireless mouse based on intelligent glasses

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN201716687U (en) * 2009-11-27 2011-01-19 晶翔微系统股份有限公司 air mouse
CN102282527A (en) * 2008-11-21 2011-12-14 伦敦健康科学中心研究公司 Hands-free pointer system
CN103513770A (en) * 2013-10-09 2014-01-15 中国科学院深圳先进技术研究院 Man-machine interface equipment and man-machine interaction method based on three-axis gyroscope
CN103543843A (en) * 2013-10-09 2014-01-29 中国科学院深圳先进技术研究院 Man-machine interface equipment based on acceleration sensor and man-machine interaction method
CN104107134A (en) * 2013-12-10 2014-10-22 中山大学 Myoelectricity feedback based upper limb training method and system
CN104919393A (en) * 2012-11-20 2015-09-16 三星电子株式会社 Transitions and Interaction Models for Wearable Electronics

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102282527A (en) * 2008-11-21 2011-12-14 伦敦健康科学中心研究公司 Hands-free pointer system
CN201716687U (en) * 2009-11-27 2011-01-19 晶翔微系统股份有限公司 air mouse
CN104919393A (en) * 2012-11-20 2015-09-16 三星电子株式会社 Transitions and Interaction Models for Wearable Electronics
CN103513770A (en) * 2013-10-09 2014-01-15 中国科学院深圳先进技术研究院 Man-machine interface equipment and man-machine interaction method based on three-axis gyroscope
CN103543843A (en) * 2013-10-09 2014-01-29 中国科学院深圳先进技术研究院 Man-machine interface equipment based on acceleration sensor and man-machine interaction method
CN104107134A (en) * 2013-12-10 2014-10-22 中山大学 Myoelectricity feedback based upper limb training method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
李涛: "《数字图像处理之红外弱目标分割方法研究》", 30 June 2016 *

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108815845A (en) * 2018-05-15 2018-11-16 百度在线网络技术(北京)有限公司 The information processing method and device of human-computer interaction, computer equipment and readable medium
CN108815845B (en) * 2018-05-15 2019-11-26 百度在线网络技术(北京)有限公司 The information processing method and device of human-computer interaction, computer equipment and readable medium
CN109782904A (en) * 2018-12-26 2019-05-21 南昌大学 A kind of wireless mouse based on intelligent glasses

Similar Documents

Publication Publication Date Title
CN106527709B (en) Virtual scene adjusting method and head-mounted intelligent device
CN107589782B (en) Method and apparatus for a gesture control interface of a wearable device
CN102184011B (en) Human-computer interaction equipment
CN112926423A (en) Kneading gesture detection and recognition method, device and system
WO2013139181A1 (en) User interaction system and method
CN107132986B (en) Method and device for intelligently adjusting touch response area through virtual keys
US10521018B2 (en) Human body-based interaction method and interaction apparatus
CN105122201A (en) Multi-touch interactions on eyewear
CN104238736A (en) Device, method, and system to recognize motion using gripped object
CN113031464B (en) Device control method, device, electronic device and storage medium
US10901507B2 (en) Bioelectricity-based control method and apparatus, and bioelectricity-based controller
CN106778565B (en) Pull-up counting method and device
JP2017068572A (en) Wearable device
WO2022179279A1 (en) Interaction method, electronic device, and interaction system
CN111158487A (en) Human-computer interaction method using wireless headset to interact with smart terminal
CN107506041A (en) A kind of wearable mouse control method based on motion sensor
CN117075727A (en) Method and equipment for realizing keyboard input in virtual space
WO2023005139A1 (en) Line drawing method and apparatus, electronic device and computer-readable storage medium
CN107908331B (en) Display control method of desktop icon and electronic equipment
US20150177891A1 (en) Wearable apparatus skin input
KR101631025B1 (en) Method for estimating data of hand joint, and recording medium thereof
CN106951109B (en) Method and device for acquiring hand gesture
CN104461365A (en) Terminal touch method and device
CN111766941A (en) A kind of gesture recognition method and system based on smart ring
CN105611143A (en) Shooting method of electronic equipment and the electronic equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20171222

RJ01 Rejection of invention patent application after publication