CN105607764B - Track input device and track input method - Google Patents

Abstract

The invention provides a track input device and a track input method. The trajectory input device includes: an attitude sensor (101) for acquiring an attitude of the trajectory input device (10); a displacement sensor (102) for acquiring a movement locus of the locus input device (10) when moving on an input medium; and a processing unit (103) which corrects the movement locus of the trajectory input device (10) on an input medium, which is acquired by the displacement sensor (102), using the posture of the trajectory input device (10) acquired by the posture sensor (101).

Description

Track input device and track input method

technical field

The invention relates to the technical field of information processing, in particular to a trajectory input device and a trajectory input method.

Background technique

Nowadays, with the popularization of computers, people use keyboards, mice, etc. for input more when writing, and voice input appeared later. These novel ways have indeed brought convenience to people, but some content is not suitable for inputting with keyboard and voice, such as calligraphy, signature and so on.

The appearance of stylus has solved the above-mentioned problem. Sensors in the stylus pick up the movement of the tip to determine the data to be input to the computer.

Most of the stylus currently on the market needs to work with a special writing medium. For example, writing on a tablet. In contrast, a mouse can be moved across a mouse pad, a tabletop, or even other generally flat media surfaces to control cursor movement.

However, in the process of using the mouse for track input (for example, writing), the mouse should be kept moving, and if the posture of the mouse changes, the displacement information cannot be input correctly.

It can be seen that the existing trajectory input device either requires a special input medium, or the input is inaccurate due to the change of posture during the movement.

Contents of the invention

In view of the above problems, the present invention provides a trajectory input device and a trajectory input method, which can correct the error caused by the posture change of the trajectory input device, so that input can be performed anytime and anywhere without special input media.

On the one hand, an embodiment of the present invention provides a trajectory input device, including: an attitude sensor, which is used to obtain the posture of the trajectory input device; a displacement sensor, which is used to obtain the trajectory input device when it moves on the input medium. the movement trajectory of the trajectory; a processing unit, which uses the posture of the trajectory input device obtained by the posture sensor to correct the movement trajectory of the trajectory input device on the input medium obtained by the displacement sensor.

Further, the trajectory input device is a stylus, the attitude sensor includes a gyroscope, an accelerometer, and a magnetometer, the displacement sensor includes a photoelectric sensor, and the attitude includes the pitch angle and roll angle of the stylus. and yaw angle.

Further, the stylus includes: a pen holder with an internal space; a pen tip located at one end of the pen holder; wherein, the gyroscope, the accelerometer, the magnetometer and the photoelectric sensor are accommodated in the pen holder and coaxial with the stylus; and the processing unit obtains a posture transfer matrix according to the posture of the stylus, and uses the posture transfer matrix to compare the pen tip on the input medium The trajectory of the movement is corrected.

Further, the correction performed by the processing unit includes: obtaining the attitude of the stylus by fusing the data obtained by the accelerometer and the magnetometer with the data obtained by the gyroscope.

Further, the processing unit uses a complementary filtering algorithm to perform the fusion.

Further, the processing unit uses a Kalman filter algorithm to perform the fusion.

Further, the input medium is paper.

On the other hand, an embodiment of the present invention provides a trajectory input method using a trajectory input device, including the following steps: step a, when the trajectory input device moves on the input medium, acquire the posture and The trajectory of the trajectory input device; step b, using the posture of the trajectory input device to correct the trajectory of the trajectory input device on the input medium.

Further, the trajectory input device is a stylus, the attitude sensor includes a gyroscope, an accelerometer, and a magnetometer, the displacement sensor includes a photoelectric sensor, and the attitude includes the pitch angle and roll angle of the stylus. and yaw angle.

Further, the stylus includes: a pen holder with an internal space; a pen tip located at one end of the pen holder; wherein, the gyroscope, the accelerometer, the magnetometer and the photoelectric sensor are accommodated in the pen holder and coaxial with the stylus; and the step b includes: the processing unit obtains an attitude transfer matrix according to the attitude of the stylus, and uses the attitude transfer matrix to calculate the position of the pen tip at the Correct the movement trajectory on the input medium.

Further, the step b further includes: the processing unit obtains the attitude of the stylus by fusing the data obtained by the accelerometer and the magnetometer with the data obtained by the gyroscope.

Further, the processing unit uses a complementary filtering algorithm to perform the fusion.

Further, the processing unit uses a Kalman filter algorithm to perform the fusion.

Further, the input medium is paper.

The present invention can obtain the displacement information of trajectory input device by displacement sensor (photoelectric sensor), can obtain the posture of trajectory input device by posture sensor (gyroscope, accelerometer, magnetometer), then pass the posture transfer matrix to the trajectory input device The displacement information is corrected to obtain accurate input data.

Description of drawings

Fig. 1 shows a schematic diagram of the track input device of the present invention.

(a) of FIG. 2 is a front view of a stylus according to one embodiment of the present invention.

(b) of FIG. 2 is a back view of the stylus of this embodiment.

FIG. 3 is a flow chart of the input method of the stylus of the present invention.

Fig. 4 is a schematic diagram of the complementary filtering of the present invention.

Detailed ways

In order to enable those skilled in the art to better understand the technical solution of the present invention, the trajectory input device and the trajectory input method provided by the present invention will be described in detail below in conjunction with the accompanying drawings and specific embodiments. In these drawings, the same reference numerals are assigned to the same or corresponding components. The following are only the best implementation modes of the input device and track input method of the present invention, and the present invention is not limited to the following content.

The trajectory input device of the present invention will be described below with reference to FIG. 1 .

Fig. 1 shows a schematic diagram of the track input device of the present invention. As shown in Figure 1, track input device 10 comprises: attitude sensor 101, and it is used to obtain the attitude of track input device 10; Displacement sensor 102, it is used to obtain the moving track when track input device 10 moves on input medium; Processing The unit 103 uses the posture of the trajectory input device 10 obtained by the posture sensor 101 to correct the movement trajectory of the trajectory input device 10 obtained by the displacement sensor 102 on the input medium.

Attitude sensors commonly used today include gyroscopes, accelerometers, and magnetometers. However, these three types of attitude sensors have deficiencies in acquiring attitude.

For example, the angular velocity output by the gyroscope is an instantaneous quantity, and the angular velocity needs to be integrated with time to calculate the angle, and the obtained angular change is added to the initial angle to obtain the target angle. The smaller the integration time, the more accurate the output angle, but the principle of the gyroscope determines that its measurement reference is itself, and there is no absolute reference outside the system. In addition, the integration time cannot be infinitely small, so the cumulative error of the integration will vary with time. As the time increases rapidly, eventually the output angle does not match the reality, so the gyroscope can only work in a relatively short time scale.

The accelerometer measures the acceleration in the direction of gravity. There is an absolute reference outside the system "gravity axis". In the case of no external acceleration, it can accurately output the roll angle and pitch angle, and there will be no cumulative error in this angle. Accurate over long time scales. However, accelerometers also have disadvantages in measuring angles. The accelerometer actually uses MEMS technology to detect the tiny deformation caused by the inertial force, and the inertial force and gravity are essentially the same, so the accelerometer cannot distinguish between the acceleration of gravity and the acceleration of external force. When the system moves at variable speeds in three-dimensional space, the output of the accelerometer produces an error.

Magnetometers measure the magnetic field strength and direction of the Earth's magnetic field. Similar to accelerometers, they can output data with no cumulative error, are accurate over longer time scales, and can get yaw angles. But its disadvantage is that the small-scale measurement is not accurate enough. For objects such as a stylus with a relatively small moving distance, it cannot provide sufficiently high-precision measurement results and is susceptible to interference.

Therefore, the present invention uses the accelerometer and the magnetometer to correct the measurement data of the gyroscope. For example, because the measurement data of the gyroscope is more accurate in a short period of time, but has accumulated errors, it is not accurate in a long period of time; on the contrary, the measurement data of the magnetometer and accelerometer in a short period of time are not accurate, but not Cumulative error, and more accurate over a longer period of time. Based on this fact, data fusion is performed on the measurement data of the three, so as to integrate the advantages of the three.

The track input device of the present invention will be further described in detail below in conjunction with FIG. 2 .

(a) of FIG. 2 is a front view of a stylus according to one embodiment of the trajectory input device of the present invention. (b) of FIG. 2 is a back view of the stylus of this embodiment.

It should be noted that the track input device of the present invention is not limited to a stylus for two-dimensional movement, but can also be any track input device for three-dimensional movement.

As shown in (a) of Figure 2, the stylus 20 includes: a penholder 201 with an internal space; a nib 202 at one end of the penholder 201; the attitude sensor includes a gyroscope 206a, an accelerometer 206b and a magnetometer 207, The displacement sensor includes a photoelectric sensor 208 , and the attitude includes the pitch angle, roll angle and yaw angle of the stylus 20 .

A gyroscope 206 a , an accelerometer 206 b , a magnetometer 207 , and a photosensor 208 are accommodated in the inner space of the pen barrel 201 coaxially with the stylus 20 .

The processing unit 103 obtains a posture transition matrix according to the posture of the stylus 20 , and uses the posture transition matrix to correct the moving track of the pen tip 202 on the input medium.

In addition, a power supply unit 203 and a power management unit 204 are installed in the inner space of the pen holder 201 . A touch switch 205 is arranged near the nib 202 for identifying whether the nib touches the paper and whether writing has started.

Here, the power supply unit 203 may be a 3.7V lithium battery for powering the entire stylus. The power management unit 204 can be a chip Tp4056, which is used for charging and managing the lithium battery.

In addition, the functions of the accelerometer 206b and the gyroscope 206a can be integrated together, such as the chip Mpu6050. The magnetometer 207 is, for example, HMC5883L. Here, the gyroscope 206 a acquires the three-axis angular velocity of the stylus 20 , the accelerometer 206 b acquires the three-axis acceleration of the stylus 20 , and the magnetometer 207 acquires the three-axis magnetic field intensity of the stylus 20 .

In addition, the photoelectric sensor 208 is located in the inner space of the pen holder 201 close to the pen tip 202 and is used to obtain the movement track of the pen tip 202 . The photoelectric sensor 208 irradiates the surface of the input medium with the light of the LED (not shown), and the reflected light is collected and imaged by the lens 209. By comparing the images before and after the movement, the moving direction and the moving distance of the nib 202 on the input medium can be determined, thereby determining moving track.

It should be noted that the gyroscope 206a, accelerometer 206b, magnetometer 207 and photoelectric sensor 208 must be installed on the central axis of the pen holder 201, that is, coaxial with the pen holder 201, as shown in FIG. 2 . Moreover, the distance between the photoelectric sensor 208 and the pen tip 202 should be determined according to the depth of field of the lens 209 . Because only the movement within the depth of field of the lens can be clearly imaged on the surface of the photoelectric sensor, thereby detecting the movement.

As shown in (b) of Figure 2, the internal space of the penholder 201 is also installed with: a turn-over chip 212, which can be an RT9193, used to step down the voltage of the power supply unit 203 to 3.3V to supply power for the entire stylus; Unit 211, configured to store and/or send detected data. Specifically, the data acquisition unit 211 may include a TF card slot and/or a communication module, wherein the TF card slot is used to store the data collected by the attitude sensor and the displacement sensor into the TF card slot, and the communication module is used to send the data Save it to the host computer in the cloud. The communication module may be a Bluetooth module.

In addition, the processing unit 103 installed in the inner space of the penholder 201 is used to control the above-mentioned components as a whole and perform data processing.

When the stylus 20 writes on the writing medium (not shown), the gyroscope 206a, the accelerometer 206b, and the magnetometer 207 acquire the pen posture of the stylus 20, and the photoelectric sensor 208 acquires the moving track of the pen tip 202 on the writing medium. The processing unit 103 uses the posture transfer matrix to correct the moving track of the pen tip 202 on the writing medium to obtain accurate input data.

The input method of the present invention will be described below with reference to FIG. 3 .

FIG. 3 is a flow chart of the trajectory input method of the trajectory input device of the present invention.

Under the condition that the trace input device is the stylus 20, the trace input method starts with the processing unit 103 judging whether the pen tip 202 touches the input medium according to the signal of the touch switch 205 (step S30). If yes, the output data of the attitude sensors 206a, 206b, and 207 of the stylus 20 are acquired (step S31).

In step S31, the values of the 3-axis angular velocity, 3-axis acceleration and 3-axis magnetic field intensity output by the sensors 206a, 206b, 207 are read through the IIC data protocol.

Three-axis angular velocity refers to the angular velocity of rotation around the penholder 201 direction (set as the z axis), and the angular velocity of the rotation around two mutually orthogonal directions (set as x and y axes) in the plane perpendicular to the penholder 201 .

The triaxial magnetic field strength refers to the magnetic field strength along the above-mentioned three axes of x, y, and z.

The 3-axis acceleration refers to the acceleration along the above-mentioned three axes of x, y, and z.

The attitude information of the stylus 20 can be obtained through the 3-axis angular velocity, 3-axis acceleration and 3-axis magnetic field intensity acquired by the sensors 206 a , 206 b and 207 .

After acquiring the output data of the posture sensor (step S31), the processing unit 103 corrects the acquired output data (step S32).

The reason for data correction is that there is a certain error in the data output by the sensor. First, the basic data of each sensor needs to be corrected individually to obtain a relatively accurate data before it can be processed.

The method of data correction for the 3-axis magnetic field strength may be the method of least squares. A data correction method for the 3-axis angular velocity and 3-axis acceleration may be to subtract the offset. Each will be described below.

(1) Regarding the 3-axis magnetic field strength

Data fitting based on the principle of the least square method is a method to correct the magnetic field strength data, which can make the processed data more accurate.

According to the error model and collecting a large amount of data, the coefficients in the model can be fitted offline as a correction model.

Next, use the yalmip toolbox of matlab to do data fitting according to the least square method.

(2) Regarding 3-axis angular velocity

The zero bias is obtained in the static state, and it is only necessary to eliminate the influence of the zero bias.

(3) About 3-axis acceleration

It is the same as the 3-axis angular velocity, only need to eliminate the influence of zero bias.

After correcting the acquired output data (step S32), the processing unit 103 calculates the gesture data of the stylus (step S33).

In step S33, the processing unit 103 obtains the attitude of the stylus 20 by fusing the data obtained by the accelerometer 206b and the magnetometer 207 with the data obtained by the gyroscope 206a.

For example, the processing unit 103 uses a complementary filtering algorithm to perform the fusion described above.

Thus, the pen posture data of the stylus 20 is obtained.

After calculating the attitude data of the stylus (step S33), the processing unit 103 determines an attitude transition matrix according to the pitch angle, roll angle and yaw angle (step S34).

In step S34, the processing unit 103 can use the strapdown inertial navigation algorithm to determine the attitude transition matrix shown in the following equation (1), where θ, φ and ψ represent the pitch angle, roll angle and heading angle of the stylus 20, respectively.

(Formula 1)

Then, the processing unit 103 acquires the output data of the displacement sensor 102 (step S35). Here, the displacement sensor 102 is, for example, a photoelectric sensor 208, which acquires the displacement of the pen head 202 in the x direction and the y direction. Since the stylus 20 only moves two-dimensionally on the writing medium such as paper, the photoelectric sensor 208 only acquires displacements in two directions. For a trajectory input device that performs three-dimensional movement, the displacement sensor needs to obtain displacements in the three directions of x, y, and z.

Finally, the processing unit 103 corrects the moving track of the pen tip 202 on the writing medium by using the posture transition matrix (step S36 ).

After step S36, the processing unit 103 judges again whether the pen tip 202 touches the input medium, if not, suspends the processing until the pen tip 202 touches the input medium again, and then continues to repeat the operations from step S30 to step S36.

Assuming that the displacement of the nib 202 obtained by the photoelectric sensor 208 is (x, y, z), the corrected displacement (x', y', z') is shown in the following formula (2):

(Formula 2)

Thus, more accurate input data can be obtained by correcting the posture change of the stylus 20 during the writing process.

It should be noted that in FIG. 3 , the output data of the attitude sensor 101 is obtained first, and then the output data of the displacement sensor 102 is obtained. However, the present invention is not limited to this order, and the output data of the displacement sensor 102 may be acquired first, and then the output data of the attitude sensor 101 may be acquired.

In the present invention, complementary filtering is used for data fusion. The weight coefficient needs to be changed according to different occasions to make the error smaller. According to the reaction situation of accelerometer, gyroscope and magnetometer to different measurement times, the measurement of angle by accelerometer 206b and magnetometer 207 is accurate for a long time, and its weight coefficient can be increased, and the measurement of gyroscope 206a in a short period of time is accurate, Its weight coefficient can be increased.

Complementary filtering will be described in detail below in conjunction with FIG. 4 .

Fig. 4 shows a schematic diagram of the complementary filtering of the present invention.

Let the data output by the magnetometer 207 be m _x , my and _{m z ,} the data output by the gyroscope 206 a be (p,q, r) ^T , and the data output by the accelerometer 206 b be a _x , a _y and a _z . The pitch angle, roll angle and heading angle of the stylus 20 are denoted as θ, φ and ψ respectively.

According to the working principle of the gyroscope 206a, the following differential equation can be known:

(Formula 3)

in, Respectively represent the angular velocity of roll angle, pitch angle and heading angle.

In addition, according to the working principle of the accelerometer 206b, the following formula can be known:

(Formula 4)

Wherein, θ _acc and φ _acc are the pitch angle and roll angle estimated by the accelerometer 206b, respectively.

Then, the output value of the magnetometer 207 is compensated by using the pitch angle and roll angle obtained in formula 4, and the heading angle estimated by the magnetometer 207 can be obtained:

(Formula 5)

Then, complementary filtering is performed on the pitch angle and roll angle estimated by the accelerometer 206 b and the pitch angle and roll angle estimated by the gyroscope 206 to obtain the fused pitch angle θ and roll angle φ.

On the other hand, complementary filtering is performed on the heading angle estimated by the magnetometer 207 and the heading angle estimated by the gyroscope 206 to obtain the fused heading angle ψ.

As mentioned above, the rule for performing complementary filtering is: increase the weight coefficients of the accelerometer 206b and the magnetometer 207 for a long time, and increase the weight coefficient of the gyroscope 206a for a short time. In the embodiment of the present invention, the long time and short time can be determined according to the control period of the stylus 20 by the processing unit 103, for example: take one control period of the processing unit 103 as the short time, and take 20 control periods of the processing unit 103 as the short time. The processing cycle is long time, the control cycle can be 6ms, then the long time is 120ms, and the short time is 6ms. The specific long time, short time and weight coefficient can be adjusted through experiments.

Alternatively, the processing unit 103 can also use the Kalman filter algorithm to fuse the data obtained by the gyroscope 206a, the accelerometer 206b and the magnetometer 207 to obtain the fused roll angle, pitch angle and yaw angle of the stylus 2. flight angle.

The trajectory input device and trajectory input method of the present invention are described in detail above by taking the stylus 20 as an example. With the stylus 20 of the present invention, it is possible to write on any medium anytime and anywhere, and accurately obtain the movement track of the stylus 20 on the writing medium, thereby inputting accurate information to a computer or the like.

In addition, in the case of the stylus 20, the input medium is paper. However, the present invention is not limited to a two-dimensional input medium such as paper, but may be any three-dimensional input medium.

The above embodiments are only exemplary embodiments adopted to illustrate the principles of the present invention, but the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention. These variations and improvements are also regarded as the protection scope of the present invention.

Claims (12)

1. A trajectory input device (10), comprising:

attitude sensor (101) comprising: a gyroscope (206a) for obtaining an angle of the trajectory input device (10); an accelerometer (206b) for obtaining roll and pitch angles of the trajectory input device (10); and a magnetometer (207) for obtaining a yaw angle of the trajectory input device (10), and correcting data obtained by the gyroscope (206a) using data obtained by the accelerometer (206b) and the magnetometer (207) to obtain an attitude of the trajectory input device (10);

a displacement sensor (102) for acquiring a movement track of the track input device (10) when moving on an input medium, wherein the movement track is displacement in each direction of space;

and a processing unit (103) that corrects the movement trajectory of the trajectory input device (10) on an input medium, which is acquired by the displacement sensor (102), using the attitude of the input device (10) acquired by the attitude sensor (101).

2. The trace input device (10) as claimed in claim 1, wherein the trace input device is a stylus (20) and the displacement sensor comprises a photosensor (208).

3. The trajectory input device (10) of claim 2, wherein the stylus (20) includes: a barrel (201) having an inner space; a pen point (202) positioned at one end of the pen holder (201); wherein,

the gyroscope (206a), the accelerometer (206b), the magnetometer (207) and the photosensor (208) are housed in the internal space of the pen barrel (201) and coaxial with the stylus (20); and is

The processing unit (103) obtains a posture transfer matrix according to the posture of the stylus (20), and corrects the movement track of the pen tip (202) on the input medium by using the posture transfer matrix, wherein the correction comprises

Obtaining the posture of the stylus (20) by fusing data acquired by the accelerometer (206b) and the magnetometer (207) and data acquired by the gyroscope (206 a).

4. The trajectory input device (10) as set forth in claim 3, wherein the processing unit (103) performs the fusion using a complementary filtering algorithm.

5. The trajectory input device (10) according to claim 3, wherein the processing unit (103) performs the fusion using a Kalman filtering algorithm.

6. The trajectory input device (10) of any of claims 1-5, wherein the input medium is paper.

7. A trajectory input method using the trajectory input device (10) of claim 1, comprising the steps of:

step a, when the track input device (10) moves on an input medium, acquiring the posture of the track input device and the movement track of the track input device, wherein the movement track is displacement in each direction of space;

and b, correcting the moving track of the track input device (10) on the input medium by using the posture of the track input device (10).

8. The trajectory input method as set forth in claim 7, wherein the trajectory input device (10) is a stylus (20), and the displacement sensor includes a photosensor (208).

9. The trajectory input method according to claim 8, wherein the stylus (20) includes: a barrel (201) having an inner space; a pen point (202) positioned at one end of the pen holder (201); wherein,

the gyroscope (206a), the accelerometer (206b), the magnetometer (207) and the photosensor (208) are housed in the internal space of the pen barrel (201) and coaxial with the stylus (20); and is

The step b comprises the following steps:

the processing unit (103) obtains a posture transfer matrix according to the posture of the stylus (20), and corrects the moving track of the pen point (202) on the input medium by using the posture transfer matrix, wherein the correction comprises

Obtaining the posture of the stylus (20) by fusing data acquired by the accelerometer (206b) and the magnetometer (207) and data acquired by the gyroscope (206 a).

10. Trajectory input method according to claim 9, wherein said processing unit (103) performs said fusion by using a complementary filtering algorithm.

11. The trajectory input method according to claim 9, wherein the processing unit (103) performs the fusion using a kalman filter algorithm.

12. The trajectory input method according to any one of claims 7 to 11, wherein the input medium is paper.