CN105816181A - Reading behavior identification method and device based on EOG - Google Patents

Abstract

The invention relates to a reading behavior recognition method and equipment based on EOG, wherein the recognition method comprises the following steps of collecting EOG signals of a person to be detected and preprocessing the collected EOG signals; identifying a starting point and an end point of a reading state of the preprocessed EOG signal through end point detection; carrying out character string coding on the obtained reading EOG signal by a wavelet packet change method; and realizing character string matching through the editing distance between the code character string and the template character string to obtain a recognition result. The method provided by the invention is used as an auxiliary method for reading disorder diagnosis, and has the advantages of high recognition accuracy, strong expansibility, high potential application value and the like.

Description

EOG-based reading behavior recognition method and equipment

technical field

The invention relates to the technical field of electro-oculogram (EOG), in particular to an EOG-based reading behavior recognition method and device.

Background technique

Modern civilization mainly uses words as the carrier, and reading is an important way to acquire knowledge, and the development of its ability is the basis for the development of other learning abilities. Dyslexia refers to a persistent difficulty in acquiring normal reading and writing skills under the premise of normal IQ, learning motivation and equal educational opportunities. Dyslexia generally occurs in adults and children, especially school-age children, and its incidence is about 5%-10%. Dyslexia will not only affect the improvement of children's academic performance, but also affect the development of children's self-confidence and social skills, and bring greater harm to children's emotional and social development. Therefore, how to effectively identify and diagnose dyslexia, help children with dyslexia to overcome reading problems as much as possible, improve reading ability, and develop good reading habits is of great practical significance for improving national cultural literacy.

At present, the diagnosis of dyslexia mainly adopts the diagnostic mode based on cognition and behavior. Usually, in the process of diagnosis, in order to improve the accuracy, the doctor will conduct a video analysis of the patient's reading behavior according to the preset reading task. Auxiliary analysis, including obtaining relevant information such as eye movement trajectory, reading duration, and reading speed, to judge the patient's reading quality. However, although the traditional video-based reading behavior analysis method is more convenient to use, it is greatly affected by light. When the light is dark or the background environment changes, the system performance will drop sharply, and even the analysis cannot be done correctly.

Contents of the invention

The object of the present invention is to provide an EOG-based reading behavior recognition method and device with high recognition accuracy, strong expansion ability and great application potential in order to avoid the shortcomings in the above-mentioned prior art. This method uses EOG as the detection method, which can realize the identification of the reading state of the subject to be tested, and assist the doctor to judge the degree of dyslexia of the patient.

A kind of reading behavior identification method based on EOG, this identification method comprises the following steps:

A), collect the horizontal EOG signal of the person to be detected, and preprocess the collected EOG signal, and determine the template character string and the error threshold of the edit distance according to the number of words in each row in the reading content;

B), performing endpoint detection on the preprocessed horizontal EOG signal, to determine the starting point and the ending point of the horizontal EOG signal corresponding to the reading state;

C), the reading EOG signal obtained in step B is encoded into a series of continuous character strings corresponding to eye movements by wavelet packet transform;

D), calculate the editing distance between the continuous character string obtained in step C and the preset template character string in step A, when the distance is less than the preset error threshold in step A, it is determined that the person to be detected is reading state, otherwise, it is a non-reading state.

An EOG-based reading behavior recognition device as claimed in claim 1, the device includes a signal acquisition and preprocessing module, and the EOG signal acquisition and preprocessing module is used to collect EOG signals and bandpass filter the EOG signals Operation; and output the EOG signal after bandpass filtering to the reading endpoint detection module;

The reading endpoint detection module is used to identify the starting point and the ending point of the reading state in the EOG signal through a combination of differential and energy methods; and output the EOG signal to the reading signal encoding module;

The reading signal encoding module is used to encode the reading EOG signal into a character string by the wavelet packet transform method; the encoded character string is output to the reading behavior identification module;

The reading behavior recognition module is used to measure the similarity between the coded string and the template string by editing distance to realize string matching, and judge whether the person to be detected is in a reading state.

Compared with the prior art, the technical effect that the present invention possesses is:

The invention preprocesses the collected original horizontal EOG signal by denoising and filtering, and then obtains the starting point and ending point of the reading state in the EOG signal through the endpoint detection method, and then realizes reading through steps such as wavelet packet transformation, threshold setting, and string encoding. Encoding of EOG signal, get the encoded string of the signal, and finally achieve string matching by editing the distance to measure the similarity between the template string and the encoded string, and judge whether the person to be detected is in the reading state, the recognition accuracy rate is high, and through the recognition The redesign of the template and a small amount of adjustment of the algorithm parameters can realize the identification of various behavior states, which has strong scalability. In addition, the present invention uses EOG means to detect reading behavior, which can effectively overcome the shortcomings of traditional video methods, and may replace traditional video detection methods, or as an important supplement to traditional video detection methods, it has a role in the diagnosis of dyslexia. High theoretical research significance and potential application value.

Description of drawings

Fig. 1 is the electrode distribution figure that is used for EOG signal acquisition among the present invention;

Fig. 2 is the horizontal EOG signal under the reading state in the present invention;

The logical block diagram of the reading recognition method in Fig. 3 the present invention;

Fig. 4 is the value of increment 1 and ten groups of new thresholds among the present invention;

Fig. 5 uses ten groups of new thresholds in the present invention to carry out reading state recognition to obtain optimal threshold value;

Fig. 6 is the logical block diagram of reading endpoint detection module in the present invention;

7 is a schematic diagram of the process steps of the endpoint detection method of the EOG signal under the natural reading paradigm in the present invention;

Fig. 8 is a schematic diagram of wavelet packet decomposition in the present invention;

Fig. 9 is the process of reading EOG signal encoding in the present invention;

Fig. 10 is a schematic diagram of the identification process of the reading state using the character string matching method in the present invention;

Fig. 11 is the display of the reading recognition results of the belt detector in the present invention.

detailed description

The present invention is further described in conjunction with Fig. 1 to Fig. 11:

A kind of reading behavior identification method based on EOG, this identification method comprises the following steps:

A), collect the horizontal EOG signal of the person to be detected, and preprocess the collected EOG signal, and determine the template character string and the error threshold of the edit distance according to the number of words in each row in the reading content;

B), performing endpoint detection on the preprocessed horizontal EOG signal, to determine the starting point and the ending point of the horizontal EOG signal corresponding to the reading state;

C), the reading EOG signal obtained in step B is encoded into a series of continuous character strings corresponding to eye movements by wavelet packet transform;

D), calculate the editing distance between the continuous character string obtained in step C and the preset template character string in step A, when the distance is less than the preset error threshold in step A, it is determined that the person to be detected is reading state, otherwise, it is a non-reading state.

The present invention adopts an EOG-based reading behavior recognition method. The method first uses three bioelectrode sensors to collect the original horizontal EOG signal, and performs denoising and filtering preprocessing on the signal, and then obtains the starting point of the reading state in the EOG signal through the endpoint detection method. and the termination point, and then realize the encoding of reading EOG signal through steps such as wavelet packet transformation, setting threshold, and string encoding, and obtain the encoded string of the signal, and finally realize the string by editing the distance to measure the similarity between the template string and the encoded string Matching, judging whether the subject is in the reading state, and the recognition accuracy is high;

Moreover, the present invention uses a string matching method when realizing reading behavior recognition, that is, calculates the edit distance between the encoded string in the reading state and the template string, and measures the similarity between the strings by the edit distance, and To judge whether it is in the reading state, the recognition of various behavior states can be realized by redesigning the recognition template string and a small amount of adjustment of algorithm parameters, which has strong scalability;

In addition, the present invention uses EOG means to detect reading behavior, which can effectively overcome the shortcomings of traditional video methods, and may replace traditional video detection methods, or as an important supplement to traditional video detection methods, it has a role in the diagnosis of dyslexia. High theoretical research significance and potential application value.

In the step A, the horizontal EOG signal of the subject to be detected is collected through three bio-electrodes, and the EOG signal is preprocessed with a 32-order band-pass filter with a cutoff frequency of 0.05 Hz to 15 Hz.

In the step B, the differential and energy method is used to detect the endpoint of the preprocessed EOG signal, and the specific steps of the differential and energy method are:

a) Perform framing and windowing processing on the preprocessed EOG signal, the window length of the preprocessed EOG signal is 1000, and the window shift is 1 (taking the data sampling rate of 250Hz as an example, the window length is set to 1000 sample points, the window shift is set to 1 sample point, and the initial differential and energy thresholds F0 and E0 are set according to experience;

b) Calculate the differential value F in the current sliding window, and compare F with the derivative limit F0; if F>F0, then this endpoint is regarded as the "reading possible starting point Si" of the signal; otherwise, the sliding window continues backward slide;;

c), calculate the energy value E of the signal in the current sliding window, and compare it with the energy threshold E0; if E<E0, then the terminal is the "reading possible termination point Ti" of the signal; otherwise, the sliding window continues slide back;

d) Calculate the number X of EOG signal sample points contained between the "possible reading start point Si" and the "possible reading end point Ti". When X>1000, that is, when the sampling rate is 250Hz, the starting point and the ending point If the interval is greater than 4 seconds, the "reading possible starting point Si" is the reading starting point, and the "reading possible ending point Ti" is the reading ending point, otherwise the signal is determined to be in a non-reading state;

e) The sliding window continues to slide, repeating steps b to d until the signal ends.

In the step C, the wavelet packet generating function in the wavelet transform step of reading the EOG signal is a Haar function, the number of decomposition layers is 3 layers, and the third coefficient is selected as the best wavelet packet from the wavelet packet coefficients obtained by decomposition Coefficient C.

The threshold values used to set the limit steps for reading EOG signals are set as: S ₁ , S ₂ , L ₁ , L ₂ , and according to these four threshold values, the optimal wavelet packet coefficient C is divided as follows:

Non-saccade area: S ₂ <C < S ₁

Small glance area: S ₁ <C<L ₁ or L ₂ <C<S ₂

Large saccade area: C>L ₁ or C<L ₂

In the step of reading the EOG signal for string encoding, when the optimal wavelet packet coefficient C is divided into the "large glance area", it is encoded as L; when the optimal wavelet packet coefficient C is divided into the "small glance area" , which is coded as r; and when the optimal wavelet packet coefficient C is divided into the "non-saccade area", it is not coded.

The initialization method of the thresholds S ₁ , S ₂ , L ₁ , and L ₂ is:

Step 1: Manually observe the value of the optimal wavelet packet coefficient C, and set the initial threshold value as: 5, -5, 60, -60 according to the observation results, that is, S ₁ =5, S ₂ =-5, L ₁ = 60, L ₂ = -60.

Step 2: Use the variable I to represent the increment of the threshold, where the values of the increment are I=1, 3, 5, 7, . . . , 19 respectively. Ten sets of new thresholds are obtained by adding different I values to the initial threshold, and then the ten sets of new thresholds are used to test multiple sets of read EOG data.

Step 3: Use ten sets of new threshold values to calculate the reading recognition accuracy rate of reading EOG data, and select the group with the highest accuracy rate from the obtained ten reading recognition accuracy rates, and the threshold value corresponding to this group is the optimal threshold.

In the step D, the process of matching the character string is to calculate the edit distance between the encoded character string and the template character string, which is used to judge whether the person to be detected is in a reading state, and if the edit distance is less than or equal to 1, it means that the encoded character If the string is correctly identified, the person to be detected is in a reading state; otherwise, it means that the person to be detected is in a non-reading state.

See Figure 1. Ag/AgCl electrodes are used for the collection of EOG signals. A total of 3 electrode sensors are used. The electrode H used to collect horizontal EOG signals can be installed at the outer edge of the left eye (or right eye) at a distance of 4.5 from the center of the pupil of the eye. cm; the ground electrode G was installed at the position of the mastoid behind the right ear, and the reference electrode C was installed at the position of the left mastoid respectively.

Referring to Figure 2, it illustrates the original waveform of the horizontal EOG signal collected in this example. Since the horizontal EOG signal can better reflect the main eye movement characteristics of the person to be detected when reading, this patent uses the horizontal EOG signal as the input signal of the algorithm. In Figure 2, S ₁ represents a small saccade, that is, the continuous reading process to the right, S ₂ represents a large left saccade generated when changing a line, F represents the gaze state during reading, and B represents the blink state during reading.

Referring to Fig. 3, a kind of equipment of reading behavior recognition based on EOG as claimed in claim 1, equipment comprises signal collection and preprocessing module 10, and described EOG signal collection and preprocessing module 10 are used for collecting EOG signal and pairing The EOG signal is band-pass filtered; and the band-pass filtered EOG signal is output to the reading endpoint detection module 20;

The read endpoint detection module 20 is used to identify the start point and end point of the read state in the EOG signal by a method combining differentiation and energy; and output the EOG signal to the read signal encoding module 30;

The reading signal encoding module 30 is used to encode the reading EOG signal into a character string by the wavelet packet transform method; the encoded character string is output to the reading behavior recognition module 40;

The reading behavior recognition module 40 is used to measure the similarity between the encoded character string and the template character string by editing distance to achieve character string matching, and determine whether the person to be detected is in a reading state. .

Referring to Fig. 4, ten new increments I are obtained by the incremental method, and the four initial thresholds S ₁ =5, S ₂ =-5, L ₁ =60, L ₂ =-60 are respectively combined with the ten new increments I Add up to get ten new threshold values.

Referring to Fig. 5, in the present embodiment, ten groups of new threshold values are used to identify and detect the reading EOG data of six persons to be detected. The average reading recognition rate of all collected reading EOG data can be seen from the figure, when the increment I increases continuously, the reading recognition rate of the 6 persons to be tested increases; when I increases to 13, the reading recognition rate increases. The recognition rate reached the maximum value, and the average reading recognition rate of the six subjects reached a maximum of 0.9678 at this time; but when I continued to increase, the reading recognition rate showed a downward trend instead. Therefore, when the threshold increment I is 13, that is, when the four threshold values are S ₁ =18, S ₂ =-18, L ₁ =73, and L ₂ =-73, the reading recognition rate reaches the highest, so the four thresholds The limit is the optimal threshold value identified by the algorithm.

Referring to Fig. 6, since the original EOG signal contains a reading state and a non-reading state, it is necessary to detect the reading state contained in the signal by endpoint detection before performing the reading state identification operation on it. The method mainly includes the following steps:

1) Frame and window the EOG signal (the window function is a Hamming window, the window length is 1000, and the window shift is 1), and set the initial value of differential and energy;

2) Calculate the differential value of each frame signal and compare it with the initial differential value to obtain the "reading possible starting point";

3) Calculate the energy value of each frame signal and compare it with the initial energy value to obtain the "possible end point of reading".

4) Calculate the interval between "reading possible starting point" and "reading possible ending point", compare it with the set interval (1000 sample points), if x>1000, determine the reading starting point and end point point, otherwise repeat steps 2-4

Referring to Figure 7, it illustrates the graph of the endpoint detection result of the EOG signal in this example, Figure 7(a) shows a random section of natural reading EOG signal, Figure 7(b) shows the differential of the signal, and Figure 7(c) shows the energy of the signal .

Referring to Fig. 8, Fig. 8(a) is a section of randomly selected reading EOG signal, and Fig. 8(b)-(o) is the decomposition coefficient diagram of wavelet packet, wherein Fig. 8(b) and Fig. 8(c) represent the first Layer wavelet packet decomposition coefficients, Figure 8(d)-(g) shows the second layer wavelet packet decomposition coefficients, Figure 8(h)-(o) shows the third layer wavelet packet decomposition coefficients, which can be obtained from the figure, the third layer The wavelet packet coefficient represented by the second node in the decomposition coefficient (as shown in Figure 8(i)) can clearly distinguish the glance signal, and this coefficient is called the optimal wavelet packet coefficient C.

Referring to Fig. 9, Fig. 9(a) is a section of preprocessed reading EOG signal, and Fig. 9(b) is an optimal wavelet coefficient C selected after wavelet packet decomposition of the EOG signal, and C is set to four detection thresholds S ₁ , S ₂ , L ₁ , and L ₂ , so that the EOG signal is divided into three glance areas. Figure 9(c) encodes the string for reading the EOG signal, and finally reads the EOG signal with a continuous string containing "L" and "r" character strings.

Referring to Figure 10, the edit distance between the encoded string and the template string is calculated to achieve string matching. If the edit distance is less than or equal to 1, it means that the encoded string is correctly identified and the person to be detected is in the reading state; otherwise, The person to be detected is in a non-reading state, so that it is convenient to detect the reading ability of the person to be detected.

Referring to Figure 11, it illustrates that in this example, the data of 6 persons to be tested are used to verify the reading recognition algorithm, in which each person to be tested collects 45 sets of reading data, and the collection time of each set of reading data is 5 minutes, as shown in the figure The abscissa table is the number of each subject to be tested, and the ordinate is the average reading recognition rate of all data collected by each subject to be tested. Among them, the average recognition rate of reading EOG data of No. 1 subject to be tested is the lowest at 81.14%, while The average recognition rate of candidate No.5 reached 93.72%, and the average recognition rate of 6 candidates reached 90.06%.

Claims (7)

1. a reading behavior recognition methods based on EOG, it is characterised in that: this recognition methods comprises the following steps:

A), gather the horizontal EOG signal of person to be detected, and the EOG signal collected is carried out pretreatment, according to the number of words often gone in reading content, determine the error threshold of template character string and editing distance；

B), pretreated horizontal EOG signal is carried out end-point detection, to determine the starting point and ending point of horizontal EOG signal corresponding to read state；

C) the reading EOG signal obtained, in stepb, is encoded into a series of continuation character strings corresponding with ocular movement by wavelet package transforms；

D), in calculation procedure C obtained by continuation character string and step A in editing distance between the template character string preset, when the error threshold preset during this distance is less than step A, assert that person to be detected is in read state, otherwise, for non-read state.

2. according to the reading behavior recognition methods based on EOG described in claim 1, it is characterized in that: in described step A, gathered the horizontal EOG signal of person to be detected by 3 bioelectrodes, and EOG signal is carried out the 32 rank bandpass filtering pretreatment that cut-off frequency is 0.05Hz to 15Hz.

Reading behavior recognition methods based on EOG the most according to claim 1 and 2, it is characterised in that: in described step B, using differential and energy method that pretreated EOG signal is carried out end-point detection, differential concretely comprises the following steps with energy method:

A), to pretreated EOG signal carry out framing and windowing process, and original differential and energy threshold F0 and E0 are rule of thumb set；

B), calculate the differential value F of signal in current sliding window, F is compared with differential thresholding F0；If F is ＞ F0, then this end points is considered as " reading possible starting point Si " of signal, otherwise, then sliding window continues to slide backward；

C), calculate the energy value E of signal in current sliding window, and it is compared with energy threshold E0；If E is ＜ E0, then this end electricity is " reading possible terminating point Ti " of signal, otherwise, then sliding window continues to slide backward；

D) comprised EOG signal sample point number X between " reading possible starting point Si " and " reading possible terminating point Ti ", is calculated, as X ＞ 1000, i.e. when sample rate is 250Hz, between starting point and terminating point, interval was more than 4 seconds, " read possible starting point Si " for reading starting point, " read possible terminating point Ti " and, for reading terminating point, otherwise judge that this segment signal is in non-read state；

E), sliding window continue to slide backward, repeat step b to d operation until signal ended.

Reading behavior recognition methods based on EOG the most according to claim 1, it is characterized in that: in described step C, it is Haar function to read EOG signal carrying out the wavelet packet generating function in wavelet package transforms step, Decomposition order is 3 layers, and obtain wavelet packet coefficient is chosen the 3rd coefficient as Optimal Wavelet Packet coefficient C from decomposition, and by pre-determined threshold S₁、S₂、L₁、L₂Gained Optimal Wavelet Packet coefficient C is divided as follows:

Non-glance district: S₂＜ C ＜ S₁

Little pan district: S₁＜ C ＜ L₁Or L₂＜ C ＜ S₂

Sweep greatly district: C ＞ L₁Or C ＜ L₂

Carry out in string encoding step reading EOG signal, when Optimal Wavelet Packet coefficient C is divided in " sweeping greatly district ", be encoded to L；When Optimal Wavelet Packet coefficient C is divided in " little pan district ", it is encoded to r；And Optimal Wavelet Packet coefficient C is when being divided in " non-glance district ", do not encode.

Reading behavior recognition methods based on EOG the most according to claim 4, it is characterised in that: described thresholding S₁, S₂, L₁, L₂Initial method is:

Step 1: the value of artificial observation Optimal Wavelet Packet coefficient C, and according to observed result, initial threshold is set as: 5 ,-5,60 ,-60, i.e. S₁=5, S₂=-5, L₁=60, L₂=-60；

Step 2: using variable I to represent the incremental change of thresholding, wherein the value of increment is respectively I=1,3,5,7 ..., 19.It is worth to ten groups of new threshold values by initial threshold numerically being increased different I, then is respectively adopted ten groups of new threshold values to how group reading EOG data are tested；

Step 3: be respectively adopted ten groups of new threshold values, in order to calculate the reading recognition correct rate reading EOG data, and reads recognition correct rate from the ten of gained and selects the group that accuracy is the highest, and threshold value corresponding to this group is optimum gate limit value.

Reading behavior recognition methods based on EOG the most according to claim 1, it is characterized in that: in described step D, editing distance between calculation code character string and template character string, for judging whether person to be detected is in read state, if editing distance is less than or equal to 1, then representing that this coded string is correctly validated, person to be detected is in read state；Otherwise, it means that person to be detected is in non-read state.

7. the equipment of a reading behavior identification based on EOG as claimed in claim 1, it is characterized in that: equipment includes that signals collecting and pretreatment module (10), described EOG signal collection and pretreatment module (10) are used for being acquired EOG signal and EOG signal being carried out bandpass filtering operation；And the EOG signal after bandpass filtering is exported to reading endpoint detection module (20)；

Described reading endpoint detection module (20) identifies the starting point and ending point of the read state in EOG signal for the method combined by differential and energy；And by EOG signal output to reading Signal coding module (30)；

Described reading Signal coding module (30) is for being encoded to character string by wavelet package transforms method by reading EOG signal；Character string after coding exports to reading behavior identification module (40)；

Described reading behavior identification module (40) is for realizing string matching by the similarity between editing distance tolerance coded string and template character string, it is judged that whether person to be detected is in read state.