CN113842113B - Intelligent identification method, system, device and storage medium for developmental dyslexia - Google Patents
Intelligent identification method, system, device and storage medium for developmental dyslexia Download PDFInfo
- Publication number
- CN113842113B CN113842113B CN202110831632.2A CN202110831632A CN113842113B CN 113842113 B CN113842113 B CN 113842113B CN 202110831632 A CN202110831632 A CN 202110831632A CN 113842113 B CN113842113 B CN 113842113B
- Authority
- CN
- China
- Prior art keywords
- recognition
- eye movement
- recognition probability
- probability
- tested person
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/103—Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
- A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
- A61B3/10—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions
- A61B3/113—Objective types, i.e. instruments for examining the eyes independent of the patients' perceptions or reactions for determining or recording eye movement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
- A61B5/7235—Details of waveform analysis
- A61B5/7246—Details of waveform analysis using correlation, e.g. template matching or determination of similarity
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Veterinary Medicine (AREA)
- Physics & Mathematics (AREA)
- Biophysics (AREA)
- Public Health (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Physiology (AREA)
- Pathology (AREA)
- Ophthalmology & Optometry (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Human Computer Interaction (AREA)
- Psychiatry (AREA)
- Signal Processing (AREA)
- Dentistry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Eye Examination Apparatus (AREA)
Abstract
本发明公开了一种发展性阅读障碍智能识别方法、系统、设备及存储介质。所述发展性阅读障碍智能识别方法包括：获取被测者的眼动信号；将所述眼动信号输入到预设的第一识别模型中进行处理，得到所述被测者患有发展性阅读障碍的第一识别概率；从所述眼动信号中提取指定的特征数据，将所述特征数据输入到预设的第二识别模型中进行处理，得到所述被测者患有发展性阅读障碍对应的第二识别概率；将所述第一识别概率和所述第二识别概率进行融合，以得到所述被测者患有发展性阅读障碍的综合识别概率。通过利用浅层学习模型和深度学习模型的建模结果具有较强的互补性，利用模型融合技术对两种模型进行集成，保证了对被测者的发展性阅读障碍识别的准确性。
The present invention discloses a method, system, device and storage medium for intelligent identification of developmental dyslexia. The method for intelligent identification of developmental dyslexia includes: obtaining eye movement signals of a subject; inputting the eye movement signals into a preset first recognition model for processing to obtain a first recognition probability that the subject suffers from developmental dyslexia; extracting specified feature data from the eye movement signals, inputting the feature data into a preset second recognition model for processing to obtain a second recognition probability corresponding to the subject suffering from developmental dyslexia; fusing the first recognition probability and the second recognition probability to obtain a comprehensive recognition probability that the subject suffers from developmental dyslexia. By utilizing the modeling results of the shallow learning model and the deep learning model to have strong complementarity, the two models are integrated using model fusion technology to ensure the accuracy of identifying the subject's developmental dyslexia.
Description
Technical Field
The invention relates to the technical field of deep learning, in particular to an intelligent identification method, system, equipment and storage medium for a developmental dyskinesia.
Background
The developmental dyskinesia (developmental dyslexia, DD) refers to that the individual has no obvious difference from other individuals in general intelligence, motivation, living environment, educational condition and the like, has no obvious damage to vision, hearing and nervous system, but has a reading performance which is obviously lower than the average level of the same age. The developmental dysreading is a high-rise disorder in children and young people, the incidence rate is as high as 5% -12%, and more than 70% of dysreading children and young people have the problem of difficulty in learning in school.
In the diagnosis of developmental dysreading, hospitals do not have specialized diagnostic services because they are not considered clinical diseases and the pathogenesis is not yet defined. Diagnosis of this disorder typically requires the patient to be tested by a test meter and collect test data, which is then evaluated by a highly trained professional based on the test data and diagnostic experience. The existing diagnostic process of the developmental dyskinesia has a plurality of problems that the implementation technology is high in cost, the accuracy of the diagnostic result depends on the professional level of professionals, the accuracy is low, and the unified standard is lacking. Obviously, the diagnosis method cannot be used for screening in a wide children group.
Disclosure of Invention
The invention aims to overcome the defects that in the prior art, the development reading disorder needs to be manually diagnosed, the implementation technology cost is high, the accuracy of a diagnosis result depends on the professional level of professionals, the accuracy is low and unified standards are lacking, and provides an intelligent identification method, system, equipment and storage medium for the development reading disorder.
The invention solves the technical problems by the following technical scheme:
the invention provides an intelligent identification method for an expandable reading disorder, which comprises the following steps:
acquiring an eye movement signal of a tested person;
Inputting the eye movement signal into a preset first recognition model for processing to obtain a first recognition probability that the tested person suffers from the developing dyskinesia;
Extracting appointed characteristic data from the eye movement signals, inputting the characteristic data into a preset second recognition model for processing to obtain a second recognition probability corresponding to the person to be tested suffering from the progressive dyskinesia;
And fusing the first recognition probability and the second recognition probability to obtain the comprehensive recognition probability that the tested person suffers from the developing reading disorder.
Preferably, before fusing the first recognition probability and the second recognition probability, the intelligent identification method for the developmental dyskinesia further comprises:
Acquiring multi-modal data representing the reading capability of the subject;
Inputting the multi-mode data into a third recognition model for processing to obtain a third recognition probability that the tested person suffers from the developing dyskinesia;
the fusing of the first recognition probability and the second recognition probability to obtain a comprehensive recognition probability that the tested person suffers from the developing reading disorder comprises the following steps:
and fusing the first recognition probability, the second recognition probability and the third recognition probability to obtain the comprehensive recognition probability that the tested person suffers from the developing reading disorder.
Preferably, the first recognition model is a two-way long-short-time memory neural network;
Before inputting the eye movement signal into the first recognition model for processing, the method further comprises:
And uniformly segmenting the eye movement signal.
Preferably, the second recognition model is an integrated decision tree model.
Preferably, before the multi-modal data is input into the third recognition model for processing, the method further comprises:
the multi-modal data is preprocessed, the preprocessing including at least one of quantization processing, missing value filling processing, and normalization processing.
Preferably, the extracting the specified characteristic data from the eye movement signal includes:
At least one of the following characteristic data, namely coordinate position information of eye fixation to a screen, fixation duration and/or pupil size information of the eye, is extracted from the eye movement signal.
Preferably, after the comprehensive recognition probability corresponding to the eye movement signal is obtained, the method further includes:
and judging whether the comprehensive identification probability reaches a preset value, if so, judging that the tested person suffers from the developmental reading disorder, and if not, judging that the tested person does not suffer from the developmental reading disorder.
The invention also provides an intelligent identification system for the developmental reading disorder, which comprises:
the eye movement signal acquisition module is used for acquiring an eye movement signal of a tested person;
The first recognition module is used for inputting the eye movement signals into a preset first recognition model for processing to obtain a first recognition probability that the tested person suffers from the developing dyskinesia;
The second recognition module is used for extracting appointed characteristic data from the eye movement signals, inputting the characteristic data into a preset second recognition model for processing, and obtaining a second recognition probability corresponding to the condition that the tested person suffers from the progressive reading disorder;
And the recognition probability fusion module is used for fusing the first recognition probability and the second recognition probability to obtain the comprehensive recognition probability that the tested person suffers from the developing reading disorder.
Preferably, the system further comprises:
the multi-mode data acquisition module is used for acquiring multi-mode data used for representing the reading capability of the tested person;
The third recognition module is used for inputting the multi-mode data into a third recognition model for processing to obtain a third recognition probability that the tested person suffers from the developing reading disorder;
the recognition probability fusion module is specifically configured to fuse the first recognition probability, the second recognition probability and the third recognition probability to obtain a comprehensive recognition probability that the tested person suffers from a developmental reading disorder.
Preferably, the first recognition model is a two-way long-short-time memory neural network;
the system also comprises an eye movement signal preprocessing module, which is used for uniformly segmenting the eye movement signal before the first identification module executes corresponding operation.
Preferably, the second recognition model is an integrated decision tree model.
Preferably, the system further comprises:
The multi-mode data preprocessing module is used for preprocessing the multi-mode data, and the preprocessing comprises at least one of quantization processing, missing value filling processing and normalization processing.
Preferably, the feature data extracted from the eye movement signal by the second recognition module comprises at least one of coordinate position information of eye gazing to a screen, gazing duration and/or pupil size information of the eye.
Preferably, the system further comprises a reading disorder determining module, configured to determine whether the comprehensive recognition probability reaches a preset value, if yes, determine that the tested person has a developing reading disorder, and if not, determine that the tested person does not have the developing reading disorder.
The invention also provides electronic equipment, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor realizes the intelligent identification method of the extended dyskinesia when executing the computer program.
The present invention also provides a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a method for intelligent identification of a developmental dyskinesia as described above.
The invention has the positive progress effects that:
According to the invention, after the eye movement signal of the tested person is obtained, the eye movement signal is directly input into the first model to perform the identification of the developmental reading disorder, and the second identification model is used for performing the identification of the developmental reading disorder according to the appointed characteristic data of the eye movement signal, and then the two identification results are fused, so that the deviation problem caused by the identification of the developmental reading disorder by only using the appointed characteristic data is reduced to a certain extent, and a more accurate identification result can be obtained. Therefore, the invention can automatically and accurately identify whether the tested person suffers from the developmental dyskinesia, and avoid the corresponding defects caused by manual diagnosis.
Drawings
Fig. 1 is a flowchart of the intelligent identification method of the extended dyskinesia in embodiment 1 of the present invention.
Fig. 2 is a flowchart of the intelligent identification method of the extended dyskinesia in embodiment 2 of the present invention.
Fig. 3 is a schematic diagram of the intelligent identification method of the extended dyskinesia in embodiment 2 of the present invention.
Fig. 4 is a block diagram showing the structure of an intelligent identification system for an extended dyskinesia according to embodiment 3 of the present invention.
Fig. 5 is a block diagram showing the structure of an intelligent identification system for an extended dyskinesia according to embodiment 4 of the present invention.
Fig. 6 is a schematic hardware structure of an electronic device according to embodiment 5 of the present invention.
Detailed Description
The invention is further illustrated by means of the following examples, which are not intended to limit the scope of the invention.
Example 1
As shown in fig. 1, the present embodiment provides an intelligent identification method for an extended dyskinesia. The intelligent identification method for the developmental dyskinesia comprises the following steps:
S101, acquiring eye movement signals of a tested person.
S102, inputting the eye movement signals into a preset first recognition model for processing, and obtaining a first recognition probability that the tested person suffers from the developing dyskinesia.
The first recognition model may be a recognition model using a deep learning technique, and may directly extract features from the eye movement signal and predict whether the subject has a developing dyskinesia.
S103, extracting appointed characteristic data from the eye movement signals, inputting the characteristic data into a preset second recognition model for processing, and obtaining second recognition probability corresponding to the fact that the tested person suffers from the progressive dyskinesia.
The second recognition model may be a shallow learning model. And (3) inputting the characteristic data of the preset (pre-defined) eye movement index into a shallow learning model by applying a characteristic engineering mode, and predicting whether the tested person has the progressive dyskinesia.
It can be seen that the characteristic data input in the second recognition model are manual characteristics, namely eye movement behavior indexes with strong priori information, and the eye movement indexes obtained by using the manual characteristics may lose information which is strong and effective for the task, namely the manually extracted characteristics are not necessarily optimal solutions for the task to a large extent, compared with the fact that the input of the first recognition model is the most original eye movement coordinate time sequence signal, the effective characteristics specific to the task are automatically extracted by using a deep learning technology, the problem that the manual characteristics have the too strong priori information is avoided to a large extent, the solution space for extracting the characteristics is larger, and the better solutions are easier to find.
S104, fusing the first recognition probability and the second recognition probability to obtain the comprehensive recognition probability that the tested person suffers from the developing reading disorder.
In summary, after the eye movement signal of the tested person is obtained, the eye movement signal is directly input into the first model to perform the identification of the developing reading disorder, and the second identification model is used for performing the identification of the developing reading disorder according to the appointed characteristic data of the eye movement signal, and then the two identification results are fused, so that the deviation problem caused by the fact that the developing reading disorder identification is performed only by the appointed characteristic data is reduced to a certain extent, and a more accurate identification result can be obtained. Therefore, the invention can automatically and accurately identify whether the tested person suffers from the developmental dyskinesia, and avoid the corresponding defects caused by manual diagnosis.
In addition, in the intelligent identification method for the developmental reading disorder in the embodiment, the modeling results of the shallow learning model and the deep learning model are high in complementarity through the common modeling of the shallow learning model and the deep learning model, the deep learning model can alleviate the characteristic deviation problem caused by the preset eye movement index, and the performance of the models can be further improved through the integration of the two models by using the model fusion technology, so that the accuracy of identification of the developmental reading disorder of a tested person is guaranteed.
Example 2
As shown in fig. 2 and 3, the intelligent identification method for the extended dyskinesia of the present embodiment is a further improvement of embodiment 1, specifically:
step S101 may acquire an eye movement signal of the subject by:
In this embodiment, the eye movement signal of the user may be collected during the short reading capability test of the eye movement device for the tested person. In addition, in the reading process, the eye movement signals of the patient are different from normal people, so that the reading capability and the obstacle level of the tested person can be comprehensively, comprehensively and objectively judged by recording the eye movement signals in the reading process.
Considering that the reading ability of the subject is in dynamic development, the reading text used for the eye movement test is called up according to the ages, for example, from 7 to 16 years, 3 basic level articles are marked up for each age pair. In order to obtain the reading efficiency of the tested person, each article is attached with a choice question aiming at the content of the article, the tested person selects through a key of the reaction box, and the eye tracker system can record the reaction time and the accuracy.
To fully evaluate the true reading state of development of the subject, an n-year old subject was tested using nine articles n-1, n, n+1. The special testee can also be adjusted according to specific situations (for example, the children with retarded mental development can reduce the test age standard according to the real mental level).
In this embodiment, the first recognition model used in step S102 is preferably a two-way long-short-time memory neural network (Bidirectional long short-term memory network, bi-LSTM), but may be implemented by any other suitable deep learning model, which is not limited in this embodiment. When the first recognition model is Bi-LSTM, the method for intelligently recognizing the developmental dyskinesia further includes, before step S102:
s201, performing uniform segmentation processing on the eye movement signals.
The preset duration d of each segment of eye movement signal is preferably 3s (seconds), and of course, the duration may be adjusted as required, and the specific duration of each segment of eye movement signal is not limited in this embodiment. Because the original data volume of the eye movement signal is larger, the eye movement signal is reflected in that the sampling frequency of the instrument is high, the sampling time is long, and in order to ensure the feasibility, the signal needs to be processed in a segmented way (for example, a 30s signal is uniformly sampled into a 10-segment 3s signal) so as to be input into a common deep neural network for modeling a time sequence signal, namely Bi-LSTM. By uniformly segmenting the eye movement signal, the time sequence of each segment of signal can be represented. Preferably, the embodiment may also normalize the eye movement signal in advance before performing the uniform segmentation process on the eye movement signal.
In this embodiment, step S103 specifically includes:
and S202, extracting at least one of the following characteristic data, namely coordinate position information of eye fixation to a screen, fixation duration and/or pupil size information of the eye from the eye movement signal.
And S203, inputting the characteristic data into a preset second recognition model for processing to obtain a second recognition probability corresponding to the occurrence of the progressive dyskinesia of the tested person.
Specifically, the second recognition model is an integrated decision tree model (Gradient Boosting Decision Tree, GBDT), although any other suitable shallow learning model may be used, and the embodiment is not limited in any way.
In this example, the two-way long and short time memory network directly extracts features from the eye movement signals, reducing to some extent the problem of feature bias caused by preset eye movement indicators. The integrated decision tree model has stronger nonlinear fitting capability and can be better suitable for the complexity of the identification of the developmental dyskinesia.
Wherein, the formula of the Bi-LSTM network is as follows:
Wherein, Representing the forward/reverse input of the eye movement signal in time sequence, W and b are parameters of a model full-connection layer, sigma (·) is a Sigmoid function,Probabilities are identified for the developmental dyskinesia corresponding to eye movement signals.
In this embodiment, before step S104, the method for intelligently identifying an extended dysreading further includes:
s204, acquiring multi-mode data used for representing the reading capability of the tested person.
In particular, the multimodal data used to characterize the reading ability of the subject may include behavioral indicators of the eye movement test (i.e., the response time and accuracy of the subject) as well as the vision of the subject (e.g., near half year vision). If the subject is a student at school, the multi-modal data may also include the subject's performance at school, for example, collecting the language performance and the mathematical performance of the subject over a predetermined period of time (half year) recently, taking the average performance as the performance at school, and using the single performance if there is only one time in the last half year.
S206, inputting the multi-mode data into a third recognition model for processing to obtain a third recognition probability that the tested person suffers from the developing dyskinesia.
The third recognition model may select a shallow learning model, such as an integrated decision tree model.
Furthermore, prior to step S206, the intelligent identification method for the developmental reading disorder further includes:
S205, preprocessing the multi-mode data. For example, the preprocessing includes at least one of quantization processing, missing value filling processing, and normalization processing.
The quantization processing refers to converting non-numeric data into data, such as converting a score "A" into 90 minutes, the missing value filling processing refers to filling the missing data according to a preset strategy, and the normalization processing refers to mapping the data into a range of 0-1.
In this embodiment, step S104 specifically includes:
s207, fusing the first recognition probability, the second recognition probability and the third recognition probability to obtain the comprehensive recognition probability that the tested person suffers from the developing reading disorder. In this embodiment, for example, the recognition probabilities may be fused in a weighted average manner.
In view of the compactness of the test results of the word learning amount evaluation and the read-write test evaluation, the attribute combination of the two evaluation results can be considered, so that the modeling process of machine learning can use the relation information between the two to jointly predict the read-write obstacle degree of the tested person. Therefore, the multi-mode data in this embodiment may further include the word learning amount evaluation result of the tested person.
In another example, after step S207, the method for intelligent identification of an extended dyskinesia further includes:
S208, judging whether the comprehensive recognition probability reaches a preset value, if so, judging that the tested person has the developing reading disorder, otherwise, judging that the tested person does not have the developing reading disorder.
Specifically, in step S208, the preset value is 0.5, that is, when the diagnostic probability of the developing dysreading of the tested object reaches 0.5, it is diagnosed that the tested object has developing dysreading, and when the diagnostic probability is less than 0.5, it is diagnosed that the tested object does not have developing dysreading.
According to the intelligent identification method for the expandable reading disorder, the eye movement signals are subjected to sectional processing and input into the two-way long-short-time memory neural network, the appointed characteristics of the eye movement signals are extracted and input into the integrated decision tree model, the multi-mode data are preprocessed and input into the integrated decision tree model, the modeling results of the integrated decision tree model and the two-way long-short-time memory neural network have strong complementarity, the deep learning model directly processes the eye movement signals so that the characteristic deviation problem caused by the preset eye movement index can be reduced, and the model fusion technology is utilized to integrate the two models of depth and shallow layer so that the performance of the model can be further improved, and the accuracy of identifying the expandable reading disorder of a tested person is further guaranteed.
Example 3
As shown in fig. 4, the present embodiment provides an intelligent identification system for an extended dyskinesia. The intelligent identification system for the developmental dyskinesia comprises:
an eye movement signal acquisition module 1 for acquiring an eye movement signal of a subject.
The first recognition module 2 is configured to input an eye movement signal into a preset first recognition model for processing, so as to obtain a first recognition probability that the tested person suffers from the developing dyskinesia.
The first recognition model may be a recognition model using a deep learning technique, and may directly extract features from the eye movement signal and predict whether the subject has a developing dyskinesia.
The second recognition module 3 is configured to extract specified feature data from the eye movement signal, input the feature data into a preset second recognition model, and process the feature data to obtain a second recognition probability corresponding to the subject suffering from the progressive dyskinesia.
The second recognition model may be a shallow learning model. And inputting the characteristic data of the preset eye movement index into a shallow learning model by applying a characteristic engineering mode, and predicting whether the tested person has the progressive dyskinesia.
It can be seen that the characteristic data input in the second recognition model are manual characteristics, namely eye movement behavior indexes with strong priori information, and the eye movement indexes obtained by using the manual characteristics may lose information which is strong and effective for the task, namely the manually extracted characteristics are not necessarily optimal solutions for the task to a large extent, compared with the fact that the input of the first recognition model is the most original eye movement coordinate time sequence signal, the effective characteristics specific to the task are automatically extracted by using a deep learning technology, the problem that the manual characteristics have the too strong priori information is avoided to a large extent, the solution space for extracting the characteristics is larger, and the better solutions are easier to find.
And the recognition probability fusion module 4 is used for fusing the first recognition probability and the second recognition probability to obtain the comprehensive recognition probability that the tested person suffers from the developing reading disorder.
In summary, after the eye movement signal of the tested person is obtained, the eye movement signal is directly input into the first model to perform the identification of the developing reading disorder, and the second identification model is used for performing the identification of the developing reading disorder according to the appointed characteristic data of the eye movement signal, and then the two identification results are fused, so that the deviation problem caused by the fact that the developing reading disorder identification is performed only by the appointed characteristic data is reduced to a certain extent, and a more accurate identification result can be obtained. Therefore, the invention can automatically and accurately identify whether the tested person suffers from the developmental dyskinesia, and avoid the corresponding defects caused by manual diagnosis.
In addition, in the intelligent identification system for the developmental dyskinesia in the embodiment, the modeling results of the shallow learning model and the deep learning model are high in complementarity through the common modeling of the shallow learning model and the deep learning model, the deep learning model can alleviate the characteristic deviation problem caused by the preset eye movement index, and the performance of the models can be further improved through the integration of the two models by using the model fusion technology, so that the accuracy of identification of the developmental dyskinesia of a tested person is guaranteed.
Example 4
As shown in fig. 5, the intelligent identification system for the extended dyskinesia of the present embodiment is a further improvement of embodiment 3, specifically:
the eye movement signal acquisition module 1 may acquire an eye movement signal of a subject by:
In this embodiment, the eye movement signal of the user may be collected during the short reading capability test of the eye movement device for the tested person. In the process of testing the tested person, the eye movement instrument records the coordinates of the screen of the eye gaze of the tested person. Usually, the answer accuracy of patients with developing dysreading is low, and in the reading process, various reading and gazing characteristics (such as indexes of the number of times of back vision, gazing time and the like) of the patients are different from those of normal people. Therefore, the reading ability and the obstacle level of the tested person can be comprehensively, comprehensively and objectively judged by recording the eye movement signals in the reading process.
Considering that the reading ability of the subject is in dynamic development, the reading text used for the eye movement test is called up according to the ages, for example, from 7 to 16 years, 3 basic level articles are marked up for each age pair. In order to obtain the reading efficiency of the tested person, each article is attached with a choice question aiming at the content of the article, the tested person selects through a key of the reaction box, and the eye tracker system can record the reaction time and the accuracy.
To fully evaluate the true reading state of development of the subject, an n-year old subject was tested using nine articles n-1, n, n+1. The special testee can also be adjusted according to specific situations (for example, the children with retarded mental development can reduce the test age standard according to the real mental level).
In this embodiment, the first recognition model adopted in the first recognition module 2 is preferably a two-way long-short-time memory neural network (Bidirectional long short-term memory network, bi-LSTM), but may be implemented by any other suitable deep learning model, which is not limited in any way. When the first recognition model is Bi-LSTM, the intelligent recognition system for the developmental dyskinesia further comprises an eye movement signal preprocessing module 5 for uniformly segmenting the eye movement signal before the first recognition module executes corresponding operation.
The preset duration d of each segment of eye movement signal is preferably 3s (seconds), and of course, the duration may be adjusted as required, and the specific duration of each segment of eye movement signal is not limited in this embodiment. Because the original data volume of the eye movement signal is larger, the eye movement signal is reflected in that the sampling frequency of the instrument is high, the sampling time is long, and in order to ensure the feasibility, the signal needs to be processed in a segmented way (for example, a 30s signal is uniformly sampled into a 10-segment 3s signal) so as to be input into a common deep neural network for modeling a time sequence signal, namely Bi-LSTM. By uniformly segmenting the eye movement signal, the time sequence of each segment of signal can be represented. Preferably, the embodiment may also normalize the eye movement signal in advance before performing the uniform segmentation process on the eye movement signal.
In the present embodiment, the feature data extracted from the eye movement signal by the second recognition module 3 includes at least one of coordinate position information of eye gaze to a screen, gaze duration, and/or pupil size information of the eye.
The second recognition model is specifically an integrated decision tree model, and of course, any other suitable shallow learning model may be used, which is not limited in this embodiment.
In this example, the two-way long and short time memory network directly extracts features from the eye movement signals, reducing to some extent the problem of feature bias caused by preset eye movement indicators. The integrated decision tree model has stronger nonlinear fitting capability and can be better suitable for the complexity of the identification of the developmental dyskinesia.
Wherein, the formula of the Bi-LSTM network is as follows:
Wherein, Representing the forward/reverse input of the eye movement signal in time sequence, W and b are parameters of a model full-connection layer, sigma (·) is a Sigmoid function,Probabilities are identified for the developmental dyskinesia corresponding to eye movement signals.
In this embodiment, the intelligent identification system for the developmental dyskinesia further includes:
A multi-mode data acquisition module 6 for acquiring multi-mode data for characterizing the reading ability of the subject.
In particular, the multimodal data used to characterize the reading ability of the subject may include behavioral indicators of the eye movement test (i.e., the response time and accuracy of the subject) as well as the vision of the subject (e.g., near half year vision). If the subject is a student at school, the multi-modal data may also include the subject's performance at school, for example, collecting the language performance and the mathematical performance of the subject over a predetermined period of time (half year) recently, taking the average performance as the performance at school, and using the single performance if there is only one time in the last half year.
And the third recognition module 7 is used for inputting the multi-mode data into the third recognition model for processing to obtain a third recognition probability that the tested person suffers from the developing reading disorder.
The third recognition model may select a shallow learning model, such as an integrated decision tree model.
In addition, the intelligent identification system for the developmental dyskinesia further comprises:
the multi-mode data preprocessing module 8 is used for preprocessing multi-mode data. For example, the preprocessing includes at least one of quantization processing, missing value filling processing, and normalization processing.
The quantization processing refers to converting non-numeric data into data, such as converting a score "A" into 90 minutes, the missing value filling processing refers to filling the missing data according to a preset strategy, and the normalization processing refers to mapping the data into a range of 0-1.
In this embodiment, the recognition probability fusion module 4 is specifically configured to fuse the first recognition probability, the second recognition probability, and the third recognition probability to obtain a comprehensive recognition probability that the tested person has a developing reading disorder. In this embodiment, for example, the recognition probabilities may be fused in a weighted average manner.
In view of the compactness of the test results of the word learning amount evaluation and the read-write test evaluation, the attribute combination of the two evaluation results can be considered, so that the modeling process of machine learning can use the relation information between the two to jointly predict the read-write obstacle degree of the tested person. Therefore, the multi-mode data in this embodiment may further include the word learning amount evaluation result of the tested person.
In another example, the intelligent identification system for the developmental reading disorder further includes a reading disorder determination module 9, configured to determine whether the comprehensive identification probability reaches a preset value, if yes, determine that the tested person has the developmental reading disorder, and if not, determine that the tested person does not have the developmental reading disorder.
Specifically, in step S208, the preset value is 0.5, that is, when the diagnostic probability of the developing dysreading of the tested object reaches 0.5, it is diagnosed that the tested object has developing dysreading, and when the diagnostic probability is less than 0.5, it is diagnosed that the tested object does not have developing dysreading.
According to the intelligent identification system for the expandable reading disorder in the embodiment, the eye movement signals are subjected to sectional processing and input into the two-way long-short-time memory neural network, the appointed characteristics of the eye movement signals are extracted and input into the integrated decision tree model, the multi-mode data are preprocessed and then input into the integrated decision tree model, the modeling results of the integrated decision tree model and the two-way long-short-time memory neural network have stronger complementarity, the problem of characteristic deviation caused by preset eye movement indexes can be relieved by directly processing the eye movement signals through the deep learning model, the performance of the models can be further improved by integrating the depth model and the shallow model through the model fusion technology, and the accuracy of identifying the expandable reading disorder of a tested person is further guaranteed.
Example 5
Fig. 6 is a schematic structural diagram of an electronic device according to embodiment 5 of the present invention. The electronic device includes a memory, a processor, and a computer program stored on the memory and executable on the processor, which when executed implements the method for intelligently identifying the developmental dyskinesia of embodiment 1 or embodiment 2. The electronic device 30 shown in fig. 6 is merely an example, and should not be construed as limiting the functionality and scope of use of embodiments of the present invention.
As shown in fig. 6, the electronic device 30 may be embodied in the form of a general purpose computing device, which may be a server device, for example. The components of the electronic device 30 may include, but are not limited to, the at least one processor 31, the at least one memory 32, and a bus 33 that connects the various system components, including the memory 32 and the processor 31.
The bus 33 includes a data bus, an address bus, and a control bus.
Memory 32 may include volatile memory such as Random Access Memory (RAM) 321 and/or cache memory 322, and may further include Read Only Memory (ROM) 323.
Memory 32 may also include a program/utility 325 having a set (at least one) of program modules 324, such program modules 324 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each or some combination of which may include an implementation of a network environment.
The processor 31 executes various functional applications and data processing such as the intelligent identification method of the extended dyskinesia of embodiment 1 or embodiment 2 of the present invention by running a computer program stored in the memory 32.
The electronic device 30 may also communicate with one or more external devices 34 (e.g., keyboard, pointing device, etc.). Such communication may be through an input/output (I/O) interface 35. Also, model-generating device 30 may also communicate with one or more networks, such as a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network, such as the internet, via network adapter 36. As shown, network adapter 36 communicates with the other modules of model-generating device 30 via bus 33. It should be appreciated that although not shown, other hardware and/or software modules may be used in connection with the model-generating device 30, including, but not limited to, microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, among others.
It should be noted that although several units/modules or sub-units/modules of an electronic device are mentioned in the above detailed description, such a division is merely exemplary and not mandatory. Indeed, the features and functionality of two or more units/modules described above may be embodied in one unit/module in accordance with embodiments of the present invention. Conversely, the features and functions of one unit/module described above may be further divided into ones that are embodied by a plurality of units/modules.
Example 6
The present embodiment provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method for intelligent identification of an extended dyskinesia of embodiment 1 or embodiment 2.
More specifically, a readable storage medium may include, but is not limited to, a portable disk, hard disk, random access memory, read only memory, erasable programmable read only memory, optical storage device, magnetic storage device, or any suitable combination of the foregoing.
In a possible embodiment, the invention may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps of implementing the method for intelligent identification of an extended dysreading as in example 1 or example 2, when said program product is run on the terminal device.
Wherein the program code for carrying out the invention may be written in any combination of one or more programming languages, which program code may execute entirely on the user device, partly on the user device, as a stand-alone software package, partly on the user device and partly on the remote device or entirely on the remote device.
While specific embodiments of the invention have been described above, it will be appreciated by those skilled in the art that this is by way of example only, and the scope of the invention is defined by the appended claims. Various changes and modifications to these embodiments may be made by those skilled in the art without departing from the principles and spirit of the invention, but such changes and modifications fall within the scope of the invention.
Claims (12)
1. The intelligent identification method for the developmental dyskinesia is characterized by comprising the following steps of:
Acquiring an eye movement signal of a tested person, wherein the eye movement signal is an original eye movement coordinate time sequence signal;
Inputting the eye movement signal into a preset first recognition model for processing to obtain a first recognition probability that the tested person suffers from the developing dyskinesia;
Extracting appointed characteristic data from the eye movement signals, inputting the characteristic data into a preset second recognition model for processing to obtain a second recognition probability corresponding to the person to be tested suffering from the progressive dyskinesia;
Fusing the first recognition probability and the second recognition probability to obtain the comprehensive recognition probability that the tested person suffers from the developing reading disorder;
Before fusing the first recognition probability and the second recognition probability, the intelligent identification method for the developmental reading disorder further comprises the following steps:
Acquiring multi-modal data representing the reading capability of the subject;
Inputting the multi-mode data into a third recognition model for processing to obtain a third recognition probability that the tested person suffers from the developing dyskinesia;
the fusing of the first recognition probability and the second recognition probability to obtain a comprehensive recognition probability that the tested person suffers from the developing reading disorder comprises the following steps:
fusing the first recognition probability, the second recognition probability and the third recognition probability to obtain the comprehensive recognition probability that the tested person suffers from the developing reading disorder;
the first recognition model is a two-way long-short-time memory neural network, and the second recognition model is an integrated decision tree model.
2. The intelligent identification method for the extended dyskinesia according to claim 1, wherein,
Before inputting the eye movement signal into the first recognition model for processing, the method further comprises:
And uniformly segmenting the eye movement signal.
3. The method of claim 1, wherein prior to entering the multimodal data into a third recognition model for processing, the method further comprises:
the multi-modal data is preprocessed, the preprocessing including at least one of quantization processing, missing value filling processing, and normalization processing.
4. The method of claim 1, wherein said extracting specified characteristic data from said eye movement signal comprises:
At least one of the following characteristic data, namely coordinate position information of eye fixation to a screen, fixation duration and/or pupil size information of the eye, is extracted from the eye movement signal.
5. The method for intelligently identifying a developmental reading disorder according to claim 1, further comprising, after the obtaining of the comprehensive identification probability corresponding to the eye movement signal:
and judging whether the comprehensive identification probability reaches a preset value, if so, judging that the tested person suffers from the developmental reading disorder, and if not, judging that the tested person does not suffer from the developmental reading disorder.
6. An intelligent identification system for developmental dyskinesia, characterized in that the intelligent identification system for developmental dyskinesia comprises:
The eye movement signal acquisition module is used for acquiring an eye movement signal of a tested person, wherein the eye movement signal is an original eye movement coordinate time sequence signal;
The first recognition module is used for inputting the eye movement signals into a preset first recognition model for processing to obtain a first recognition probability that the tested person suffers from the developing dyskinesia;
The second recognition module is used for extracting appointed characteristic data from the eye movement signals, inputting the characteristic data into a preset second recognition model for processing, and obtaining a second recognition probability corresponding to the condition that the tested person suffers from the progressive reading disorder;
The recognition probability fusion module is used for fusing the first recognition probability and the second recognition probability to obtain the comprehensive recognition probability that the tested person suffers from the developing reading disorder;
The system further comprises:
the multi-mode data acquisition module is used for acquiring multi-mode data used for representing the reading capability of the tested person;
The third recognition module is used for inputting the multi-mode data into a third recognition model for processing to obtain a third recognition probability that the tested person suffers from the developing reading disorder;
The recognition probability fusion module is specifically configured to fuse the first recognition probability, the second recognition probability and the third recognition probability to obtain a comprehensive recognition probability that the tested person suffers from a developmental reading disorder;
the first recognition model is a two-way long-short-time memory neural network, and the second recognition model is an integrated decision tree model.
7. The intelligent identification system for an extended reading disorder of claim 6,
The system also comprises an eye movement signal preprocessing module, which is used for uniformly segmenting the eye movement signal before the first identification module executes corresponding operation.
8. The intelligent identification system for an extended reading disorder of claim 6, further comprising:
The multi-mode data preprocessing module is used for preprocessing the multi-mode data, and the preprocessing comprises at least one of quantization processing, missing value filling processing and normalization processing.
9. The intelligent identification system of an extended reading disorder of claim 6, wherein the characteristic data extracted from the eye movement signal by the second identification module comprises at least one of coordinate position information of eye gaze to screen, gaze duration and/or pupil size information of eye.
10. The intelligent identification system of claim 6, further comprising a reading disorder determination module configured to determine whether the integrated identification probability reaches a predetermined value, and if so, determine that the subject has a developing reading disorder, and if not, determine that the subject does not have a developing reading disorder.
11. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the intelligent identification method of the developmental dyskinesia according to any of claims 1 to 5 when executing the computer program.
12. A computer readable storage medium having stored thereon a computer program, wherein the computer program when executed by a processor implements the method for intelligent identification of a developmental reading disorder according to any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110831632.2A CN113842113B (en) | 2021-07-22 | 2021-07-22 | Intelligent identification method, system, device and storage medium for developmental dyslexia |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110831632.2A CN113842113B (en) | 2021-07-22 | 2021-07-22 | Intelligent identification method, system, device and storage medium for developmental dyslexia |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113842113A CN113842113A (en) | 2021-12-28 |
| CN113842113B true CN113842113B (en) | 2024-12-27 |
Family
ID=78975191
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110831632.2A Active CN113842113B (en) | 2021-07-22 | 2021-07-22 | Intelligent identification method, system, device and storage medium for developmental dyslexia |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113842113B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115363581B (en) * | 2022-08-19 | 2023-05-05 | 山东心法科技有限公司 | Method, equipment and medium for predicting dysreading for young children |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106491129A (en) * | 2016-10-10 | 2017-03-15 | 安徽大学 | A kind of Human bodys' response system and method based on EOG |
| CN111114556A (en) * | 2019-12-24 | 2020-05-08 | 北京工业大学 | Lane change intention identification method based on LSTM under multi-source exponential weighting loss |
| KR20210019266A (en) * | 2019-08-12 | 2021-02-22 | 주식회사 비엔알아이 | Apparatus and method for diagnosis of reading ability based on machine learning using eye tracking |
| CN112950609A (en) * | 2021-03-13 | 2021-06-11 | 深圳市龙华区妇幼保健院(深圳市龙华区妇幼保健计划生育服务中心、深圳市龙华区健康教育所) | Intelligent eye movement recognition analysis method and system |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105816181A (en) * | 2016-03-11 | 2016-08-03 | 安徽大学 | Reading behavior identification method and device based on EOG |
| CN106897426A (en) * | 2017-02-27 | 2017-06-27 | 上海禹放信息科技有限公司 | Specific data genaration system and method based on eyeball tracking technology |
| EP3697302A4 (en) * | 2017-10-17 | 2021-10-20 | Satish Rao | Machine learning based system for identifying and monitoring neurological disorders |
| CN109567817B (en) * | 2018-11-19 | 2023-12-12 | 北京育铭天下科技有限公司 | Reading capability assessment method and system and auxiliary device thereof |
| CN111985282A (en) * | 2019-05-24 | 2020-11-24 | 深圳市希科普股份有限公司 | Learning ability training and evaluating system |
| CN110827986A (en) * | 2019-11-11 | 2020-02-21 | 科大讯飞股份有限公司 | Method, device and equipment for screening developmental reading disorder and storage medium |
| KR102262890B1 (en) * | 2019-11-29 | 2021-06-09 | 주식회사 비엔알아이 | Reading ability improvement training apparatus for providing training service to improve reading ability in connection with reading ability diagnosis apparatus based on eye tracking and apparatus for providing service comprising the same |
| GB2594966B (en) * | 2020-05-13 | 2024-07-31 | Sony Interactive Entertainment Inc | Gaze tracking apparatus and systems |
| CN112381287A (en) * | 2020-11-13 | 2021-02-19 | 中国科学院心理研究所 | Chinese developmental reading disorder prediction system and prediction method thereof |
-
2021
- 2021-07-22 CN CN202110831632.2A patent/CN113842113B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106491129A (en) * | 2016-10-10 | 2017-03-15 | 安徽大学 | A kind of Human bodys' response system and method based on EOG |
| KR20210019266A (en) * | 2019-08-12 | 2021-02-22 | 주식회사 비엔알아이 | Apparatus and method for diagnosis of reading ability based on machine learning using eye tracking |
| CN111114556A (en) * | 2019-12-24 | 2020-05-08 | 北京工业大学 | Lane change intention identification method based on LSTM under multi-source exponential weighting loss |
| CN112950609A (en) * | 2021-03-13 | 2021-06-11 | 深圳市龙华区妇幼保健院(深圳市龙华区妇幼保健计划生育服务中心、深圳市龙华区健康教育所) | Intelligent eye movement recognition analysis method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113842113A (en) | 2021-12-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112069329B (en) | Text corpus processing method, device, equipment and storage medium | |
| CN109087670B (en) | Emotion analysis method, system, server and storage medium | |
| CN110570941B (en) | System and device for assessing psychological state based on text semantic vector model | |
| US10431338B2 (en) | System and method for weighting manageable patient attributes during criteria evaluations for treatment | |
| CN111145903B (en) | Method and device for acquiring vertigo inquiry text, electronic equipment and inquiry system | |
| US20210312942A1 (en) | System, method, and computer program for cognitive training | |
| CN109783631A (en) | Method of calibration, device, computer equipment and the storage medium of community's question and answer data | |
| CN112164459A (en) | An Information Assessment Method for Depressive Symptoms | |
| CN111477320B (en) | Treatment effect prediction model construction system, treatment effect prediction system and terminal | |
| CN111403039A (en) | Dynamic balance evaluation method, device, equipment and medium | |
| CN114372201A (en) | Method and system for intelligent recommendation of medical examination information, storage medium, and computing device | |
| CN112115240B (en) | Classification processing method, device, server and storage medium | |
| CN117809798A (en) | Verification report interpretation method, system, equipment and medium based on large model | |
| CN111161884A (en) | Disease prediction method, device, equipment and medium for unbalanced data | |
| CN119007993A (en) | Method, device and equipment for generating multimodal fusion psychological assessment report | |
| CN113842113B (en) | Intelligent identification method, system, device and storage medium for developmental dyslexia | |
| CN119153077B (en) | AIGC technology-based multi-mode chronic disease risk prediction method | |
| US20240180482A1 (en) | Systems and methods for digital speech-based evaluation of cognitive function | |
| CN113241178B (en) | Device for determining severity of depression of tested person | |
| Yan et al. | Optimizing MFCC parameters for the automatic detection of respiratory diseases | |
| CN114201613B (en) | Test question generation method, test question generation device, electronic device, and storage medium | |
| CN114743686A (en) | Pediatric nurse emergency nursing method and system | |
| CN113808709A (en) | A method and system for predicting resilience based on text analysis | |
| CN113537565A (en) | Deep learning-based student academic early warning method and system | |
| US12257062B2 (en) | System and process for computerized screening and monitoring for cognitive characteristics |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |