[go: up one dir, main page]

CN110693510A - Auxiliary diagnostic device for attention deficit hyperactivity disorder and using method thereof - Google Patents

Auxiliary diagnostic device for attention deficit hyperactivity disorder and using method thereof Download PDF

Info

Publication number
CN110693510A
CN110693510A CN201910987452.6A CN201910987452A CN110693510A CN 110693510 A CN110693510 A CN 110693510A CN 201910987452 A CN201910987452 A CN 201910987452A CN 110693510 A CN110693510 A CN 110693510A
Authority
CN
China
Prior art keywords
acceleration
data
hyperactivity disorder
attention deficit
deficit hyperactivity
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910987452.6A
Other languages
Chinese (zh)
Inventor
蒋鑫龙
黄武亮
陈益强
张腾
邢云冰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Computing Technology of CAS
Original Assignee
Institute of Computing Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Computing Technology of CAS filed Critical Institute of Computing Technology of CAS
Priority to CN201910987452.6A priority Critical patent/CN110693510A/en
Publication of CN110693510A publication Critical patent/CN110693510A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/16Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/168Evaluating attention deficit, hyperactivity
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Measuring devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor or mobility of a limb
    • A61B5/1121Determining geometric values, e.g. centre of rotation or angular range of movement
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/40Detecting, measuring or recording for evaluating the nervous system
    • A61B5/4076Diagnosing or monitoring particular conditions of the nervous system
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6802Sensor mounted on worn items
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/725Details of waveform analysis using specific filters therefor, e.g. Kalman or adaptive filters
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/7264Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B2562/00Details of sensors; Constructional details of sensor housings or probes; Accessories for sensors
    • A61B2562/02Details of sensors specially adapted for in-vivo measurements
    • A61B2562/0219Inertial sensors, e.g. accelerometers, gyroscopes, tilt switches

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Veterinary Medicine (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Surgery (AREA)
  • Biophysics (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Physiology (AREA)
  • Artificial Intelligence (AREA)
  • Psychiatry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Developmental Disabilities (AREA)
  • Neurology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Geometry (AREA)
  • Dentistry (AREA)
  • Mathematical Physics (AREA)
  • Fuzzy Systems (AREA)
  • Neurosurgery (AREA)
  • Evolutionary Computation (AREA)
  • Child & Adolescent Psychology (AREA)
  • Educational Technology (AREA)
  • Hospice & Palliative Care (AREA)
  • Psychology (AREA)
  • Social Psychology (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The invention provides an attention deficit hyperactivity disorder auxiliary diagnosis device and a using method thereof, wherein the device comprises the following steps: the display screen is used for interacting with the tested person and displaying a test scene; the wearable acceleration sensor is used for acquiring the limb acceleration of the measured person in the test scene; and the data processing module is used for preprocessing the acceleration of the limb to obtain data to be detected, inputting the data to be detected into a deep neural network-based attention deficit hyperactivity disorder detection model for identification to obtain an auxiliary diagnosis result, and sending the auxiliary diagnosis result to the display screen. The method for classifying the samples of the attention deficit hyperactivity disorder group and the control group based on the deep learning method has accurate and objective results and convenient popularization, and can assist in clinical auxiliary diagnosis of patients with attention deficit hyperactivity disorder.

Description

Attention deficit hyperactivity disorder auxiliary diagnosis device and using method thereof
Technical Field
The invention relates to the field of data classification based on a convolutional neural network, in particular to an attention deficit hyperactivity disorder auxiliary diagnosis device and a using method thereof.
Background
Attention deficit hyperactivity disorder is also called hyperactivity disorder in China, and is the most common mental disorder in childhood. The disease rate is 3% -7% through domestic and foreign investigation, and the male and female ratio is (4-9): 1. in China, there are 1461 to 1979 thousands of children patients, and the prevalence rate is as high as 4.31% -5.83%. The onset of hyperactivity brings learning disability, life disability and social function impairment to patients, and causes great medical and economic burden to society. Therefore, the attention deficit hyperactivity disorder of the children is evaluated and diagnosed as early as possible, and active intervention and treatment are adopted, so that the patients can be improved obviously, and the malignant development of the disease is greatly reduced.
Currently, the auxiliary diagnosis of attention deficit hyperactivity disorder at home and abroad mainly comprises the following four types: the system is based on clinical scale evaluation diagnosis technology, wearable sensor diagnosis technology, neuroelectrophysiology diagnosis technology, brain imaging and brain function imaging diagnosis technology.
(1) The clinical scale assessment and diagnosis technology is based on a behavior characterization comparison table summarized and made by doctors through clinical experience induction, and diagnosis is realized by combining communication between doctors and patients. However, the clinical rating scale is only descriptive language and lacks of quantitative standards, the scale scoring mainly depends on the subjective experience judgment of doctors, the interference of human factors is large, and the referential value of clinical diagnosis is not high.
(2) The wearable sensor diagnosis technology uses various motion sensors such as an accelerometer, a gyroscope, a magnetometer and the like, collects data information in a motion process by being worn on a specific part of a human body, obtains classification of user behaviors by analyzing and processing the data, and can be used for diagnosis of mental diseases. The wearable monitoring device is convenient to wear, is not limited by the environment of a patient, and can monitor the behavior of the user in various scenes, such as in class, in motion, in sleep and the like. The wearable sensors commonly used at present mainly comprise an accelerometer, a gyroscope, a magnetometer and the like, wherein the accelerometer can detect the magnitude value of three-axis acceleration in space; the gyroscope can detect the magnitude value of the three-axis angular acceleration in space; magnetometers are capable of detecting magnitude values of the three-axis magnetic field in space. The wearing positions of the sensors are mainly the wrist, the ankle, the head, the waist, the backpack and the like. Due to the fact that user behaviors under different scenes are complex, classification of the patients with the hyperactivity and the control group is different based on data collected by the sensor, the collected data and the symptoms have no strong association relationship, and wrong diagnosis results are easy to generate.
(3) The neuroelectrophysiological examination technology can provide abundant information for clinical diagnosis of mental disorders based on brain waves, which can reflect the collective activity of cerebral neurons, the periodicity and the rhythmicity of waveforms, amplitudes and frequencies of the brain waves, special positioning of spatial domains and the like. For example: patent CN201810934990 proposes a brain cognitive nerve function evaluation system and method based on EEG and ERPs, which can be used for clinical early diagnosis of brain diseases. But it does not identify the co-morbidities well, is not very specific, and can be checked for false positives due to patient emotional stress.
(4) The brain imaging and brain functional imaging diagnosis technology obtains the brain anatomical structure characteristics by means of the nuclear magnetic resonance imaging technology, and the provided images can reflect the activity characteristics of the brain functions and can be used for diagnosing attention deficit hyperactivity disorder. For example: patent CN201711330864 proposes a functional nuclear magnetic resonance image-based analysis system for attention deficit hyperactivity disorder of children, which can be used for analysis and diagnosis of attention deficit hyperactivity disorder of children. However, the brain imaging apparatus is expensive and may cause damage and danger to the human body.
Among the four methods, the neuroelectrophysiological examination technology, the brain imaging and brain functional imaging examination technology are limited by low equipment cost and low detection specificity; the clinical scale assessment and diagnosis technology is limited by subjective judgment of doctors, and the influence of artificial factors is large; in the wearable sensor diagnosis technology, the behavior of the experimental object in the free environment is difficult to control, and the characterization of the collected data on the sensor is difficult to distinguish, so that the recognition accuracy is often low.
Currently, wearable sensor-based diagnostic techniques are not harmful to the human body because of their non-invasive nature, and can be independent of detection location, environment, etc. due to the low cost of the sensor. When the method is applied to clinical detection, the application cost and the manpower requirement in the detection process can be reduced, and the method is convenient to deploy.
Disclosure of Invention
In order to overcome the defects in the prior art, the invention provides an attention deficit hyperactivity disorder auxiliary diagnosis device, which comprises:
the display screen is used for interacting with the tested person and displaying a test scene;
the wearable acceleration sensor is used for acquiring the limb acceleration of the measured person in the test scene;
and the data processing module is used for preprocessing the acceleration of the limb to obtain data to be detected, inputting the data to be detected into a deep neural network-based attention deficit hyperactivity disorder detection model for identification to obtain an auxiliary diagnosis result, and sending the auxiliary diagnosis result to the display screen.
The auxiliary diagnosis device for attention deficit hyperactivity disorder comprises a plurality of wearable acceleration sensors which are respectively positioned on the head, the waist, the wrist and the ankle of a person to be tested.
The device for assisting in diagnosing the attention deficit hyperactivity disorder comprises a test scene, wherein the test scene comprises a character image, one of four limbs of the character image is displayed as a preset color during a diagnosis test process, a tested person makes a corresponding limb reaction according to the preset color, and the wearable acceleration sensor collects the limb reaction and records the acceleration of the limb.
The attention deficit hyperactivity disorder-assisted diagnosis apparatus described above, wherein the preprocessing includes: and carrying out low-pass filtering on the limb acceleration to eliminate the gravity value, carrying out continuous acceleration data windowing on the low-pass filtered limb acceleration, and generating a pair of acceleration images as the data to be detected according to the acceleration values in the vertical direction and the horizontal direction by using the acceleration data in each window.
The attention deficit hyperactivity disorder auxiliary diagnosis device is characterized in that the process of eliminating the gravity value is shown as formula (1):
in the formula (1)Representing the acceleration of gravity, N representing the number of points included in the selected window, ax(j),ay(j),az(j) Representing the three-axis acceleration collected by the wearable acceleration sensor, calculating the gravity acceleration according to the formula (1), and subtracting the gravity vector from the acceleration vector collected by the wearable acceleration sensor to obtain the actual acceleration vector generated only by the human motion
Figure BDA0002237138640000033
Using windowed approach to acceleration vectors
Figure BDA0002237138640000034
The continuous acceleration data can be divided into windowed data by setting the window length and the window step length of the windowing parameters;
vector decomposition is carried out on the windowed data, the windowed data are further decomposed into two directions of vertical and horizontal directions, the numerical values of the two directions are used as coordinates of an X axis and a Y axis in a two-dimensional coordinate system, and each point in a window is generated into a picture in the mode, specifically:
(a) acceleration is resolved into horizontal and vertical directions:
Figure BDA0002237138640000035
Figure BDA0002237138640000036
wherein in the formula (3)Represents the acceleration in the vertical direction and,
Figure BDA0002237138640000038
the representative point is a product of the point,represents the square of the modulus, in equation (4)
Figure BDA00022371386400000310
Representing the acceleration in the horizontal direction.
(b) Each point within the window is dispersed into the two-dimensional graph:
dispersing each point in the window into a two-dimensional graph according to the magnitude of the acceleration value, wherein the formulas are shown in the following (5) and (6); (x, y) obtained according to the formulas (5) and (6) can be used as coordinates of points in the graph; wherein midx,lengthx,midy,lengthyTo generate half of the lateral length, half of the longitudinal length, th of the acceleration patternx,thyDispersion thresholds in the lateral and longitudinal directions;
Figure BDA0002237138640000041
Figure BDA0002237138640000042
image (x, y) + 1 formula (7)
After the pattern dispersion is completed, the image is the acceleration image in two-dimensional gray scale.
The invention also provides a use method of the attention deficit hyperactivity disorder auxiliary diagnosis device, which comprises the following steps:
step 1, interacting with a tested person by using a display screen, and displaying a test scene;
step 2, collecting the limb acceleration of the tested person in the test scene by using a wearable acceleration sensor;
and 3, preprocessing the limb acceleration by using a data processing module to obtain data to be detected, inputting the data to be detected into a deep neural network-based attention deficit hyperactivity disorder detection model for identification to obtain an auxiliary diagnosis result, and sending the auxiliary diagnosis result to the display screen.
The use method of the attention deficit hyperactivity disorder auxiliary diagnosis device comprises a plurality of wearable acceleration sensors which are respectively positioned at the head, the waist, the wrist and the ankle of a person to be tested.
The use method of the attention deficit hyperactivity disorder auxiliary diagnosis device comprises the steps that the test scene comprises a character image, one of four limbs of the character image is displayed as a preset color during a diagnosis test process, a tested person makes a corresponding limb reaction according to the preset color, the wearable acceleration sensor collects the limb reaction, and the acceleration of the limb is recorded.
The method for using the attention deficit hyperactivity disorder-assisted diagnosis device comprises the following steps: and carrying out low-pass filtering on the limb acceleration to eliminate the gravity value, carrying out continuous acceleration data windowing on the low-pass filtered limb acceleration, and generating a pair of acceleration images as the data to be detected according to the acceleration values in the vertical direction and the horizontal direction by using the acceleration data in each window.
The use method of the attention deficit hyperactivity disorder auxiliary diagnosis device is characterized in that the process of eliminating the gravity value is shown as the formula (1):
Figure BDA0002237138640000051
in the formula (1)
Figure BDA0002237138640000052
Representing the acceleration of gravity, N representing the number of points included in the selected window, ax(j),ay(j),az(j) Representing the three-axis acceleration collected by the wearable acceleration sensor, calculating the gravity acceleration according to the formula (1), and subtracting the gravity vector from the acceleration vector collected by the wearable acceleration sensor to obtain the actual acceleration vector generated only by the human motion
Figure BDA0002237138640000053
Using windowed approach to acceleration vectors
Figure BDA0002237138640000054
The continuous acceleration data can be divided into windowed data by setting the window length and the window step length of the windowing parameters;
vector decomposition is carried out on the windowed data, the windowed data are further decomposed into two directions of vertical and horizontal directions, the numerical values of the two directions are used as coordinates of an X axis and a Y axis in a two-dimensional coordinate system, and each point in a window is generated into a picture in the mode, specifically:
(a) acceleration is resolved into horizontal and vertical directions:
Figure BDA0002237138640000055
Figure BDA0002237138640000056
wherein in the formula (3)
Figure BDA0002237138640000057
Represents the acceleration in the vertical direction and,
Figure BDA0002237138640000058
the representative point is a product of the point,
Figure BDA0002237138640000059
represents the square of the modulus, in equation (4)
Figure BDA00022371386400000510
Representing the acceleration in the horizontal direction.
(b) Each point within the window is dispersed into the two-dimensional graph:
dispersing each point in the window into a two-dimensional graph according to the magnitude of the acceleration value, wherein the formulas are shown in the following (5) and (6); (x, y) obtained according to the formulas (5) and (6) can be used as coordinates of points in the graph; wherein midx,lengthx,midy,lengthyTo generate half of the lateral length, half of the longitudinal length, th of the acceleration patternx,thyDispersion thresholds in the lateral and longitudinal directions;
Figure BDA0002237138640000061
image (x, y) + 1 formula (7)
After the pattern dispersion is completed, the image is the acceleration image in two-dimensional gray scale.
According to the scheme, the invention has the advantages that:
currently, clinical diagnosis of the impairment/hyperactivity disorder depends on the observation and inquiry of doctors and the rating scale filled out by parents, and has high requirements on the experience of doctors, and is subjective when different people fill out the rating scale. Conventional objective methods such as brain wave and neuroelectric based methods tend to have problems of high cost, invasiveness or misdiagnosis due to the stress of the test subject.
According to the invention, the wearable acceleration sensor with high precision is utilized, and a cartoon limb conflict detection scene constructed according to a clinical general evaluation table and a related detection paradigm is used, so that the data acquisition which is low in cost, non-invasive, high in precision, convenient to deploy and capable of reflecting the attention defect hyperactivity disorder movement condition is realized. In addition, the method for classifying the attention deficit hyperactivity disorder group and the control group samples based on the deep learning method has accurate and objective results and convenient popularization, and can assist in clinical auxiliary diagnosis of attention deficit hyperactivity disorder patients.
Drawings
FIG. 1 is a general system layout of the present invention;
FIG. 2 is a flowchart of the operation of the system for diagnosing assist in attention deficit hyperactivity disorder;
FIG. 3 is a schematic view of a screen interaction detection scenario;
FIG. 4 is a diagram illustrating a specific process of executing a test task;
FIG. 5 is a diagram of the main steps of model offline training;
FIG. 6 is a diagram of the main steps of model online detection;
FIG. 7 is a schematic diagram of a convolutional neural network model in a system.
Detailed Description
The invention comprises the following key points:
key point 1: a limb conflict detection scene is constructed based on a clinical general assessment table and a related detection paradigm, a plurality of high-precision wearable acceleration sensors are used for sensing the body motion condition of a test object, and accurate data acquisition is realized. The technical effects are as follows: the limb conflict detection scene has a cartoon image, is convenient for a test object to accept, reduces the diagnosis 'false positive' caused by the tension of the test object in a clinical environment, and compared with a diagnosis method based on methods such as nerve electricity and brain waves, the method provided by the invention has the characteristics of low cost, non-invasion and convenient deployment, and realizes the collection of the movement data of the attention-deficit hyperactivity disorder patient with convenient deployment and extremely high precision;
key point 2: and classifying the preprocessed acceleration data by using a deep learning algorithm, realizing accurate classification of the movement behaviors of the attention deficit hyperactivity disorder group and the control group through the acceleration image, and assisting clinical diagnosis of the attention deficit hyperactivity disorder. The technical effects are as follows: compared with the inquiry, observation and rating scale method of doctors in clinic, the method provided by the invention has objectivity and is beneficial to the accuracy of diagnosis. Compared with a manual feature selection method, the intervention of the depth algorithm has higher adaptability and is beneficial to popularization in clinic. In conclusion, the invention can realize accurate, objective and convenient-to-popularize auxiliary diagnosis of the attention-deficit hyperactivity disorder patient.
In order to make the aforementioned features and effects of the present invention more comprehensible, embodiments accompanied with figures are described in detail below.
First, overall system design
The objective of the present invention is to solve the above-mentioned problems in the background art that the implementation of attention deficit hyperactivity disorder diagnosis based on a rating scale in clinical practice is inaccurate, limited by the subjectivity of doctors and parents, and the like, and the neuroelectric or brain imaging technology is expensive in cost or invasive, and the like, which is not easy to popularize, and to provide a non-invasive wearable sensor to implement objective auxiliary diagnosis of attention deficit hyperactivity disorder based on a motion acceleration image, thereby improving the accuracy of clinical attention deficit hyperactivity disorder diagnosis.
In the invention, the data acquisition mode uses a wearable acceleration sensor. In order to complete the collection of the motion data, and because the data collection environment of the present invention is indoors and is not limited by the distance limitation of the data transmission device outdoors, six wearable motion sensors are selected in the present study, and are respectively located at the head (one), the waist (one), the wrist (one for each hand, two for each), and the ankle (one for each foot, two for each), of the subject, and each sensor can record the magnitude of the acceleration of the worn position.
The invention realizes an attention deficit hyperactivity disorder auxiliary diagnosis system, and the overall design concept of the system is shown in figure 1.
The front-end interface of the system provided by the invention is responsible for the design of an interactive part with a user, including a display screen and a test scene; the back-end framework processes and analyzes data uploaded by a user, and comprises a data processing part and a detection model part.
The data processing part is responsible for processing the acquired acceleration original data into data required by the detection model, and the detection model part applies a related deep learning algorithm to classify the data of the user. After the user uploads the selected data, the data are sent to a back-end data analysis system by means of a network, and after the analysis system completes data analysis, the obtained scores are displayed to the user on a display screen in the front end of the user.
The specific work flow of the system is shown in fig. 2, and the main steps include:
(a) wearing an acceleration sensor in a test scene constructed according to the general rating scale to finish all tests;
(b) the acceleration sensor finishes the operation of collecting and uploading the original acceleration data;
(c) preprocessing the original acceleration data acquired by the sensor;
(d) constructing an attention defect hyperactivity disorder detection model for recognition by the preprocessed data based on a depth algorithm, and sending a recognition result to a front-end interface;
(e) the diagnosis result is displayed to the user in the front-end interactive interface;
(f) judging whether to continue diagnosis, if so, turning to the step (a); otherwise, ending.
Second, test scenario design
The invention is based on a clinical general mental disorder rating scale DSM-V and a detection scene of a relevant paradigm, in particular to a detection scene of screen interaction. As shown in fig. 3, a cartoon character image appears on the screen, and both hands and feet of the cartoon character image randomly appear red or green. When the red color appears, the test object is required to be kept still, and when the green color appears, the test object is required to be lifted or lifted by the corresponding four limbs, for example, the left hand or the right foot of the cartoon character is green, and the test object should lift the left hand or the right foot.
The specific execution process of the test task is shown in fig. 4, and the main steps include:
(a) checking the connection condition of the wearable acceleration sensor;
(b) judging whether the connection of the acceleration sensor is normal or not, and turning to the step (c) if the connection of the acceleration sensor is normal; otherwise, turning to the step (a);
(c) setting the total time of the test;
(d) the cartoon characters in the video curtain are displayed in red or green color at random for a moment;
(e) the test object should make a corresponding reaction, and record a corresponding result or no reaction;
(f) judging whether the total testing time is reached, if so, turning to the step (g); otherwise, turning to the step (d);
(g) and counting and recording the result, and ending.
Identification method based on acceleration image
The invention realizes the identification method based on the acceleration image, realizes the perception of the motion characteristics by means of a series of motion data acquired by a high-precision acceleration sensor, and further realizes the auxiliary diagnosis of attention defect hyperactivity disorder. The identification method comprises a series of preprocessing operations such as gravity elimination, windowing, acceleration image generation and the like, and a classification algorithm based on a convolutional neural network.
Because the acceleration sensor worn by the experimental object is generally a piezoelectric sensor, the detection of the magnitude of the acceleration value is realized by detecting the voltage change caused by the action of forces in three directions in the acquisition chip. The acceleration data acquisition realized by the method inevitably introduces additional influence of gravity. In order to avoid the influence of gravity on the collected data, and the influence of gravity only caused by different angles of the sensor in the data collection process, the gravity belongs to a low-frequency signal compared with the acceleration change generated by human motion, and therefore the gravity value is eliminated by using a low-pass filtering method.
After gravity elimination is completed, the generation of the acceleration image comprises windowing continuous acceleration data, and generating the acceleration data in each window into a pair of acceleration images according to the magnitude of the acceleration values in the vertical direction and the horizontal direction.
After acceleration data preprocessing is completed to generate an acceleration image, the method constructs an attention deficit hyperactivity disorder detection model based on the algorithm of the convolutional neural network, and the model comprises two stages: off-line training of the model and on-line attention deficit hyperactivity disorder detection.
The off-line training of the model is shown in fig. 5, and comprises the following specific steps:
(a) acquiring motion data of a scene of completing a limb conflict test by a defect attention hyperactivity disorder child and a normal control group in an off-line manner;
(b) processing the motion data by using the acceleration sensor data preprocessing method, and dividing the motion data into training, verifying and testing data sets;
(c) training the constructed convolutional neural network model on the three types of data sets;
(d) the threshold value is set according to the test sample of each individual so that the classification of the diagnosis results of as many individuals as possible is correct.
On-line attention deficit hyperactivity disorder detection is shown in fig. 6, and the specific steps are as follows:
(a) the test object finishes the collection of test data in a detection scene;
(b) completing acceleration data preprocessing according to a data preprocessing method to generate a test data set of the object;
(c) classifying through a detection model;
(d) judging the classification result of the test object according to a set threshold value, and turning to the step (e) if the test object is an attention-deficit hyperactivity disorder group; otherwise, turning to the step (f);
(e) outputting a detection result of attention deficit hyperactivity disorder, and turning to the step (g);
(f) outputting a detection result of 'normal', and turning to the step (g);
(g) and (6) ending.
The specific implementation algorithm of gravity elimination in the acceleration image preprocessing process is shown in the following formula (1).
Figure BDA0002237138640000101
In the formula (1)Representing the acceleration of gravity, belonging to a vector, N representing the number of points contained in the selected window (i.e. the number of data in the window), ax(j),ay(j),az(j) Representing the magnitude of the triaxial acceleration acquired by the motion sensor. After the gravity vector is calculated according to the formula (1), the gravity vector is subtracted from the acceleration vector acquired by the motion sensor to obtain the actual acceleration vector generated only by the motion of the human body. Is represented by the following formula (2), whereinRepresents the actual acceleration vector at the i-th point,
Figure BDA0002237138640000104
representing the three-axis acceleration [ a ] acquired by the sensor in the above formulax(j),ay(j),az(j)]。
Figure BDA0002237138640000105
After the elimination of the gravity is completed, the acceleration generated only by the motion can be obtained
Figure BDA0002237138640000106
Compared with the original data collected by the sensor, the acceleration is further subjected to data analysis, and the motion process of the human body can be reflected better.
The obtained acceleration data without gravity is divided by using a windowing method in the invention. By setting the windowing parameter window length and window step length, continuous acceleration data can be divided into windowed data.
The image sample generation process carries out vector decomposition on the windowed data, and further decomposes the data into a vertical direction (the same as the gravity direction) and a horizontal direction, and the numerical values of the two directions are used as the coordinates of an X axis and a Y axis in a two-dimensional coordinate system. Each point within the window is generated in this way into a picture as input for a further neural network.
The specific algorithm flow is as follows.
(a) Acceleration is resolved into horizontal and vertical directions
Figure BDA0002237138640000111
Figure BDA0002237138640000112
Wherein in the formula (3)
Figure BDA0002237138640000113
Represents the acceleration in the vertical direction and,
Figure BDA0002237138640000114
the representative point is a product of the point,
Figure BDA0002237138640000115
representing the square of the modulus. In the formula (4)
Figure BDA0002237138640000116
Representing the acceleration in the horizontal direction, which is actually in the XOY plane, i.e. to generate a two-dimensional picture, the method only preserves the more important acceleration value information in the XOY plane, leaving the direction information down.
(b) Each point in the window is dispersed into the two-dimensional graph
Each point in the window is dispersed into a two-dimensional graph according to the magnitude of the acceleration value, and the formulas are shown in the following (5) and (6). The (x, y) obtained from equations (5) and (6) can be used as the coordinates of the points in the graph. Wherein midx,lengthx,midy,lengthyIs half of the transverse length, half of the longitudinal length, and the like of the acceleration pattern,Longitudinal length, thx,thyAre dispersion thresholds in the lateral and longitudinal directions.
Figure BDA0002237138640000117
Figure BDA0002237138640000118
image (x, y) + 1 formula (7)
After the pattern dispersion is completed, the image is a two-dimensional grayscale image. In order to make all the pixel values of the two-dimensional gray image within 0 to 1, a normalization method image/N is finally used to make all the pixel values between 0 and 1. The generated acceleration image is a gray scale image, the brightness of each point represents the number of the acquisition points in the acceleration state in the window, and the brighter points represent the larger number of the acquisition points in the acceleration state.
Constructing a detection model and performing offline training:
the present invention is based on a Convolutional Neural Network (CNN) as a detection model for the two-dimensional image generated above. One implementation of the model constructed in the present invention may be a three-layer convolutional layer.
The model used in the invention is divided into three convolution and pooling layers and two full-connected layers.
In convolutional layers, the number of convolutional kernels per layer is doubled in turn. The higher number of low-level convolution kernels facilitates obtaining more low-level information. Each convolutional layer uses a convolution kernel of 5 × 5 size, and the activation function is unified as a modified linear unit (ReLU) function.
Pooling layers in the model consistently used maximum pooling (Max-Pool) of 2 × 2 size.
Two fully-connected layers were used after a random deactivation (Dropout) layer with a deactivation rate of 0.25, where the first fully-connected layer contained 50 nodes with activation functions as modified linear units (relus). The last layer of fully-connected layer has two nodes, and the activation function uses a flexible maximum (Softmax) as an output layer.
The model used by the invention selects the cross entropy as a loss function, and the back propagation algorithm selects a first-order optimization algorithm Adam algorithm for replacing a random gradient descent algorithm in the traditional BP neural network. The method has the characteristic of relatively simple parameter adjustment and is suitable for the condition that an input matrix is sparse, wherein the initial learning rate is set to be 0.001, and the exponential decay rates of first-order moment estimation and second-order moment estimation are respectively 0.9 and 0.999. At the same time, in order to alleviate the problem of overfitting,
in the sample, each picture sample label from a patient with hyperactivity is "hyperactivity", and each picture sample label from a control group is "control group". In the implementation of the present dichotomous model, [1,0] was used as a label for patients with hyperactivity and [0,1] was used as a label for controls.
After the model training is finished, a voting type output result is selected in the testing process aiming at a new patient sample. If the picture samples exceeding the set threshold value in all the samples of the patient are classified as the hyperactivity disorder by the model, the final output result of the patient is the hyperactivity disorder. On the contrary, if any picture sample exceeding the set threshold is classified as a normal sample by the model, the final output result of the patient is the non-hyperactivity disorder.
Experimental results show that the acceleration sensor based on low cost has good identification effect on objective auxiliary diagnosis of attention deficit hyperactivity disorder.
The following are examples of methods of use corresponding to the above-described apparatus examples, and this embodiment can be implemented in cooperation with the above-described embodiment. The related technical details mentioned in the above embodiments are still valid in this embodiment, and are not described herein again in order to reduce repetition. Accordingly, the related-art details mentioned in the present embodiment can also be applied to the above-described embodiments.
The invention also provides a use method of the attention deficit hyperactivity disorder auxiliary diagnosis device, which comprises the following steps:
step 1, interacting with a tested person by using a display screen, and displaying a test scene;
step 2, collecting the limb acceleration of the tested person in the test scene by using a wearable acceleration sensor;
and 3, preprocessing the limb acceleration by using a data processing module to obtain data to be detected, inputting the data to be detected into a deep neural network-based attention deficit hyperactivity disorder detection model for identification to obtain an auxiliary diagnosis result, and sending the auxiliary diagnosis result to the display screen.
The use method of the attention deficit hyperactivity disorder auxiliary diagnosis device comprises a plurality of wearable acceleration sensors which are respectively positioned at the head, the waist, the wrist and the ankle of a person to be tested.
The use method of the attention deficit hyperactivity disorder auxiliary diagnosis device comprises the steps that the test scene comprises a character image, one of four limbs of the character image is displayed as a preset color during a diagnosis test process, a tested person makes a corresponding limb reaction according to the preset color, the wearable acceleration sensor collects the limb reaction, and the acceleration of the limb is recorded.
The method for using the attention deficit hyperactivity disorder-assisted diagnosis device comprises the following steps: and carrying out low-pass filtering on the limb acceleration to eliminate the gravity value, carrying out continuous acceleration data windowing on the low-pass filtered limb acceleration, and generating a pair of acceleration images as the data to be detected according to the acceleration values in the vertical direction and the horizontal direction by using the acceleration data in each window.
The use method of the attention deficit hyperactivity disorder auxiliary diagnosis device is characterized in that the process of eliminating the gravity value is shown as the formula (1):
in the formula (1)
Figure BDA0002237138640000132
Representing the acceleration of gravity, N representing the number of points included in the selected window, ax(j),ay(j),az(j) Representing the three-axis acceleration collected by the wearable acceleration sensor, calculating the gravity acceleration according to the formula (1), and subtracting the gravity vector from the acceleration vector collected by the wearable acceleration sensor to obtain the actual acceleration vector generated only by the human motion
Figure BDA0002237138640000133
Using windowed approach to acceleration vectorsThe continuous acceleration data can be divided into windowed data by setting the window length and the window step length of the windowing parameters;
vector decomposition is carried out on the windowed data, the windowed data are further decomposed into two directions of vertical and horizontal directions, the numerical values of the two directions are used as coordinates of an X axis and a Y axis in a two-dimensional coordinate system, and each point in a window is generated into a picture in the mode, specifically:
(a) acceleration is resolved into horizontal and vertical directions:
Figure BDA0002237138640000141
Figure BDA0002237138640000142
wherein in the formula (3)Represents the acceleration in the vertical direction and,
Figure BDA0002237138640000144
the representative point is a product of the point,
Figure BDA0002237138640000145
represents the square of the modulus, in equation (4)
Figure BDA0002237138640000146
Representing the acceleration in the horizontal direction.
(b) Each point within the window is dispersed into the two-dimensional graph:
dispersing each point in the window into a two-dimensional graph according to the magnitude of the acceleration value, wherein the formulas are shown in the following (5) and (6); (x, y) obtained according to the formulas (5) and (6) can be used as coordinates of points in the graph; wherein midx,lengthx,midy,lengthyTo generate half of the lateral length, half of the longitudinal length, th of the acceleration patternx,thyDispersion thresholds in the lateral and longitudinal directions;
Figure BDA0002237138640000147
Figure BDA0002237138640000148
image (x, y) + 1 formula (7)
After the pattern dispersion is completed, the image is the acceleration image in two-dimensional gray scale.

Claims (10)

1. An attention deficit hyperactivity disorder-assisted diagnosis device comprising:
the display screen is used for interacting with the tested person and displaying a test scene;
the wearable acceleration sensor is used for acquiring the limb acceleration of the measured person in the test scene;
and the data processing module is used for preprocessing the acceleration of the limb to obtain data to be detected, inputting the data to be detected into a deep neural network-based attention deficit hyperactivity disorder detection model for identification to obtain an auxiliary diagnosis result, and sending the auxiliary diagnosis result to the display screen.
2. The device for assisting in diagnosing attention deficit hyperactivity disorder according to claim 1, comprising a plurality of the wearable acceleration sensors respectively located at the head, waist, wrist, and ankle of the subject.
3. The device of claim 1, wherein the test scene comprises a human figure, one of the four limbs of the human figure is displayed as a predetermined color during the diagnostic test, the subject makes a corresponding limb reaction according to the predetermined color, the wearable acceleration sensor collects the limb reaction, and records the acceleration of the limb.
4. The attention deficit hyperactivity disorder-assisted diagnosis device according to claim 1, wherein the preprocessing includes: and carrying out low-pass filtering on the limb acceleration to eliminate the gravity value, carrying out continuous acceleration data windowing on the low-pass filtered limb acceleration, and generating a pair of acceleration images as the data to be detected according to the acceleration values in the vertical direction and the horizontal direction by using the acceleration data in each window.
5. The attention deficit hyperactivity disorder-assisted diagnosis device according to claim 4, wherein the process of eliminating the gravity value is shown in formula (1):
Figure FDA0002237138630000011
in the formula (1)
Figure FDA0002237138630000012
Representing the acceleration of gravity, N representing the number of points included in the selected window, ax(j),ay(j),az(j) Representing the three-axis acceleration collected by the wearable acceleration sensor, calculating the gravity acceleration according to the formula (1), and subtracting the gravity vector from the acceleration vector collected by the wearable acceleration sensor to obtain the actual acceleration vector generated only by the human motion
Figure FDA0002237138630000013
Using windowed approach to acceleration vectors
Figure FDA0002237138630000014
The continuous acceleration data can be divided into windowed data by setting the window length and the window step length of the windowing parameters;
vector decomposition is carried out on the windowed data, the windowed data are further decomposed into two directions of vertical and horizontal directions, the numerical values of the two directions are used as coordinates of an X axis and a Y axis in a two-dimensional coordinate system, and each point in a window is generated into a picture in the mode, specifically:
(a) acceleration is resolved into horizontal and vertical directions:
Figure FDA0002237138630000021
Figure FDA0002237138630000022
wherein in the formula (3)Represents the acceleration in the vertical direction and,
Figure FDA0002237138630000024
the representative point is a product of the point,
Figure FDA0002237138630000025
represents the square of the modulus, in equation (4)
Figure FDA0002237138630000026
Representing the acceleration in the horizontal direction.
(b) Each point within the window is dispersed into the two-dimensional graph:
dispersing each point in the window into a two-dimensional graph according to the magnitude of the acceleration value, wherein the formulas are shown in the following (5) and (6); (x, y) obtained according to the formulas (5) and (6) can be used as coordinates of points in the graph; wherein midx,lengthx,midy,lengthyTo generate half of the lateral length, half of the longitudinal length, th of the acceleration patternx,thyDispersion thresholds in the lateral and longitudinal directions;
Figure FDA0002237138630000027
image (x, y) + 1 formula (7)
After the pattern dispersion is completed, the image is the acceleration image in two-dimensional gray scale.
6. A method for using the device for assisting in diagnosing attention deficit hyperactivity disorder according to any one of claims 1-5, comprising:
step 1, interacting with a tested person by using a display screen, and displaying a test scene;
step 2, collecting the limb acceleration of the tested person in the test scene by using a wearable acceleration sensor;
and 3, preprocessing the limb acceleration by using a data processing module to obtain data to be detected, inputting the data to be detected into a deep neural network-based attention deficit hyperactivity disorder detection model for identification to obtain an auxiliary diagnosis result, and sending the auxiliary diagnosis result to the display screen.
7. The use method of the attention deficit hyperactivity disorder-assisted diagnosis device according to claim 6, which comprises a plurality of the wearable acceleration sensors respectively located at the head, waist, wrist and ankle of the subject.
8. The method as claimed in claim 6, wherein the test scene includes a human figure, one of the limbs of the human figure is displayed as a predetermined color during the diagnostic test, the subject makes a corresponding limb reaction according to the predetermined color, the wearable acceleration sensor collects the limb reaction, and records the acceleration of the limb.
9. The method of using an attention deficit hyperactivity disorder-assisted diagnosis device according to claim 6, wherein the preprocessing includes: and carrying out low-pass filtering on the limb acceleration to eliminate the gravity value, carrying out continuous acceleration data windowing on the low-pass filtered limb acceleration, and generating a pair of acceleration images as the data to be detected according to the acceleration values in the vertical direction and the horizontal direction by using the acceleration data in each window.
10. The method of using an attention deficit hyperactivity disorder-assisted diagnosis device according to claim 9, wherein the process of removing the gravity values is shown in formula (1):
Figure FDA0002237138630000031
in the formula (1)
Figure FDA0002237138630000032
Representing the acceleration of gravity, N representing the number of points included in the selected window, ax(j),ay(j),az(j) Representing the three-axis acceleration collected by the wearable acceleration sensor, calculating the gravity acceleration according to the formula (1), and subtracting the gravity vector from the acceleration vector collected by the wearable acceleration sensor to obtain the actual acceleration vector generated only by the human motion
Using windowed approach to acceleration vectors
Figure FDA0002237138630000034
The continuous acceleration data can be divided into windowed data by setting the window length and the window step length of the windowing parameters;
vector decomposition is carried out on the windowed data, the windowed data are further decomposed into two directions of vertical and horizontal directions, the numerical values of the two directions are used as coordinates of an X axis and a Y axis in a two-dimensional coordinate system, and each point in a window is generated into a picture in the mode, specifically:
(a) acceleration is resolved into horizontal and vertical directions:
Figure FDA0002237138630000035
Figure FDA0002237138630000036
wherein in the formula (3)Represents the acceleration in the vertical direction and,
Figure FDA0002237138630000042
the representative point is a product of the point,
Figure FDA0002237138630000043
represents the square of the modulus, in equation (4)Representing the acceleration in the horizontal direction.
(b) Each point within the window is dispersed into the two-dimensional graph:
root each point in the windowThe values are dispersed into a two-dimensional graph according to the magnitude of the acceleration, and the formulas are shown in the following (5) and (6); (x, y) obtained according to the formulas (5) and (6) can be used as coordinates of points in the graph; wherein midx,lengthx,midy,lengthyTo generate half of the lateral length, half of the longitudinal length, th of the acceleration patternx,thyDispersion thresholds in the lateral and longitudinal directions;
Figure FDA0002237138630000045
Figure FDA0002237138630000046
image (x, y) + 1 formula (7)
After the pattern dispersion is completed, the image is the acceleration image in two-dimensional gray scale.
CN201910987452.6A 2019-10-17 2019-10-17 Auxiliary diagnostic device for attention deficit hyperactivity disorder and using method thereof Pending CN110693510A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910987452.6A CN110693510A (en) 2019-10-17 2019-10-17 Auxiliary diagnostic device for attention deficit hyperactivity disorder and using method thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910987452.6A CN110693510A (en) 2019-10-17 2019-10-17 Auxiliary diagnostic device for attention deficit hyperactivity disorder and using method thereof

Publications (1)

Publication Number Publication Date
CN110693510A true CN110693510A (en) 2020-01-17

Family

ID=69201488

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910987452.6A Pending CN110693510A (en) 2019-10-17 2019-10-17 Auxiliary diagnostic device for attention deficit hyperactivity disorder and using method thereof

Country Status (1)

Country Link
CN (1) CN110693510A (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111539352A (en) * 2020-04-27 2020-08-14 支付宝(杭州)信息技术有限公司 Method and system for judging human body joint motion direction
CN113160967A (en) * 2021-03-12 2021-07-23 中国科学院计算技术研究所 Method and system for identifying attention deficit hyperactivity disorder subtype
CN114376575A (en) * 2022-01-17 2022-04-22 华民康(成都)科技有限公司 System for assisting in identifying patients with childhood hyperkinetic syndrome

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106419938A (en) * 2016-11-28 2017-02-22 中国科学院心理研究所 Attention deficit hyperactivity disorder (ADHD) detection method and system based on kinetic energy release estimation
CN108389000A (en) * 2018-03-09 2018-08-10 新华网股份有限公司 Processing method, device and the computing device of child attention concentration degree information
CN110022768A (en) * 2016-08-26 2019-07-16 阿克利互动实验室公司 The cognition platform coupled with physiology component

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110022768A (en) * 2016-08-26 2019-07-16 阿克利互动实验室公司 The cognition platform coupled with physiology component
CN106419938A (en) * 2016-11-28 2017-02-22 中国科学院心理研究所 Attention deficit hyperactivity disorder (ADHD) detection method and system based on kinetic energy release estimation
CN108389000A (en) * 2018-03-09 2018-08-10 新华网股份有限公司 Processing method, device and the computing device of child attention concentration degree information

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
MARIO MUÑOZ-ORGANERO ET AL: "Automatic Extraction and Detection of Characteristic Movement Patterns in Children with ADHD Based on a Convolutional Neural Network (CNN) and Acceleration Images", 《SENSORS》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111539352A (en) * 2020-04-27 2020-08-14 支付宝(杭州)信息技术有限公司 Method and system for judging human body joint motion direction
CN113160967A (en) * 2021-03-12 2021-07-23 中国科学院计算技术研究所 Method and system for identifying attention deficit hyperactivity disorder subtype
CN114376575A (en) * 2022-01-17 2022-04-22 华民康(成都)科技有限公司 System for assisting in identifying patients with childhood hyperkinetic syndrome
CN114376575B (en) * 2022-01-17 2024-08-20 华民康(成都)科技有限公司 System for assisting in identifying patients with childhood hyperactivity

Similar Documents

Publication Publication Date Title
CN109157231A (en) Portable multi-channel Depression trend assessment system based on emotional distress task
CN104274191B (en) A kind of Psychological Evaluation method and system thereof
CN109069081A (en) For predicting, screening and monitoring encephalopathy/delirium equipment, system and method
US20160029965A1 (en) Artifact as a feature in neuro diagnostics
CN110693510A (en) Auxiliary diagnostic device for attention deficit hyperactivity disorder and using method thereof
CN114224296B (en) Quantitative evaluation system for Parkinson's motor symptoms based on wearable sensor device
CN110236515A (en) A non-contact heart rate detection method based on near-infrared video
CN109124595A (en) Intelligent sphygmus diagnostic method based on dynamic 3 D pulse wave image
CN107273677A (en) A kind of multi-channel nerve function quantitative evaluation system
Chen et al. An interpretable deep learning optimized wearable daily detection system for Parkinson’s disease
Ghosh et al. Multi-modal detection of fetal movements using a wearable monitor
CN115736920A (en) Depression state identification method and system based on bimodal fusion
CN112043273A (en) Brain image data processing method, device, device and storage medium
Ma et al. Deep learning based walking tasks classification in older adults using fNIRS
Jiang et al. Use of deep belief network model to discriminate mild cognitive impairment and normal controls based on EEG, eye movement signals and neuropsychological tests
CN112168167A (en) Method and electronic device for detecting single auditory evoked signal in magnetoencephalography
CN117357114A (en) Auxiliary diagnosis system for attention deficit hyperactivity disorder syndrome based on omnibearing cognition experiment set and multi-flow double-row decision neural network model
TWI734222B (en) A system that combines brain waves and artificial intelligence to diagnose dyslexia
CN116869516A (en) A comprehensive motion assessment method and system based on multi-source heterogeneous data
Shanthakumari et al. Glaucoma Detection using Fundus Images using Deep Learning
WO2007029012A1 (en) Categorising movement data
CN113139449A (en) Physical learning cognitive load evaluation system based on physiological signals
CN115500822B (en) Parkinson detection system and equipment based on wrist tremor signal
US20250040894A1 (en) Personalized chest acceleration derived prediction of cardiovascular abnormalities using deep learning
TWI802486B (en) Method of Interpreting Ultrasonic Images of Children's Kidneys Using Artificial Intelligence

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20200117

RJ01 Rejection of invention patent application after publication