CN111462841A - An intelligent diagnosis device and system for depression based on knowledge graph - Google Patents

Abstract

The invention provides a knowledge graph-based depression intelligent diagnosis device and system, wherein the device comprises: the data acquisition module is used for acquiring human body data of a user; the human body data comprises video data, audio data, electroencephalogram data and heart rate data; the entity attribute value acquisition module is used for acquiring an entity and a corresponding entity attribute value from human body data by adopting a trained learning model; and the knowledge map module is used for connecting the entity and the entity attribute value to form a knowledge map so as to obtain a depression diagnosis result. The invention can intelligently output the depression diagnosis result and assist doctors in diagnosing the depression.

Description

An intelligent diagnosis device and system for depression based on knowledge graph

technical field

The present invention relates to the technical field of an intelligent diagnosis device for depression, and more particularly, to an intelligent diagnosis device and system for depression based on a knowledge graph.

Background technique

Among the different forms of psychological and mental diseases, depression involves the most extensive population. However, due to the deviation of the cognition of depression, many people are ashamed to seek medical treatment, miss the opportunity to seek medical treatment, and some of them have the courage to enter the hospital Because the diagnosis of depression depends heavily on the experience of doctors, the level of doctors varies from place to place, and the doctors’ lives are seriously insufficient, resulting in some patients not being able to receive timely diagnosis and treatment. Failure to diagnose and treat depression in time may bring huge losses to the society, so the use of scientific and technological means to assist doctors in diagnosing depression has become an important research topic in the field of health. This will be of great significance to improving people's health and social stability.

Knowledge graph is essentially a semantic network, whose nodes represent entities or concepts, and edges represent semantic relationships between entities/concepts. Knowledge graphs provide a better ability to organize and manage information. At present, the application of knowledge graph in the medical field is mainly the question answering system. Due to the complexity of the diagnosis of mental diseases, this system is not suitable. Therefore, it is urgent to design a knowledge graph device and system, which is suitable for the diagnosis of depression.

SUMMARY OF THE INVENTION

In order to overcome the shortcomings and deficiencies in the prior art, the purpose of the present invention is to provide an intelligent depression diagnosis device and system based on a knowledge graph, which can intelligently output depression diagnosis results and assist doctors in diagnosing depression.

In order to achieve the above-mentioned purpose, the present invention is realized through the following technical solutions: a kind of intelligent diagnosis device for depression based on knowledge graph, it is characterized in that: comprising:

The data acquisition module is used to collect the user's human body data; the human body data includes video data, audio data, EEG data and heart rate data;

The entity attribute value acquisition module is used to obtain the entity and the corresponding entity attribute value from the human body data by using the learning model;

And a knowledge graph module, which is used to connect entities and entity attribute values to form a knowledge graph to obtain the diagnosis results of depression.

Preferably, in the entity attribute value acquisition module, the learning model includes an expression recognition model, an action recognition model, a dress pattern recognition model, a speech rate and intonation calculation model, a text analysis model, an emotion recognition model and a stress classification model;

Obtain picture sequences and pictures from video data; the entities obtained from the picture sequences by the expression recognition model are expressions; the entities obtained from the picture sequences by the action recognition model include actions and reactions; The entity is the dress form;

The entities obtained from the audio data by the speech rate and intonation calculation model include speech rate and intonation; the entities obtained from the audio data by the text analysis model include semantic information; the entities obtained from the EEG data by the emotion recognition model are emotional information; The entity obtained from the heart rate data using the stress classification model is stress information.

Preferably, the facial expression recognition model, action recognition model, clothing pattern recognition model, speech rate and intonation calculation model and text analysis model respectively adopt a convolutional neural network model or a recurrent neural network model;

The emotion recognition model and the stress classification model use machine learning models.

Preferably, in the entity attribute value acquisition module, entities and corresponding entity attribute values are obtained from medical data through natural language processing technology to train the learning model; in the knowledge graph module, natural language processing technology is used to obtain entities and corresponding entity attribute values from medical data. Obtain the entity and the corresponding entity attribute value, and then construct the knowledge graph through the relationship between the entities.

Preferably, the knowledge graph module is constructed by using a knowledge graph, extracting the relationship between entities, and performing knowledge inference on the constructed knowledge graph to obtain deeper entity relationships, and then obtain an expanded knowledge graph; The knowledge graph is stored in the Neo4j graph database.

Preferably, the knowledge graph module also implements iteration and improvement through a closed-loop system.

A system comprising the above-mentioned intelligent diagnosing device for depression based on knowledge graph, characterized in that: comprising:

The client layer includes a data collection module, a scale generation module for generating an answering scale, a scale answering module for triggering user emotions and filling in the scale, and a reporting module for displaying the diagnosis results of depression. An annotation module for building knowledge graphs;

The data storage layer is used to store the data and knowledge graph from the client layer, and to transfer the data to the data processing layer;

And the data processing layer, which is used to preprocess, extract and classify the data collected by the client to obtain each entity in the knowledge graph and the corresponding entity attribute value, and then calculate the strength index between different nodes, and then construct the knowledge Graph to obtain the diagnosis result of depression; the entity attribute value acquisition module and the knowledge graph module are located in the data processing layer respectively.

Compared with the prior art, the present invention has the following advantages and beneficial effects:

Compared with the existing depression identification device and system, the present invention utilizes natural language processing technology to extract entities and entity attribute values in medical data, then establishes a learning model to calculate entity attribute values, obtains entities and entity attribute values, and calculates each entity. The relationship between them builds a knowledge graph to simulate the doctor's diagnosis process and achieve intelligent and comprehensive depression diagnosis.

Description of drawings

Fig. 1 is the structural block diagram of the depression intelligent diagnosis device based on knowledge graph of the present invention;

Fig. 2 is a structural block diagram of the system of the present invention.

Detailed ways

The present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

Example 1

In this embodiment, an intelligent diagnosis device for depression based on a knowledge graph, as shown in FIG. 1 , includes a data acquisition module, an entity attribute value acquisition module, and a knowledge graph module.

The data acquisition module is used to collect the user's human body data; the human body data includes video data, audio data, EEG data and heart rate data. The camera is used to collect video data, and the microphone is used to collect audio data; in order to judge stress and emotional conditions, a polyconductive physiological instrument can be used to collect stimulated EEG data and heart rate data. Video data is divided into frames into picture sequences, which are stored as picture sequences and pictures respectively, and other data are stored after cleaning.

In the entity attribute value acquisition module, entities and corresponding entity attribute values are obtained from medical data through natural language processing technology to train a learning model; in the knowledge map module, entities and entity attribute values are obtained from medical data through natural language processing technology. Corresponding entity attribute values, and then build a knowledge graph through the relationship between entities.

The entity attribute value acquisition module is used to obtain the entity and the corresponding entity attribute value from the human body data by using the learning model;

In the entity attribute value acquisition module, the learning model includes an expression recognition model, an action recognition model, a dress pattern recognition model, a speech rate and intonation calculation model, a text analysis model, an emotion recognition model and a stress classification model.

Obtain picture sequences and pictures from video data; the entities obtained from the picture sequences by the expression recognition model are expressions; the entities obtained from the picture sequences by the action recognition model include actions and reactions; The entity is the dress form;

The entities obtained from the audio data by the speech rate and intonation calculation model include speech rate and intonation; the entities obtained from the audio data by the text analysis model include semantic information; the entities obtained from the EEG data by the emotion recognition model are emotional information; The entity obtained from the heart rate data using the stress classification model is stress information.

Entities and entity attribute values are as follows:

1) Dress form - sloppy, strange clothes, normal;

2) Speaking speed - fast, slow, normal;

3) Intonation - high, low, normal;

4) Response - too fast, too slow, normal;

5) Expressions - anger, disgust, fear, happiness, sadness, surprise, neutral, tears;

6) Movement - restless, rich body language, many small movements, normal;

7) Stress information - neutrality, stress, pleasure;

8) Emotional information--emotional classification level.

The facial expression recognition model, action recognition model, clothing pattern recognition model, speech rate and intonation calculation model and text analysis model respectively adopt a convolutional neural network model or a recurrent neural network model. Convolutional neural network models include 2D and 3D convolutional neural network models. 2D convolutional neural network models are often used to process two-dimensional information such as pictures, and 3D convolutional neural network models are often used to process three-dimensional information such as videos. Convolutional neural network models generally consist of convolutional layers, pooling layers and fully connected layers. Common convolutional neural network models include LeNet, AlexNet, VggNet, ResNet, etc. The main function of the cyclic neural network model is to process and predict sequence data. It can memorize previous information, and the current output will be affected by the output of the previous period. It is often used in speech processing, and can also be used in video in combination with the convolutional neural network model. Classification. The two deep network models, convolutional neural network model or recurrent neural network model, can complete the two tasks of feature extraction and classification. Therefore, the preprocessed data is directly input into the deep network model, and the hyperparameters of the deep network model are adjusted to train the network. Classification results.

For signal processing such as pictures and videos, there are currently two mainstream methods. One is to manually extract features, and then input the features into the classifier for classification, and the other directly input the data into the deep network model to complete the feature extraction and classification tasks. To obtain the classification results, deep neural networks have achieved rapid development in recent years, and have achieved better results than traditional methods in computer vision, natural language processing and other fields, so the establishment of image and video analysis models mainly uses deep network model methods.

Video-based facial expression recognition and tearing recognition, because the rich background contained in the picture will have a negative impact on the analysis, so it is necessary to use the face detection algorithm to detect the face from the picture, and cut and save the face picture sequence. In deep learning, CNN is often used to extract spatial features of images, and RNN is often used to extract temporal features because of its time series analysis capabilities. Therefore, spatial and temporal features can be extracted by combining the characteristics of the two, which can be used when the data set is small. Similar datasets fine-tune the CNN network trained on large-scale data, the fine-tuned model is used to extract spatial features, and then the spatial features of a certain length are input into the RNN network to extract spatial features, and finally perform classification.

Action-related attribute values include restlessness, rich body language, and many small movements, that is, pay more attention to the frequency of actions, build a video-based action recognition deep network model, and count the number of various actions per unit time. The attribute values of the dress form include sloppy, strange clothes, and normal. Based on image analysis, it only contains spatial information, so it can be classified by using convolutional neural networks.

Intonation is the configuration and change of the tone in a sentence. The lexical meaning of a sentence plus the intonation meaning is the complete meaning. The pitch feature in the audio contains intonation information, and the intonation information can be obtained by obtaining the pitch information.

Speech rate is the lexical speed at which human language symbols express meaning in unit time. The recognition of the speed of speech can be expressed by the number of characters contained in a unit of time from the beginning to the end of a speech. When a small threshold is reached, the speech rate is considered to be slow, and if it reaches a large threshold, the speech rate is considered to be faster.

Speech recognition technology can convert speech signals into text signals. The training of speech recognition models requires a very large amount of data. When the amount of data is insufficient, speech recognition can be performed using the public speech recognition interface, and the obtained text information is processed by natural language. Technology for keyword extraction and semantic understanding.

The emotion recognition model and the stress classification model use machine learning models. Because EEG data and heart rate data contain less information, machine learning models can be used to obtain good classification results. The processing process can be divided into three steps: first, the data is preprocessed, and the interference noise such as current and other physiological signals in the signal is removed by conventional de-manipulation methods to obtain relatively pure EEG signals and heart rate signals; then features are extracted, and traditional signal processing methods are used. Extract linear and nonlinear features according to the characteristics of EEG signals and heart rate signals; then input the features into common classifiers such as SVM, KNN or random forest to get the classification results.

The knowledge graph module is used to connect entities and entity attribute values to form a knowledge graph, obtain the diagnosis results of depression, and realize the iteration and improvement of the entire system through a closed-loop system.

The knowledge graph module is constructed by using the knowledge graph, extracting the relationship between the entities, and performing knowledge reasoning on the constructed knowledge graph to obtain deeper entity relationships, and then obtain the expanded knowledge graph; then store the constructed knowledge graph in the Neo4j graph database.

A knowledge graph is a knowledge network that contains entities, entity attribute values, and relationships between entities. The expression of the knowledge graph is:

G=(E, R, S)

where E={e1,e2,e3,...,en} represents the set of entities, R={r1,r2,r3,...,rn} represents the set of relations, S∈E*R*E represents (entity , relationship, entity) triples, and the connection between entities forms a network-like knowledge structure.

The above module has established the calculation process of entities and entity attribute values. This module uses entities and entity attribute values as nodes in the knowledge graph, uses the relational reasoning model to extract the relationship between entities, and obtains the knowledge graph, which can then be retrieved from knowledge through knowledge retrieval. draw conclusions from the graph.

Store the constructed knowledge graph in the Neo4j graph database. Neo4j is an embedded, disk-based, Java persistence engine with full transactional features. It can store structured data on a network (called a graph from a mathematical point of view) instead of a table, and can later retrieve knowledge from Depression diagnosis results are obtained from the knowledge map. When new data is generated, entities and entity attribute values can be expanded according to the phonetic keywords, the attribute calculation model can be supplemented, the attribute calculation model can be further trained with the new data, and the relational reasoning model can be trained, so as to update the knowledge graph and enable the identification of the depression knowledge graph. higher rate.

Embodiment 2

This embodiment describes a system including the knowledge graph-based intelligent diagnosis device for depression described in Embodiment 1, as shown in FIG. 2 , including a client layer, a data storage layer, and a data processing layer.

The client layer includes a data acquisition module, a scale generation module for generating an answering scale, a scale answering module for triggering user emotions and filling in the scale, and a reporting module for displaying the diagnosis results of depression. An annotation module for building knowledge graphs.

The diagnosis process of depression mainly relies on doctors to make judgments on the situation of patients answering the questions in the scale, so a scale generation module is established to facilitate the establishment of various scales, and they are converted into xml form, and the appropriate scale is used in the scale answering module. displayed to the user in the form of . The scale consists of a series of questions, including applicable diseases, scale types, etc. The questions can include questions, optional answers, and corresponding audio and video materials. Among them, the scale is not limited to the traditional mini scale, depression self-measurement scale, SCL90 scale, etc., but also a 'scale' composed of multiple different scenes or videos used when stress and emotion are induced.

The scale answering module is displayed to the user using a web browser. In the report module, different visual data information is displayed according to different user roles.

The data labeling module is mainly used to mark the collected signals, display the results of the inference of each model, and correct the wrong results by professionals to facilitate further training of the model. The data labeling module is constructed based on the entities of the knowledge graph. The entities include personal information, clothing patterns, speech, behaviors, reactions, and physiological signals. Doctors and experts mark the attribute values of each entity to construct a calculation model of each entity attribute; after the model is perfected, the attribute values of each entity obtained through model analysis are displayed, which is convenient for professionals to correct erroneous results and further train the model.

The report module is used to display the attribute value of each entity and the final result. Different roles pay attention to different content and focuses, so different content is displayed for different roles. Use visualizations to display results more easily understandable.

The front-end pages are written based on the react framework, the BFF layer is written based on the egg framework, and the background code is written in java. Data visualization can be achieved using the javascript library Data-Driven Documents, which uses the SVG format, allowing the rendered shapes to be scaled up or down without loss of quality.

The data storage layer is used to store the data and knowledge graphs from the client layer, and to transfer the data to the data processing layer; it plays the role of connecting the client layer and the data processing layer. The user's basic information can be saved using the relational database mysql, the knowledge graph can be saved using the graph database Neo4j, and the audio, video, and picture data used in the scale, as well as the collected physiological signals, audio, and video information can be stored in OSS cloud storage.

The data processing layer is used to preprocess, extract and classify the data collected by the client to obtain each entity in the knowledge graph and the corresponding entity attribute value, and then calculate the strength index between different nodes, and then construct the knowledge graph. The diagnosis result of depression is obtained; the entity attribute value acquisition module and the knowledge map module are respectively located in the data processing layer. In the data processing layer, call each model interface to obtain various information from the oss side for calculation and return to the data storage layer, and then display it in the annotation module; retrieve the knowledge graph to draw the final conclusion, return the result to the data storage layer, and then Displayed in the report module.

Compared with the existing depression identification method and system, the present invention utilizes natural language processing technology to extract entities and entity attribute values in medical data, then establishes an algorithm model to calculate entity attribute values, obtains entities and entity attribute values, and calculates each entity. The relationship between groups builds a knowledge map to achieve intelligent and comprehensive depression diagnosis.

The above-mentioned embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited by the above-mentioned embodiments, and any other changes, modifications, substitutions, combinations, The simplification should be equivalent replacement manners, which are all included in the protection scope of the present invention.

Claims (7)

1. a depression intelligent diagnosis device based on knowledge map, is characterized in that: comprise: The data acquisition module is used to collect the user's human body data; the human body data includes video data, audio data, EEG data and heart rate data; The entity attribute value acquisition module is used to obtain the entity and the corresponding entity attribute value from the human body data by using the trained learning model; And a knowledge graph module, which is used to connect entities and entity attribute values to form a knowledge graph to obtain the diagnosis results of depression. 2. The knowledge graph-based intelligent diagnosis device for depression according to claim 1, wherein: in the entity attribute value acquisition module, the learning model comprises an expression recognition model, an action recognition model, a dress form recognition model, a speech rate Tone calculation model, text analysis model, emotion recognition model and stress classification model; Obtain picture sequences and pictures from video data; the entities obtained from the picture sequences by the expression recognition model are expressions; the entities obtained from the picture sequences by the action recognition model include actions and reactions; The entity is the dress form; The entities obtained from the audio data by the speech rate and intonation calculation model include speech rate and intonation; the entities obtained from the audio data by the text analysis model include semantic information; the entities obtained from the EEG data by the emotion recognition model are emotional information; The entity obtained from the heart rate data using the stress classification model is stress information. 3. The intelligent diagnosis device for depression based on knowledge graph according to claim 2, is characterized in that: described facial expression recognition model, action recognition model, dress form recognition model, speech rate and intonation calculation model and text analysis model respectively adopt volume product neural network model or recurrent neural network model; The emotion recognition model and the stress classification model use machine learning models. 4. The intelligent diagnosis device for depression based on knowledge graph according to claim 1, characterized in that: in the entity attribute value acquisition module, the entity and the corresponding entity attribute value are obtained from medical data through natural language processing technology to carry out Training a learning model; in the knowledge graph module, entities and corresponding entity attribute values are obtained from medical data through natural language processing technology, and then a knowledge graph is constructed through relationships between entities. 5. The intelligent diagnosis device for depression based on a knowledge graph according to claim 1, wherein the knowledge graph module adopts a knowledge graph to realize construction, extracts the association relationship between each entity, and performs knowledge on the constructed knowledge graph. Inference, get deeper entity relationships, and then get the expanded knowledge graph; then store the constructed knowledge graph in the Neo4j graph database. 6 . The intelligent diagnosis device for depression based on a knowledge graph according to claim 1 , wherein the knowledge graph module also implements iteration and improvement through a closed-loop system. 7 . 7. A system comprising the knowledge graph-based intelligent diagnosis device for depression according to any one of claims 1 to 6, characterized in that: comprising: The client layer includes a data acquisition module, a scale generation module for generating an answering scale, a scale answering module for triggering user emotions and filling in the scale, and a reporting module for displaying the diagnosis results of depression. An annotation module for building knowledge graphs; The data storage layer is used to store the data and knowledge graph from the client layer, and to transfer the data to the data processing layer; And the data processing layer, which is used to preprocess, extract and classify the data collected by the client to obtain each entity in the knowledge graph and the corresponding entity attribute value, and then calculate the strength index between different nodes, and then construct the knowledge Graph to obtain the diagnosis result of depression; the entity attribute value acquisition module and the knowledge graph module are located in the data processing layer respectively.

Images