CN107273677A - A kind of multi-channel nerve function quantitative evaluation system - Google Patents

Abstract

The invention discloses a multi-channel neural function quantitative evaluation system, which uses key technologies such as intelligent perception, multi-channel fusion, and natural interaction, and uses equipment such as depth cameras, handheld terminals, handwriting terminals, microphones, data transponders, and mobile workstations to complete the construction. A set of human movement posture data collection system, which combines the medical detection methods of perception, movement, and cognition with multi-channel data fusion analysis technology, establishes an abnormality system covering gesture analysis, gait analysis, voice analysis, and upper limb function analysis. The multi-channel neurological function quantitative evaluation system for early detection of behavior overcomes the difficulty of obtaining more comprehensive and quantitative physiological parameters in traditional nervous system disease detection tools, and eliminates the interference of traditional collection methods on sensory data. Influence on authenticity and accuracy of subsequent diagnosis.

Description

A multi-channel neurological function quantitative evaluation system

technical field

The invention belongs to the fields of computer application and medical rehabilitation, and in particular relates to a multi-channel nerve function quantitative evaluation system.

Background technique

Human society has entered an aging society, and chronic diseases that harm the elderly have become important diseases that threaten human health. Among them, the incidence of cerebrovascular diseases, dementia, and Parkinson's disease, which affect the nervous system function, is on the rise worldwide, with the characteristics of high mortality and high disability rate, which seriously affects the life expectancy and quality of life of patients. bring a heavy burden.

The diagnosis process of these diseases usually needs to go through the process of collecting clinical data, making judgments by doctors based on knowledge and experience, and finally forming a diagnosis. Therefore, the evaluation and collection of in vivo data on the function and structure of the nervous system is the cornerstone of the diagnosis and research of nervous system diseases.

The nervous system includes three functions of perception, cognition, and movement. Due to the functional positioning characteristics of the nervous system, the evaluation of the above three functions can most directly determine whether the nervous system is damaged and where it is damaged. For a long time, various somatosensory, motor function evaluation methods and cognitive function evaluation methods have been widely used in clinical practice, and can be used to evaluate vision, hearing, touch, memory, attention, operational executive function, muscle strength, muscle tension, coordination, Balance and many other neurological functions.

Neurologists and neurological disease researchers use the evaluation methods described above in their daily work. Obviously, these evaluation methods rely entirely on the experience of the evaluators, and perform subjective and qualitative evaluation of neurological function. Although some evaluation methods have a grading concept, this grading is undoubtedly not quantitative data of equivalent differences. Its shortcomings mainly include the following aspects: ① The subjective judgment of the evaluator determines the evaluation result; ② Non-quantitative data; ③ Relying on on-site evaluation, the degree of informatization and digitization is low, and the key elements of the detection cannot be stored and played back for analysis.

Therefore, due to the complexity of the structure and function of the nervous system, the existing in vivo data evaluation and collection methods have the characteristics of relying on subjective experience judgments, being unable to quantify and lacking integrity, which brings great benefits to the clinical diagnosis and clinical research of nervous system diseases. of uncertainty.

Contents of the invention

In view of the above problems, the present invention provides a multi-channel neurological function quantitative evaluation system, using key technologies such as intelligent perception, multi-channel fusion, natural interaction, etc., using depth cameras, handheld terminals, handwriting terminals, microphones, data transponders, mobile workstations and other equipment to complete a set of human motion posture data acquisition system, combining the medical detection methods of perception, motion, and cognition with multi-channel data fusion analysis technology, and establishing a system covering gesture analysis, gait analysis, speech analysis, upper limb function analysis, etc. A multi-channel neurological function quantitative evaluation system for early detection of abnormal behaviors in neurological diseases.

In order to achieve the above object, the present invention adopts the following technical solutions:

A multi-channel neurological function quantitative evaluation system, comprising: human body gait and posture data acquisition and analysis module, hand movement function feature acquisition module, speech feature acquisition module and multi-channel data fusion analysis module;

The human body gait and posture data acquisition and analysis module is used to collect the gait and body posture data of the human body when walking in a natural state and analyze and obtain corresponding characteristic data;

The hand movement function feature acquisition module is used to collect hand movement function feature data related to nervous system diseases during touch and writing;

The voice feature collection module is used to collect voice feature data related to neurological diseases in daily conversations and medical testing;

The multi-channel data fusion analysis module is used to fuse the feature data obtained by the human gait and posture data collection and analysis module, the hand movement function feature collection module and the speech feature collection module, and associate them with specific nervous system diseases , to obtain the parameters of the quantitative evaluation model of nervous system diseases, so as to be used in the quantitative evaluation of neurological function.

Further, the human gait and posture data collection and analysis module includes a single or multiple depth cameras arranged in the data collection space, and the gait and body posture data collected by the depth cameras include color image streams, depth image streams , infrared image stream, sound data stream, and skeletal joint point attitude data stream. The characteristic data analyzed by this module include attitude parameters such as speed, acceleration, trajectory, angle, duration, cycle, step height, pace, and step distance.

Further, the human body gait and posture data acquisition and analysis module also includes a data playback labeling sub-module, which is used to manually monitor and analyze features (such as finger pinching parameters, fast rotation parameters, etc.) that require manual supervision and analysis by medical personnel. label.

Further, the hand movement function feature collection module includes a hand-held terminal and a handwriting terminal, and the hand-held terminal is used to collect hand information related to nervous system diseases through the sensor of the hand-held terminal itself by monitoring the interactive actions in the process of using the hand-held terminal. Hand movement function feature data (including sound, touch pressure, touch area, hand tremor), the handwriting terminal is used to collect hand movement function feature data (including pressure, trajectory, hand tremor) related to nervous system diseases during handwriting input attitude inclination).

Further, the handheld terminal also includes a wireless control terminal module, which is used to remotely control the entire system.

Further, the wireless control terminal module further includes: an information input submodule, used to scan and input the information of the collected person (mainly the ID number of the collected person); a scale score input submodule, used to input the information collected The scale score of the operator (handedness, dysarthria, etc.); the timing sub-module of the inspection item is used to record the time of the inspection item (start time, stop time, duration); the remote control sub-module is used for data collection. Remote control of the process (start, pause, stop of data collection, information of the person to be collected, and record of the running status of the collection system).

Further, the speech feature collection module is used to collect speech data during daily conversations and medical testing through a microphone, including continuous vowel pronunciation (such as "ah....") or continuous sentence pronunciation (a sentence or a paragraph speech), to obtain speech features related to neurological diseases.

Further, the multi-channel data fusion analysis module collects a large amount of motor function test data of normal unaffected people and patients with neurological diseases, and calculates the characteristic parameter values of their motor behavior. On this basis, the above characteristic parameters Data correlation analysis between the value and specific neurological diseases, determine the normal value range of neurological function from the characteristic parameters of the normal non-diseased population, and compare and analyze with the characteristic parameters of the corresponding neurological disease patients, and obtain the correlation between neurological diseases The model parameters of the associated quantitative neurological function evaluation are used for the quantitative evaluation of neurological function.

The beneficial effects of the present invention are as follows:

Based on the feature-level, intermediate-level and semantic-level fusion analysis data provided by the multi-channel physiological interaction knowledge base, the present invention integrates a mixed classification model of nervous system disease diagnosis rules and statistical inference models to construct quantitative prediction, screening, diagnostic model. Combine medical detection methods with intelligent perception, multi-channel fusion, and natural interaction, perceive and understand key features such as user connection, gait, voice, and upper limb function analysis. Under the guidance of neurological function evaluation methods, combined with nervous system-related diseases The specific data, analyze and mine the key features corresponding to the symptoms, write data analysis algorithms, and finally establish a multi-channel neurological function quantitative evaluation system.

Collect data on medical movements and postures without interference, and use sensors such as vision, touch and stylus to collect medical data related to nervous system diseases in the state of natural walking, touch operation and writing, thus overcoming the traditional The behavioral characteristics collected by wearable medical sensors are not completely consistent with the behavioral characteristics under normal living conditions, and the influence of the interference of the collection method on the authenticity of the perceived data and the accuracy of subsequent diagnosis is eliminated.

Quantitatively collect medical feature data of multiple channels, including data parameters of upper limbs, lower limbs, sitting posture, writing, touch, voice and other motor behaviors, and integrate and analyze the mapping between heterogeneous sensory data collected in multiple modalities and specific nervous system diseases It overcomes the problem that it is difficult to obtain more comprehensive and quantitative physiological parameters by traditional nervous system disease detection tools.

Description of drawings

Fig. 1: Technical architecture diagram of the multi-channel neural function quantitative evaluation system of the present invention.

Fig. 2: The technical architecture diagram of collecting human gait and body posture data in the present invention.

Fig. 3: The flow chart of the present invention to realize the quantitative evaluation of multi-channel neurological function.

Figure 4: An example of a depth map collected by the depth camera of the present invention.

detailed description

The multi-channel neurological function quantitative evaluation system is shown in Figure 1, mainly including:

1) Depth camera: collect the real-time color map and depth map data stream of the gait and body posture of the human body when walking in the natural state, which can be connected with the data collection sub-station, and the collected data will be stored in the The data collection substation or the local memory connected with the data collection substation. The time synchronization with the mobile workstation is carried out through the data transponder, the status information in the collection process is transmitted, and it is monitored by the mobile workstation. When there are two depth cameras deployed in the detection area, the technical architecture diagram of data collection is shown in Figure 2. The two depth cameras collect real-time images of the gait and body posture of the human body when walking in a natural state at different angles. Color map, depth map data stream, infrared image stream, sound data stream, among them, the infrared image stream is used for target bone data tracking under extreme conditions, specific implementation: by letting the target patient wear 3M retroreflective strip markers, in the infrared The highlighted pixels in the image are used to complete the tracking of the target patient in video scenes and complex extreme situations, and the audio data stream is used for remote voice control of medical staff. At the same time, the two depth cameras are respectively connected to two data collection sub-stations, and the collected data is stored in the data collection sub-station or a local memory connected to the data collection sub-station through the data collection sub-stations. The time synchronization with the mobile workstation is carried out through the data transponder, the status information in the collection process is transmitted, and it is monitored by the mobile workstation.

2) Handheld terminal: It contains two functions, one is used as a collection device, used by the person to be collected, for touch interaction, and to collect the user's upper limb hand movement data in the task state, including two-finger alternate buttons, dialing and drawing lines Action time, contact pressure, contact area, trajectory, space posture of equipment, etc. When collecting data, the person to be collected is required to perform specific data collection steps including two-finger alternate keys, dialing, drawing specific patterns of lines, etc., with fingers, and the collected data is saved locally; the other is used as a wireless control system Terminal, used by the system administrator, communicates with and controls the mobile workstation through wireless, and can remotely control the entire data collection system, including scanning and entry of the information of the collected person (mainly the ID number of the collected person), quantity Table score entry (handedness, poor articulation, etc.), timing of inspection items (start time, stop time, duration), remote control of the server-side data collection process (data collection start, pause, stop, information on the person being collected) , acquisition system running status record) and other functions.

3) Handwriting terminal: collect hand movement characteristic data during handwriting input. Including pressure, trajectory, attitude inclination, etc., and stored in the connected data acquisition substation.

4) Microphone: Collect speech data during daily conversations and medical testing, including continuous vowel pronunciation (such as "ah...") or continuous sentence pronunciation (one sentence or a paragraph), and obtain information related to neurological diseases. The speech characteristics (such as: vocal fold vibration cycle, noise inclusions, vibration state of vocal organs, etc.) are evaluated for neurological diseases.

5) Data transponder: Build a communication network between each acquisition device and mobile workstation through wired or wireless, and forward commands and information such as time synchronization commands, acquisition start and stop commands, and working status. Due to the large amount of collected image data flow, the data collected by the depth camera and the data collected by the handheld terminal are transmitted and stored in the local storage device.

6) Mobile workstation: the core control device of the entire data acquisition system, connected with the data transponder by wire or wirelessly. Time synchronization of the entire data acquisition system and key information synchronization. Monitor and record the running status of the entire data acquisition system. Instructions can be sent to specific data collection substations to control the data collection process. Receive the collection data flow uploaded by the data collection sub-station, and dump it into the local memory.

7) Data collection substation: It can be connected with depth camera or handwriting terminal to collect data, and store the collected data in itself or in the connected local memory. Through wired or wireless connection with the data transponder, build a communication network with the mobile workstation, perform time synchronization, upload key information, and be monitored by the mobile workstation.

The present invention realizes the flow process of multi-channel neural function quantitative evaluation as shown in Figure 3, mainly includes:

According to the nervous system motor function inspection process in the medical field, non-interference multi-modal motion data collection is carried out, including the collection of medical data related to nervous system diseases in the natural walking, touch operation, and writing states of the collected subjects.

Among them, the collection of medical data related to neurological diseases in the natural walking state includes rapid rotation, seat standing, 3-meter walking, etc., using depth cameras to collect color images of the human body's gait and body posture, depth Image (as shown in Figure 4) data. Analyze the real-time color map and depth map data stream in the collected data to construct the 3D point cloud data of the human body during walking; Calculate the corresponding attitude parameter data including speed, cycle, step height, pace and step distance. For some features that require manual supervision and analysis, such as finger kneading parameters, fast rotation parameters, etc., it is necessary to use the playback labeling function of the system to be manually marked by medical professionals.

The collection of medical data related to neurological diseases under touch operation is to use handheld terminals, such as mobile phones, to develop functional testing applications for neurological diseases, such as line drawing, unlocking, and dialing. By monitoring the interactive actions in daily mobile phone use, use The sensor of the mobile phone itself collects hand movement function characteristic data related to nervous system diseases such as voice (call voice or medical voice detection application), touch pressure, touch area, and hand tremor.

The collection of medical data related to nervous system diseases in the writing state is to use handwriting terminals to conduct cognitive examinations (including MMSE, MOCA, etc.), and to collect hand movement characteristic data during the writing process, including pressure, trajectory, posture inclination, etc. Data, analysis and extraction include characteristic data such as completion time, number of errors, and average bending degree.

Through the application of the multi-channel neurological function quantitative evaluation system, the gesture, gait, voice, and upper limb function tests of the normal unaffected population and patients with neurological diseases are performed, the corresponding medical examination data are collected, and the characteristic parameter values of their motor behaviors are calculated. , on this basis, the pathological data correlation analysis of the above characteristic parameter values and specific nervous system diseases is carried out, and through long-term follow-up monitoring of normal non-diseased population, based on the distribution interval definition of the characteristic parameter values of normal non-diseased population Out of the normal range of neurological function, compared with the characteristic parameters of the corresponding patients, and combined with the doctor's diagnosis conclusion to carry out data calibration and verification, the model parameters of quantitative neurological function evaluation associated with the disease can be obtained, so that it can be used For the prediction, screening and auxiliary diagnosis of specific neurological diseases.

Afterwards, when any subject is tested, it is only necessary to carry out the inspection items of these characteristic parameters through this system, quantitatively collect the relevant parameter values, input the model for comparison, and then obtain the quantitative evaluation results of neurological function, thus giving The likelihood of illness is advised.

Claims (10)

1. a kind of multi-channel nerve function quantitative evaluation system, including：Body gait and attitude data collection analysis module, hand Motor function collection apparatus module, phonetic feature acquisition module and multi-channel data convergence analysis module；

Body gait and attitude data the collection analysis module is for gathering gait and body during human body walking under nature Body attitude data simultaneously analyzes the corresponding characteristic of acquisition；

The hand movement function collection apparatus module is used to gather hand related to the nervous system disease when touch-control and writing Motor function characteristic；

The phonetic feature acquisition module is used to gather related to the nervous system disease in every-day language and medical science detection process Voice feature data；

The multi-channel data convergence analysis module is used to transport the body gait and attitude data collection analysis module, hand The characteristic that dynamic functional character acquisition module and phonetic feature acquisition module are obtained is merged, and with specific nervous system disease Disease is associated, the nervous system disease quantitative evalution model parameter is obtained, for nervous function quantitative assessment.

2. a kind of multi-channel nerve function quantitative evaluation system as claimed in claim 1, it is characterised in that the body gait And attitude data collection analysis module includes being laid in single or multiple depth cameras in data acquisition space.

3. a kind of multi-channel nerve function quantitative evaluation system as claimed in claim 2, it is characterised in that the depth camera The gait and body posture data that machine is collected include color flow image, depth image stream, infrared image stream, audio data stream and Skeletal joint point attitude data stream.

4. a kind of multi-channel nerve function quantitative evaluation system as claimed in claim 2, it is characterised in that the body gait And the obtained characteristic of attitude data collection analysis module analysis include speed, acceleration, track, angle, duration, the cycle, Step height, leg speed and step pitch.

5. a kind of multi-channel nerve function quantitative evaluation system as claimed in claim 1, it is characterised in that the body gait And attitude data collection analysis module also include data readback mark submodule, for by medical science related personnel to need carry out The feature of manual oversight analysis is manually marked.

6. a kind of multi-channel nerve function quantitative evaluation system as claimed in claim 1, it is characterised in that the hand exercise Functional character acquisition module includes handheld terminal and hand-written terminal, and the handheld terminal is used to use handheld terminal mistake by monitoring Interactive action in journey, gathers the hand movement function related to the nervous system disease special by the sensor of handheld terminal in itself Data are levied, the hand-written terminal is used to gather hand movement function characteristic related to the nervous system disease during handwriting input According to.

7. a kind of multi-channel nerve function quantitative evaluation system as claimed in claim 6, it is characterised in that the handheld terminal Also include wireless control terminal module, for carrying out remote control to whole system.

8. a kind of multi-channel nerve function quantitative evaluation system as claimed in claim 7, it is characterised in that the controlled in wireless Terminal module includes again：Data Enter submodule, typing is scanned for the information to gathered person；Scale score typing Module, the scale score for typing gathered person；The timing submodule of inspection project, for record check object time； Remote control submodule, remote control is carried out for the process to data acquisition.

9. a kind of multi-channel nerve function quantitative evaluation system as claimed in claim 1, it is characterised in that the phonetic feature Acquisition module is used for by the voice feature data during microphone collection every-day language and in medical science detection process.

10. a kind of multi-channel nerve function quantitative evaluation system as claimed in claim 1, it is characterised in that the voice is special Data are levied including sustained vowel to pronounce or continuous statement pronunciation.