KR102314213B1 - System and Method for detecting MCI based in AI - Google Patents

Abstract

The present invention provides a content generation unit for generating content to induce speech of a subject to be detected, a connection device determination and content providing unit for determining a connection device used by the subject to be detected and providing the content accordingly, and an utterance signal of the subject to be detected A consonant separation unit for recognizing voice information by receiving and extracting consonants and vowels from the voice information; MCI comprising: a conversion unit; a feature extracting unit for extracting at least one speech characteristic by receiving the spectrum from the spectrum conversion unit; A detection device is provided.

Description

Method and apparatus for detecting mild cognitive impairment using artificial intelligence technology {System and Method for detecting MCI based in AI}

The present invention relates to a method and apparatus for detecting mild cognitive impairment.

Around 2018, the domestic baby boom generation began to exceed the age of 60, and the elderly population is rapidly increasing, and 10% of the elderly population corresponds to dementia patients. Meanwhile, the number of people aged 50 and over in Korea is 21 million as of 2019, accounting for 40% of the total population in Korea.

The social problems caused by dementia are still in a very serious stage, and for the above reasons, the increase in dementia patients is expected to be steeper in the future, and the social cost is expected to exceed 40 trillion won in the future.

On the other hand, although a technology or therapeutic agent for curing dementia has not yet yielded clear results, various methods for delaying the progression of dementia have already been developed. After all, if an early diagnosis is possible, even if treatment is difficult, a higher quality of life can be guaranteed by delaying the progression of dementia even with the current level of medical technology.

Therefore, early diagnosis of mild cognitive impairment (MCI), which is the early stage of dementia or the stage immediately preceding dementia, is very important.

However, the conventional MCI or dementia diagnosis method is made in a hospital and is only possible through expensive medical imaging equipment such as MRI and PET-CT as disclosed in Patent Registration No. 10-1929965, so the personal/social cost of diagnosis is high. There is a problem.

In many cases, early diagnosis of MCI is not possible due to the high cost of diagnosis and the inconvenience of visiting the hospital, so the reality is that many elderly people cannot respond properly just before or in the early stages of dementia and are moving to a severe stage.

In order to solve the above-mentioned problems, the present invention is to provide a method and apparatus for detecting MIC that can detect MCI at a low cost and have high convenience in use.

Another object of the present invention is to provide a method and apparatus for detecting an interactive MIC based on voice recognition that cause the least discomfort and psychological rejection of the subject to be detected.

In addition, it is an object of the present invention to provide an MCI detection method and apparatus that performs MCI detection detection with sufficient accuracy using artificial intelligence technology, and the detection accuracy becomes progressively higher.

A mild cognitive impairment (MCI) detection device according to an aspect of the present invention includes a content generation unit that generates content to induce a utterance of a detection target, and a connection device that determines a connection device used by the detection target and provides the content accordingly A device identification and content providing unit, a consonant separation unit for extracting consonants and vowels by recognizing voice information from the received speech signal of the detection target, a consonant arrangement unit for aligning the separated consonants, and a consonant arrangement A spectrum converter that converts a speech sound into a frequency domain, a feature extractor that receives a spectrum from the spectrum converter and extracts at least one speech feature; includes wealth.

The connection device determination and content providing unit converts the prepared text content into voice when the connection device used by the user is a wired phone capable of only voice conversation, and transmits the audio and video content in the case of a smart device with a display device.

The consonant-vowel aligning unit extracts the utterance time for each phoneme from the recognized spoken voice and transmits it to the feature extraction unit, and furthermore, the interval between each word, the average of the interval between words in the entire speech, the frequency of use of the specific word, the specific word and at least one of the blank time of the following word may be extracted and transmitted to the feature extraction unit.

The feature extraction unit extracts at least one of a formant value, a pitch value, a decibel value, and a density value for each phoneme as a feature value. In addition, the process of extracting a differential value that can be distinguished from other tones through comparison with a general average value for each of the extracted feature values, analysis of the amount of change in values, and analysis of changes in values may be further performed.

The MCI determining unit generates a learning model through supervised learning based on the at least one speech feature in the learning step, and detects whether MCI is present based on the at least one speech feature and the learning model in the inference step.

According to another aspect of the present invention, collecting a speech response for content for inducing speech, extracting at least one feature from the speech response, and learning by performing learning based on the at least one feature There is provided an MCI sensing method comprising generating a model, and detecting whether the sensing subject is MCI based on the learning model and the speech characteristics of the sensing subject.

It may further comprise the step of notifying the result to the subject detected as MCI and providing a guide for recommending hospital diagnosis, in addition or optionally, making a diagnosis reservation to a pre-contracted hospital on behalf of the subject and sending the reservation result to the sensing subject It may further include the step of notifying.

In addition, although the learning is performed based on the speech data of a normal person, the speech data of the subject himself/herself is continuously accumulated for every utterance together or separately, and the learning model can be built and updated by learning based on this.

According to the present invention, it is possible to diagnose early MCI easily and inexpensively compared to the current dementia screening process conducted in specialized institutions such as hospitals, thereby improving the convenience of use.

It is not a temporary or one-time test, but it is possible to increase the detection accuracy and detect MCI early by detecting MCI as a result of continuous observation of the subject through continuous state change data accumulation.

In addition, by adopting a voice recognition-based sensing method, the subject to be diagnosed can participate in the diagnosis without hesitation through the AI conversation service, and it is possible to provide various information and further diagnose the subject's psychological state.

It is easy to construct learning/empirical data for real-user-centered MCI detection because a higher degree of participation can be obtained compared to the conventional method.

1 is a conceptual diagram for explaining the basic idea of the present invention.
2 is a block diagram of an MCI sensing device according to an embodiment of the present invention.
3 is a time-frequency graph in which an MCI sensing device extracts formants for each phoneme of a spoken voice according to the present invention;
4 is a block diagram of an MCI sensing device according to another embodiment of the present invention.
5 is a block diagram of an MCI sensing device according to another embodiment of the present invention.
6 is a block diagram of an MCI sensing device according to another embodiment of the present invention.
7 is a flowchart of an MCI detection method according to an embodiment of the present invention.

Objects and effects of the present invention are not limited to those mentioned above, and the objects and effects of the present invention, and technical configurations for achieving them will become clear with reference to the embodiments described below in detail in conjunction with the accompanying drawings.

In describing the present invention, if it is determined that a detailed description of a well-known function or configuration may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms. Each of the following examples are provided so that the disclosure of the present invention is complete, and to completely inform those of ordinary skill in the art to which the present invention belongs, the scope of the invention, and it is intended to limit the scope of the present invention no.

Throughout the specification, when a part "includes" or "includes" a certain element, it means that other elements may be further included, rather than excluding other elements, unless otherwise stated. . In addition, terms such as "...unit", "...device", "...device", "...unit" or "...module" described in the specification refer to at least one function or operation. It means a processing unit, which may be implemented as hardware or software or a combination of hardware and software.

On the other hand, in each embodiment of the present invention, each component, functional blocks or means may be composed of one or more sub-components, and the electrical, electronic, mechanical, and logical functions performed by each component are An electronic circuit, an integrated circuit, an ASIC (Application Specific Integrated Circuit), etc. may be implemented as various well-known elements, programs, or mechanical elements, and may be implemented separately or two or more may be integrated into one.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

The basic technical idea of the present invention is to detect whether mild cognitive impairment (MCI) is present by analyzing the subject's voice. That is, as shown in FIG. 1 , when the sensing target transmits his/her voice to the remote MCI sensing device 100 through the device 200 such as a landline phone, a smart phone, or a PC, the MCI sensing device is artificial intelligence-based to detect signs of MCI. According to the current medical law, the final decision must be made by a medical person, so the MCI detection device of the present invention informs the subject when MCI signs are found and induces them to visit a hospital and receive MCI judgment through precise diagnosis, so that MCI can be diagnosed early. let it be

The speech of patients with dementia after the middle stage has a slurred or other characteristic that is different from that of a normal person, so even ordinary people without specialized knowledge can recognize or detect it relatively easily. It is relatively difficult to determine whether MCI is present and the accuracy is low.

Therefore, in the present invention, consonants and vowels are separated from the subject's voice, and the waveform pattern of the vowel is analyzed to determine whether MCI is there, but the MCI is inferred by learning the pattern of a normal person and the pattern of an MCI patient based on an artificial neural network.

Furthermore, through the interactive service, the detection accuracy is further improved by adding MCI detection according to the linguistic interpretation aspect such as the subject's vocabulary selection, the response speed during conversation, and the content of the conversation.

The greatest advantage of the present invention is that it is possible to accumulate characteristic data (including voice pattern, used vocabulary, response speed, conversation content, etc.) of the subject's unique normal state through continuous interactive service from the normal state, and such subject characteristics The point is that the accuracy of MCI detection can be further improved by judging by combining data and other people's big data.

Hereinafter, the basic configuration of the present invention will be described in more detail.

As shown in FIG. 2 , the MCI sensing device 100 according to the present invention includes a content generating unit 110 , an access device determining and content providing unit 120 , a consonant separation unit 130 , a consonant vowel aligning unit ( 140 ), a spectrum conversion unit 150 , a feature extraction unit 160 , and an MCI determination unit 170 .

The content generation unit 110 generates content for inducing the detection target's voice. Since the content can be provided in various forms such as video, audio, and text according to the connection device (landline phone, smart device, PC, etc.) used by the target, the content is generated for each of these forms.

Contents include vocalization-guided content that induces a specific pronunciation (a specific word), short-answer content to identify intellectual abilities (a question such as expressing a deer, an elephant, and a lion in order and telling them in this order), a specific topic or topical It takes the form of questions or invitations that lead to vocalizations and conversations, such as content-guided content that induces content responses to issues.

Speech-inducing content is used for MCI detection through phonemic speech pattern analysis, which is the basic gist of the present invention, and short-answer-type content is used to supplement MCI detection by analyzing the intellectual activity level of the subject, and content-derived content is It can be used to supplement MCI detection through semantic analysis of the subject's conversational content.

Considering that the age of the target is high, it is preferable that the provision of content is basically performed based on voice. For example, it is preferable that the voice guidance is also provided even when short-answer content is provided in which the animal images are provided in order and then the animal's name is provided in the same order.

To this end, the content generating unit 110 continuously understands the subject's response, including an artificial intelligence chatbot (not shown), and provides interactive content suitable for the counterpart, status, and content of the conversation so that the conversation proceeds and interacts. It is preferable to

The connection device determination and content providing unit 120 determines whether the user device that the subject connects to the MCI sensing device according to the present invention is a landline phone, a smart device (smartphone, tablet, etc.), a PC (including a laptop), etc. It is determined, and the content received from the content generating unit 110 is provided for each user device.

For example, when the connection device used by the user is a landline phone capable of only voice conversation, the prepared text content is converted into voice and transmitted, and in the case of a smart device or a PC, voice, text and/or video content is transmitted. Content can be provided in various forms in smart devices/PCs. It can also be configured to transmit text content so that the subject expresses the sentences to be uttered on the screen of the smart device/PC, or to induce speech by transmitting voice or video. have.

The identification of the access device is possible from the method of accessing the MCI detection device and the communication network (mobile communication, public telephone network, Internet network) used, or there is a method of identifying the type of device as informed by the subject using the access device. In the latter case, providing a UI to the connected device and allowing the subject to select the device type using the UI allows the subject to more easily inform the connected device type to the MIC sensing device.

In the foregoing description, for convenience of understanding, the interactive content generating unit 110 and the connecting device determining and content providing unit 115 have been described respectively, but these components 110 and 115 are separate SW modules or separate HW modules. It may be implemented as a device, or it may be implemented as one SW module or HW device.

The consonant-vowel separation unit 130 recognizes the voice information by receiving the subject's speech signal, which is transmitted by converting the subject's voice into an electrical signal in the connection device, and separating the recognized voice information into phoneme units to extract consonants and vowels and classify

The consonant-vowel aligning unit 140 matches the individually extracted consonants and vowels with sound wave values, and measures the utterance time for each phoneme. The measured firing time is transmitted to the feature extraction unit 160 .

In addition, a process of calculating a blank period between each word in the modeled speech sentence may be further performed. At this time, the average of the gap period between words in the entire utterance, the frequency of use of a specific word (eg, inter-projection such as Ah/Um/Jean/Jeong, commonly used words by detection target, etc.), the space time between a specific word and the word following it, etc. can be further extracted. The extracted blank period, frequency of use, etc. may be transmitted to the feature extraction unit 160 and used as features for learning and inference.

In order to extract a feature parameter, the spectrum converter 150 converts the uttered sound wave aligned with consonants into a frequency domain (spectrum) to determine the frequency of each phoneme for each time period. A time zone-frequency conversion graph for each phoneme is shown in FIG. 3 . Here, the frequency is the frequency of the formant that has a great influence on the characteristics of the voice, and f1, f2, f3 or higher formant frequencies are identified.

It is preferable to convert a phoneme for a vowel having a large influence on the tone, but it is not limited thereto, and it is of course possible to convert a consonant or both a vowel and a consonant.

The feature extraction unit 160 extracts a formant value, a pitch value, a decibel value, a density value, and the like from the converted speech spectrum as main features. In addition, for each extracted feature, a differential value that can be distinguished from other tones can be extracted and included in the feature through comparison with a general average value, analysis of the amount of change in values, and analysis of changes in values.

Also, a feature value may be additionally extracted from a phoneme duration and a blank time for each word (ie, a blank time from the moment one word ends to the moment the next word is spoken). The phoneme duration may be received from the consonant-vowel arranging unit 140 or the feature extraction unit 160 may directly extract from the time-frequency conversion graph transmitted from the spectrum converting unit 150 . The blank time for each word may also be extracted and delivered by the consonant-vowel aligning unit 140 or voice information received from the consonant-vowel arranging unit 140 and directly extracted from the feature extracting unit 160 .

The MCI determining unit 170 is composed of an artificial neural network, and in the learning stage, learning is performed using some or all of the above features, and in the inference stage, whether the uttered MCI of the subject is detected based on the learning result.

Learning is basically performed on the basis of each feature data of the speech voice of a normal person and each feature data of the speech voice of an MCI patient.

A learning model is created through supervised learning using the values for a plurality of variables as feature values based on the time axis, and the LSTM-VAE technique, which is one of the deep learning techniques, can be used, but it is not limited thereto, and other techniques can be used. is of course

Inference detects whether the subject has MCI through an inference process based on a learning model created by inputting feature data extracted from the subject's speech voice and learning from the feature data of normal people and MCI patients.

On the other hand, it is also possible to continuously accumulate feature data extracted from the speech voice of the subject in a normal state, and separately configure a learning model for this, and detect whether the subject himself/herself MCI is based on this or by reflecting it in the above-described learning model. Since the speech characteristics are different for each individual, a more accurate reasoning result may be obtained compared to the speech characteristics data of other people whose speech characteristics were in a normal state.

Therefore, whether MCI is detected based on the inference result according to the subject's own steady-state speech characteristics, or the inference result based on the subject's own steady-state speech characteristics and the big data learning model according to the speech characteristics of others. It is also good to detect by merging, but adding more weight to the inference result based on the subject's own speech characteristics, and lowering the weight if the hospital's confirmation result is normal even if the subject's own speech characteristics change afterwards.

As another embodiment of the present invention, as shown in FIG. 4 , the MCI detection apparatus of the present invention further includes a text conversion unit 180 and a natural language processing unit 190 to perform MCI detection in a linguistic aspect in parallel. can do.

To this end, the text conversion unit 180 converts the received subject's utterance into text and transmits it to the natural language processing unit 190 .

The natural language processing unit 190 extracts the characteristics of the speech content, and performs an analysis of the accuracy of words/sentences in response to the subject's response to a question, the type and number of words used, and a semantic relationship analysis of words/sentences, This result is transmitted to the MCI determining unit 170, and the MCI determining unit 170 detects whether the MCI is MCI by synthesizing the linguistic analysis results as well as the speech characteristics.

For the same purpose as described above, it is also good to use the linguistic analysis result of the subject himself, who was in a normal state, as basic data, and detect whether MCI is based on it or by merging it with other people's big data. This is because, even in the normal state, the differences in the accuracy of the words/sentences used by each individual, the analysis of the types and number of words used, and the semantic relationship of the words/sentences are not small.

As another embodiment, as shown in FIG. 5 , the content generated by the content generating unit 110 is delivered to the MCI determining unit 170, and the MCI determining unit analyzes the adequacy relationship of the response to the question and detects the MCI. It is also good to use it additionally for inference for

On the other hand, as shown in FIG. 6 , when the MCI determination unit 170 detects an abnormal symptom, the content generation unit 110 generates or prepares more sophisticated question content for additional determination, and the access device It is preferable to notify or control the determination and content providing unit 120 to provide content for additional determination.

Alternatively, the content generation unit 110 continuously receives the processing result of the natural language processing unit 190 and determines it to generate contents for inducing a target utterance for more precise MCI determination. Although the content generation unit 100 does not have an MCI determination function, it may be implemented to set a predetermined standard in advance, and if it does not meet the criteria, it may be implemented to generate content for a precise conversation.

Compared to the embodiment shown in FIG. 2 , the embodiment shown in FIGS. 4 to 6 may derive a more accurate MCI detection result.

Hereinafter, the MCI sensing method of the present invention will be described with reference to FIG. 7, and the previously described details will be abbreviated or omitted to avoid duplication, so the MCI sensing method according to the present invention is not limited to the following description. It is desirable to supplement the description throughout the specification for understanding.

First, the MCI sensing device recognizes the connection device used by the sensing target (S710), and provides content prepared in advance according to the connection device or generated or selected according to the response of the target. For example, in the case of a smart device including a display device, a speech sentence or a speech guidance question is transmitted and displayed (S720), and in the case of a voice-only device such as a landline phone, it is converted into voice and provided (S730).

When the speaker responds to or talks to the content, this voice information is collected (S740), features are extracted from it (S750), and in the learning step, a learning model is generated by performing learning based on the features in the inference step. Detects whether MCI exists by inferring based on the learning model and the collected voice information (S760).

It is good to periodically provide content, collect uttered responses, and detect MCI. Since the present invention is a conversation-based sensing method, it is possible to periodically call the sensing subject and listen to the voice while performing the care service, and identify whether the sensing subject is normal, mild, or impaired at an early stage. In addition, the MCI determination unit 170 may be implemented to further include the step of early determination of various mental symptoms, such as the subject's psychology and depressive symptoms.

In particular, in order to arouse the interest of the subject, the conversational context that can lead the user's conversation is secured and the conversational scenario technology that can automatically provide a dialogue for each situation is reflected to periodically collect the subject's spoken voice while interacting It is preferable to do

In addition, the step of continuously collecting the speech characteristics of the subject, and using the LSTM-VAE technique, etc., based on the collected data, comparing with the previous characteristics of the same subject, analyzing, and statistically determining whether or not MCI is further included can do.

On the other hand, in the present specification, the MCI determination unit 170 is not limited to determining only MCI despite the term, but is an artificial intelligence module that also determines dementia (AD), psychological states or pathological signs.

In addition, the MCI sensing device according to the present invention is preferably implemented to function as an interactive artificial intelligence device that accumulates medical knowledge or other knowledge and responds to a question of a sensing target. It is possible to increase the detection accuracy by extracting speech characteristics while eliciting as many conversations as possible and detecting MCI based on this, or by performing MCI detection in parallel with speech characteristics analysis and linguistic analysis. can be encouraged to participate.

In addition, the MCI detection method according to the present invention performs a procedure of notifying a result of the detection of an MCI symptom and recommending the detection subject to go to a hospital for diagnosis.

In addition, the step of making a reservation for diagnosis to a pre-contracted hospital and notifying the detection target of the reservation result may be additionally performed.

The configuration of the present invention has been described in detail with reference to several preferred embodiments, but these are only examples for helping understanding, and the protection scope of the present invention is not limited to the configuration of each embodiment. It goes without saying that various modifications and variations of the above-described embodiments are possible within the scope of the technical spirit of the present invention. For example, the MCI sensing apparatus of the present invention may be implemented in one processor, or may be implemented in two or more different processors.

Accordingly, the protection scope of the present invention should be defined by the description of the following claims.

Claims (12)

A summary mild cognitive impairment (MCI) detection device, comprising:
A content generating unit that generates content to induce the detection target's utterance;
A connection device determination and content providing unit that determines the connection device used by the sensing target and provides the content accordingly;
a consonant separation unit for recognizing voice information from the received speech signal of the detected subject and extracting consonants and vowels;
A consonant alignment unit for aligning the separated consonants,
A spectrum converter that converts consonant-arranged utterances into a frequency domain;
a feature extracting unit receiving the spectrum from the spectrum converting unit and extracting at least one speech feature;
Comprising an MCI determining unit for detecting whether or not MCI based on the at least one speech characteristic,
The feature extraction unit performs a process of extracting a differential value that can be distinguished from other tones through analysis of changes in values for each extracted utterance feature value. According to claim 1, wherein the content generating unit,
MCI sensing device for generating at least one of speech-induced content that induces a specific pronunciation, short-answer-type content for judging intellectual ability, and content-guided content that induces a content response to a specific topic or current issue.
According to claim 1, wherein the content generating unit,
MCI detection device further comprising an artificial intelligence chatbot.
According to claim 1, wherein the connection device determination and content providing unit,
If the connection device used by the user is a landline phone capable of only voice conversation, it converts the prepared text content into voice and transmits it, and if it is a smart device with a display device, the MCI detection device transmits voice and video content.
According to claim 1, wherein the consonant aligning unit,
MCI sensing device to measure the utterance time for each phoneme and transmit it to the feature extraction unit. According to claim 1, wherein the consonant aligning unit,
MCI sensing device for extracting at least one of the blank period between each word in the modeled speech sentence, the average of the blank period between words in the entire utterance, the frequency of use of a specific word, and the blank time between a specific word and a word following it
According to claim 1, wherein the feature extraction unit,
An MCI sensing device for extracting at least one of a formant value, a pitch value, a decibel value, and a density value as a feature value.
The method of claim 7, wherein the feature extraction unit,
The MCI sensing device further performs a process of extracting a differential value that can be distinguished from other tones through comparison with a general average value for each extracted feature value and analysis of the amount of change in the values.
The method of claim 1, wherein the MCI determining unit,
generating a learning model through supervised learning based on the at least one speech feature in the learning step;
MCI sensing apparatus that detects whether MCI is based on the at least one speech feature and the learning model in the inference step.
An MCI detection method by an MCI detection device, comprising:
A step of providing, by a content providing unit, content for inducing utterance;
extracting, by a feature extraction unit, at least one feature from a speech response to the content;
generating, by the MCI determining unit, a learning model by performing learning based on the at least one characteristic;
Comprising the step of detecting, by the MCI determination unit, whether the detection target is MCI based on the learning model and the speech characteristics of the detection target,
The step of extracting the feature is
Comprising the step of performing, by the feature extraction unit, a process of extracting a differential value that can be distinguished from other tones through analysis of changes in values for each extracted utterance feature value
MCI detection method.
11. The method of claim 10, If the sensing step is detected as MCI,
The MCI detection device notifies the detection target of the result and provides a guide for recommending a hospital diagnosis and
Make a reservation for a diagnosis at the pre-contracted hospital and notify the detection target of the reservation result
MCI detection method further comprising the step of performing at least one of.
The method of claim 10, wherein generating the learning model comprises:
The MCI detection method comprising the step of generating a learning model based on the self-response data constructed by the MCI determining unit continuously accumulating the conversation contents of the sensing subject for each response.

Images