KR102713521B1 - Hearing aid using artificial intelligiece - Google Patents

Abstract

According to the present invention, a hearing assistance device is provided, which comprises two earplug bodies. Each of the earplug bodies comprises a microphone for converting sound waves of ambient sound into electrical signals to generate ambient sound signals, a speaker for converting the electrical signals of sound into sound waves, and a control unit for processing the ambient sound signals to generate the electrical signals of sound. The control unit comprises an artificial intelligence sound analysis module for performing artificial intelligence analysis on the ambient sound signals to distinguish and separate voice signals from non-voice signals that are not voice signals, and an output sound signal generation module for generating an output sound signal that increases the volume of the voice signal output by being separated from the artificial intelligence sound analysis module and decreases the volume of the non-voice signal, and transmitting the generated output sound signal to the speaker.

Description

HEARING AID USING ARTIFICIAL INTELLIGIECE

The present invention relates to a hearing assistance device that can be effectively used in noisy environments such as dental clinics.

When treating patients, creating a comfortable and relaxing environment is of utmost importance. Depending on how the disease is treated, a quiet environment can be provided to the patient, but in the dental field, sharp and high-frequency noises generated during treatment can cause discomfort or fear in the patient. However, there is insufficient protection against such high-frequency noises.

In addition to the medical field such as dentistry, the importance of hearing protection for workers is increasing in noisy industrial fields such as military bases and construction sites. To this end, workers currently mainly use shielding earplugs, but these block not only noise but also alarm sounds within the facility and the voices of other workers, causing problems in communication and even causing safety accidents. This problem is especially serious for the hearing impaired.

Meanwhile, ACTIVE NOISE CONTROL technology is a technology that receives surrounding noise through a microphone attached separately to headphones or earphones, and generates destructive interference to cancel out the noise in the noise-canceling circuit to block out the noise. Previously, ACTIVE NOISE CONTROL technology was installed in headphones or earphones and used only for multimedia enjoyment, and there has been little development of technology for diverse applications in other environments.

Republic of Korea Patent Registration No. 10-1739441 (May 24, 2017)

The purpose of the present invention is to provide a hearing assistance device that enables conversation while eliminating noise in noisy environments, such as in the medical field such as dentistry and surgery, and in industrial fields such as military units and construction sites.

In order to achieve the above object of the present invention, according to one aspect of the present invention, there is provided a hearing assistance device including two earplug bodies, each of the two earplug bodies having a microphone for converting sound waves of ambient sound into electrical signals to generate ambient sound signals, a speaker for converting the electrical signals of sound into sound waves, and a control unit for processing the ambient sound signals to generate the electrical signals of sound, wherein the control unit has an artificial intelligence sound analysis module for performing artificial intelligence analysis on the ambient sound signals to distinguish and separate voice signals from non-voice signals that are not voice signals, and an output sound signal generation module for generating an output sound signal by increasing the volume of the voice signal and lowering the volume of the non-voice signal that is separated and output from the artificial intelligence sound analysis module and transmitting the output sound signal to the speaker.

According to the present invention, all of the above-described purposes of the present invention can be achieved. Specifically, by using artificial intelligence sound analysis to separate voice signals and non-voice signals from ambient sound signals and processing them separately, the volume of the voice signal is increased and the volume of the non-voice signal is decreased, thereby making conversation possible even in noisy situations such as dental clinics.

FIG. 1 is a perspective view illustrating a hearing assistance device according to one embodiment of the present invention.
Figure 2 is an exploded perspective view of the hearing aid device illustrated in Figure 1.
FIG. 3 is a block diagram showing the configuration of the earplug body in the hearing aid device illustrated in FIGS. 1 and 2.
Figure 4 is a block diagram showing the configuration of the control unit illustrated in Figure 3.
FIG. 5 is a diagram showing three operating modes of the hearing aid device illustrated in FIGS. 1 and 2.

Hereinafter, the configuration and operation of an embodiment of the present invention will be described in detail with reference to the drawings.

A hearing assistance device according to one embodiment of the present invention is illustrated in a perspective view in FIG. 1 and in an exploded perspective view in FIG. 2. Referring to FIGS. 1 and 2, a hearing assistance device (100) according to one embodiment of the present invention includes two earplugs (110) and a necklace means (190) for connecting the two earplugs (110) so that the two earplugs can be hung around the user's neck. The hearing assistance device (100) operates to intensively remove ambient noise, facilitate conversation in noisy situations, or enable hearing sound information provided by an external device.

Each of the two earplugs (110) is used by being inserted into each of the user's two ears. Each of the two earplugs (110) has an earplug body (120) and an earcap (180) that is detachably connected to the earplug body (120).

The earplug body (120) is provided with an extension part (121) that is extended in a columnar shape. A necklace means (190) is connected to the extension part (121). Although not shown, the earplug body (120) is provided with a secondary battery that supplies the power required for operation and is rechargeable. Charging of the secondary battery provided in the earplug body (120) is performed in a state where the necklace means (190) is removed.

FIG. 3 illustrates a block diagram of the configuration of an earplug body (120). Referring to FIG. 3, the earplug (120) comprises an operating unit (122) for selecting an operation mode of a hearing aid device (100 of FIGS. 1 and 2), a microphone (130) for converting sound waves of ambient sound into electrical signals to generate ambient sound signals, a speaker (140) for converting electrical signals of sound into sound waves so that a user can hear them with his or her ears, a communication unit (145) for communicating with an external device (A), and a control unit (150) for controlling the overall operation of the earplug body (120).

The operating unit (122) is for operating the hearing aid device (100 of FIGS. 1 and 2), and the user can select the operation mode of the hearing aid device (100 of FIGS. 1 and 2) by using the operating unit (122). In this embodiment, the operating unit (122) is described as being formed in the form of a button on the outside of the ear plug body (120) as shown in FIGS. 1 and 2. The operating mode selection signal by the operating unit (122) is transmitted to the control unit (150).

The microphone (130) converts sound waves of ambient sound into electrical signals to generate ambient sound signals. The microphone (130) includes a configuration of a typical microphone. The ambient sound can be divided into voices, which are human voices, and non-voices, which are not voices. The ambient sound can include noise. The ambient sound signals generated by the microphone (130) are transmitted to the control unit (150).

The speaker (140) converts the electrical signal of sound transmitted from the control unit (150) into sound waves so that the user can hear it with his ears. The speaker (140) includes a configuration of a typical speaker.

The communication unit (145) connects the hearing aid device (100 of FIGS. 1 and 2) to an external device (A) such as a smartphone so that it can communicate with it. The communication unit (145) exchanges control signals and data with the control unit (150). The communication unit (145) can receive sound data such as a sound source from the external device (A). In this embodiment, it is described that the communication unit (145) uses a short-range wireless communication technology such as Bluetooth.

The control unit (150) controls the overall operation of the earplug body (120). FIG. 4 illustrates the configuration of the control unit (150) as a block diagram. Referring to FIG. 4, the control unit (150) includes an operation mode confirmation unit (152) that confirms the operation mode of the hearing aid device (100 of FIGS. 1 and 2), a noise removal module (155) that removes noise from an ambient sound signal output from a microphone (130 of FIG. 3), an artificial intelligence sound analysis module (160) that performs artificial intelligence analysis on an ambient sound signal output from a microphone (130 of FIG. 3), and an output sound signal generation module (165) that generates an output sound signal by adjusting the volume of a voice signal and the volume of a noise signal that are separated and output from the artificial intelligence sound analysis module (160), respectively.

The operation mode confirmation unit (152) confirms the operation mode of the hearing aid device (100 of FIGS. 1 and 2). The operation mode confirmation unit (152) confirms the operation mode selection signal output by the operating unit (122 of FIG. 3) to confirm the operation mode of the hearing aid device (100 of FIGS. 1 and 2). FIG. 5 shows three operation modes of the hearing aid device (100 of FIGS. 1 and 2). Referring to FIG. 5, the hearing aid device (100 of FIGS. 1 and 2) operates in one of three operation modes: a focused noise cancellation mode (M1), a conversation support mode (M2), and an external device connection mode (M3). The focused noise cancellation mode (M1) and the conversation support mode (M2) can be switched and changed. External device connection mode (M3) uses a Bluetooth connection, and either the focused noise cancellation mode (M1) or the conversation focus mode (M2) can operate within the external device connection mode (M3).

In the focused noise cancellation mode (M1), the hearing aid device (100 in FIGS. 1 and 2) operates to remove noise from the surrounding sounds.

In conversation support mode (M2), the hearing assistance device (100 in FIGS. 1 and 2) operates to enable smooth conversation in noisy situations.

In external device connection mode (M3), the hearing aid device (100 in FIGS. 1 and 2) operates to output sound data transmitted from an external device (A).

Referring to FIG. 4, the noise removal module (155) removes noise from an ambient sound signal output from a microphone (130 of FIG. 3) to generate a noise removal sound signal. The noise removal sound signal generated by the noise removal module (155) is transmitted to a speaker (140 of FIG. 3) and converted into a sound wave and output. The noise removal module (155) can use a low pass filter (LPF) or a notch filter. The low pass filter (LPF) passes a low frequency band in an ambient sound signal and blocks a high frequency band, thereby reducing or blocking high frequency noise distributed in a relatively high frequency band. The notch filter is a filter that removes only components of a specific frequency band. The frequency band that the noise removal module (155) removes can be set differently depending on the environment in which the hearing aid device (100 of FIGS. 1 and 2) is used. When the hearing aid device (100 of FIGS. 1 and 2) is used in dentistry, it is preferable that the noise removal filter module (155) be set to block a high frequency band of about 3500 Hz or higher. The noise removal module (155) can function when the hearing aid device (100 of FIGS. 1 and 2) operates in a focused noise removal mode (M1 of FIG. 5).

The artificial intelligence sound analysis module (160) performs artificial intelligence analysis on the ambient sound signal output from the microphone (130 of FIG. 3) to distinguish and separate voice signals from non-voice signals that are not voice signals. In the artificial intelligence sound analysis module (160), non-voice signals excluding voice signals are recognized as noise signals. The voice signals and noise signals separated by the artificial intelligence sound analysis module (160) are transmitted to the output sound signal generation module (165). The artificial intelligence sound analysis module (160) is equipped with an artificial intelligence sound analysis model (162) that performs artificial intelligence analysis on the ambient sound signal to separate voice signals from noise signals. The artificial intelligence sound analysis module (160) can function when the hearing assistance device (100 of FIGS. 1 and 2) operates in the focused noise removal mode (M1 of FIG. 5) and the conversation support mode (M2 of FIG. 5).

The artificial intelligence sound analysis model (162) is created by deep learning on the collected sound data. The learning process is explained as follows using a dental case as an example.

First, as a first step, voice data is collected by dental hospital treatment type/sound generation type (suction, handpiece sound, baby crying sound, etc.)/department (conservative dentistry, pediatric dentistry, and other treatment environments) and pre-classification and tagging are performed (step 1).

Next, as the second step, only the voices of medical staff are classified from the actual field data where the collected voice + sound data are mixed. At this time, the method of tagging the voices of medical staff by distinguishing the start and end points on the timeline or by distinguishing by frequency can be used. Here, in the case of classification of the voices of medical staff, after distinguishing whether it is the voice of medical staff or not, the voices of medical staff are converted into text (sound to text), and the process of selecting and organizing them by sentence and tagging them is carried out. In the case of classification by noise, after distinguishing whether it is noise or not, the process of determining the type of dental noise and tagging it is carried out.

Next, as the third step, a deep learning learning algorithm is generated based on the tagged data. Specifically, an environmental noise classification algorithm based on a convolutional neural network (CNN) is generated to improve speech discrimination and intelligibility in a noisy environment. The Hz over time is imaged as an input waveform, and the speech waveform and noise waveform are classified through this. Through this, it is used for the following two functions.

First, it is used to develop an algorithm that can classify the voices of medical staff and various noises of dental treatment.

Next, it can be utilized as a key solution for voice activity detection.

It is desirable to use a residual convolutional neural network (Residual CNN). The residual learning method effectively solves the problem of increasing the amount of calculation and vanishing/explosion of gradients by adding residual connections to a specific layer, thereby showing excellent performance even in deep neural network structures. Accordingly, a method of identifying noise by applying a residual convolutional neural network that combines a synthetic convolutional neural network and a residual learning method is used.

The output sound signal generation module (165) is separated from the artificial intelligence sound analysis module (160) and generates an output sound signal by adjusting the volume of the output voice signal and the volume of the noise signal, respectively. The output sound signal generation module (165) increases the volume of the voice signal and decreases the volume of the noise signal and outputs it. The output sound signal generated by the output sound signal generation module (165) is transmitted to a speaker (140 of FIG. 3) and converted into a sound wave and output. The artificial intelligence sound analysis module (160) can function when the hearing assistance device (100 of FIGS. 1 and 2) operates in a conversation support mode (M2 of FIG. 5). Accordingly, even in a noisy surrounding environment, the noise is heard small and the voice is heard large, so that a conversation can be conducted smoothly.

Referring to FIGS. 1 and 2, the earcap (180) is detachably connected to the earplug body (120). The earcap (180) can be replaced with various sizes to suit the user's characteristics and used. The earcap (180) can be formed in a configuration that can be commonly used in earphones.

The necklace means (190) connects two earplugs (110) so that they can be hung around the user's neck. The necklace means (190) has two connecting parts (192) that are connected to the earplug body (120) and a connecting line (196) that connects the two connecting parts (192).

Each of the two connecting parts (192) is detachably connected to each of the two earplug bodies (120). The connecting part (192) is generally in the shape of a ring and is connected to the earplug body (120) by wrapping around the extension part (121) of the earplug body (120). The connecting part (192) is made of an elastic material.

A connecting line (196) connects two connecting parts (192). The connecting line (196) is hung around the user's neck.

Now, referring to FIGS. 1 to 5, the operation of the hearing aid device (100) according to the operating mode will be described. The hearing aid device (100) operates in one of three operating modes: a focused noise cancellation mode (M1), a conversation support mode (M2), and an external device connection mode (M3).

First, the focused noise removal mode (M1) will be described. The focused noise removal mode (M1) is performed by the operation mode confirmation unit (152) confirming the focused noise removal mode selection signal output by the operation unit (122) operated by the user. In the focused noise removal mode (M1), the noise removal module (155) operates to block a set frequency band, thereby performing the function. In addition, in the focused noise removal mode (M1), the artificial intelligence sound analysis module (160) and the output sound signal generation module (165) also operate together. The focused noise removal mode (M1) is very severe in ambient noise, and is useful when noise needs to be blocked intensively.

Next, the conversation support mode (M2) is described. The conversation support mode (M2) is performed by the operation mode confirmation unit (152) confirming the conversation support mode selection signal output by the operation unit (122) operated by the user. In the conversation support mode (M2), the artificial intelligence sound analysis module (160) operates to separate the voice signal and the noise signal from the surrounding sound signal, and the output sound signal generation module (165) operates to increase the volume of the voice signal and decrease the volume of the noise signal. The conversation support mode (M2) is useful when a smooth conversation is required in a noisy surrounding environment.

Next, the external device connection mode (M3) will be described. The external device connection mode (M3) is performed by the operation mode confirmation unit (152) confirming the external device connection mode selection signal output by the operation unit (122) operated by the user. The external device connection mode (M3) is performed by the communication unit (145) connecting the external device (A) so that it can be communicated. In the external device connection mode (M3), sound data such as a sound source is received from the external device (A) and output through the speaker (140). The external device connection mode (M3) is useful when the user wants to listen to music to concentrate on work or to achieve psychological stability during dental treatment. In addition, the focused noise removal mode (M1) and the conversation support mode (M2) can be used together in the external device connection mode (M3). For the psychological stability of the user, it is preferable that the external device connection mode (M3) and the focused noise removal mode (M1) are used together. For example, in a dental environment, listening to music in a quiet environment can reduce dental phobia, and in an industrial/work environment, it can reduce perceived noise.

Although the present invention has been described through the above examples, the present invention is not limited thereto. The above examples may be modified or changed without departing from the spirit and scope of the present invention, and those skilled in the art will recognize that such modifications and changes also belong to the present invention.

100: Hearing aid device 110: Ear plug
120: Ear plug body 121: Extension part
122: Control panel 130: Microphone
140: Speaker 150: Control Unit
152: Operation mode check unit 155: Noise reduction module
160: AI Sound Analysis Module 162: AI Sound Analysis Model
165: Output signal generation module 170: Communication unit
180: Ear cap 190: Necklace means
192: Joint 196: Connecting line

Claims (10)

Two earplug bodies; and
Includes a necklace means for connecting the two earplug bodies so that they can be worn around the user's neck,
Each of the above two earplug bodies has a microphone that converts sound waves of ambient sound into electrical signals to generate ambient sound signals, a speaker that converts electrical signals of sound into sound waves, a control unit that allows the user to select an operating mode, a communication unit for communicating with an external device, and a control unit that processes the ambient sound signals to generate electrical signals of sound.
The control unit comprises an operation mode confirmation unit that confirms an operation mode selection signal output by the operation unit, a noise removal module that removes noise from the ambient sound signal to generate a noise removal signal and transmits the noise removal signal to the speaker, an artificial intelligence sound analysis module that performs artificial intelligence analysis on the ambient sound signal to distinguish and separate a voice signal from a non-voice signal that is not a voice signal, and an output sound signal generation module that generates an output sound signal that increases the volume of the voice signal output by being separated from the artificial intelligence sound analysis module and decreases the volume of the non-voice signal and transmits the output sound signal to the speaker.
Depending on the above operation mode selection signal, the selected mode among the focused noise cancellation mode, conversation support mode, and external device connection mode operates.
In the above-mentioned focused noise removal mode, the noise removal module, the artificial intelligence sound analysis module and the output sound signal generation module operate.
In the above conversation support mode, the artificial intelligence sound analysis module and the output sound signal generation module operate.
In the above external device connection mode, the communication unit operates to receive sound from the external device and output it through the speaker.
In the above external device connection mode, either the focused noise cancellation mode or the conversation support mode can be additionally selected and operate together.
The above-mentioned focused noise cancellation mode and the above-mentioned conversation support mode are switched and changed by the above-mentioned operating unit.
The above necklace means comprises two connecting parts that are detachably connected to each of the two earplug bodies, and a connecting string connecting the two connecting parts.
Hearing assistance devices. In claim 1,
The above artificial intelligence sound analysis module is provided with an artificial intelligence sound analysis model that is generated by deep learning and separates speech signals and non-speech signals from the surrounding sound signals.
Hearing assistance devices. In claim 2,
The above deep learning is performed based on a residual convolutional neural network.
Hearing assistance devices. delete In claim 1,
The above noise removal module comprises a low-pass filter or a notch filter.
Hearing assistance devices. delete delete delete delete In claim 1,
Further comprising an ear cap detachably connected to the earplug body.
Hearing assistance devices.

Images